Spark 之 HistoryServer and FsHistoryProvider

HistoryServer

a inner join with one empty side and one large side, the plan would finish and the large side is still running.

• Use SparkListenerSQLExecutionEnd to track final SQL status in UI
  https://issues.apache.org/jira/browse/SPARK-40834

Spark HistoryServer 3.4 以后修复了这个问题

JsonProtocol

JsonProtocol 类为 HistoryServer 提供解析 eventLog 的基本工具类库。

other 选项为其提供了 基本反射功能，可以解决各类 event 事件

FsHistoryProvider

  override def start(): Unit = {
    initThread = initialize()
  }

  private[history] def initialize(): Thread = {
    if (!isFsInSafeMode()) {
      startPolling()
      null
    } else {
      startSafeModeCheckThread(None)
    }
  }