Big Data 平障录

zhixingheyi_tian2024-05-08 9:20

Hive

Hive 生成带压缩的格式，需要如此设置

复制代码

SET parquet.compression=SNAPPY;

yarn.scheduler.fair.assignmultiple

yarn.scheduler.fair.assignmultiple是YARN Fair Scheduler的一个配置参数.

复制代码

Whether to allow multiple container assignments in one heartbeat. Defaults to false.

相关代码位置：

org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler

复制代码

assignMultiple = this.conf.getAssignMultiple();

org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairSchedulerConfiguration

复制代码

  public boolean getAssignMultiple() {
    return getBoolean(ASSIGN_MULTIPLE, DEFAULT_ASSIGN_MULTIPLE);
  }

private static final String CONF_PREFIX =  "yarn.scheduler.fair.";

  /** Whether to assign multiple containers in one check-in. */
  public static final String  ASSIGN_MULTIPLE = CONF_PREFIX + "assignmultiple";
  protected static final boolean DEFAULT_ASSIGN_MULTIPLE = false;