Mapreduce 之 nativetask

hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/NativeTask.h

cpp 复制代码
#define NATIVE_SORT_TYPE "native.sort.type"
#define MAPRED_SORT_AVOID "mapreduce.sort.avoidance"
#define NATIVE_SORT_MAX_BLOCK_SIZE "native.sort.blocksize.max"
#define MAPRED_COMPRESS_MAP_OUTPUT "mapreduce.map.output.compress"
#define MAPRED_MAP_OUTPUT_COMPRESSION_CODEC "mapreduce.map.output.compress.codec"
#define MAPRED_MAPOUTPUT_KEY_CLASS "mapreduce.map.output.key.class"
#define MAPRED_OUTPUT_KEY_CLASS "mapreduce.job.output.key.class"
#define MAPRED_MAPOUTPUT_VALUE_CLASS "mapreduce.map.output.value.class"
#define MAPRED_OUTPUT_VALUE_CLASS "mapreduce.job.output.value.class"
#define MAPRED_IO_SORT_MB "mapreduce.task.io.sort.mb"
#define MAPRED_NUM_REDUCES "mapreduce.job.reduces"
#define MAPRED_COMBINE_CLASS_OLD "mapred.combiner.class"
#define MAPRED_COMBINE_CLASS_NEW "mapreduce.job.combine.class"

#define NATIVE_LOG_DEVICE "native.log.device"

/dev/shm/yarn/log/application_1785138663180_0056/container_1785138663180_0056_01_000003

cpp 复制代码
2026-07-28 14:00:01,533 INFO [main] org.apache.hadoop.mapred.nativetask.NativeBatchProcessor: NativeHandler: direct buffer size: 1048576
2026-07-28 14:00:01,541 INFO [main] org.apache.hadoop.mapred.nativetask.util.OutputUtil: nativetask.output.manager = org.apache.hadoop.mapred.nativetask.util.NativeTaskOutputFiles
2026-07-28 14:00:01,542 INFO [main] org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator: Native output collector can be successfully enabled!
2026-07-28 14:00:01,542 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator
2026-07-28 14:00:05,032 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
2026-07-28 14:00:07,532 WARN [Thread-12] mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
2026-07-28 14:00:12,619 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1785138663180_0056_m_000001_0 is done. And is in the process of committing
2026-07-28 14:00:12,701 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1785138663180_0056_m_000001_0' done.
2026-07-28 14:00:12,705 INFO [main] org.apache.hadoop.mapred.Task: Final Counters for attempt_1785138663180_0056_m_000001_0: Counters: 29
相关推荐
远航计算机2 小时前
职业技能培训机构如何利用QClaw+Skills做豆包GEO:知识库搭建×内容创作×效果监测的完整实操手册
大数据·人工智能·自动化·aigc·火山引擎
IanSkunk3 小时前
视光中心承接近视防控需求,核心是先把服务流程做成标准化工程
大数据
SEO_juper3 小时前
用Google Search Console数据做内容审计:找出网站上哪些页面在“吃白饭“
大数据·人工智能·外贸独立站
独行侠影a5 小时前
Mojo:专为AI而生的“Python++”,能否真正挑战CUDA与C++的统治地位?
大数据·人工智能·深度学习
greenbbLV5 小时前
中小公司积分商城选型:SaaS与私有化方案对比解析
大数据·前端·小程序
半兽先生6 小时前
大模型技术开发与应用——5.大模型Agent开发(CrewAI)
大数据·人工智能·python·机器学习·ai
空杆推不起6 小时前
穿透 Flink CDC 表层用法:数据库日志捕获机制、Flink Source 运行时、端到端一致性底层原理详解
大数据·数据库·flink
跨境生态圈7 小时前
2026跨境出海服务商深度测评:凌赟科技如何用“SEO+GEO双引擎”重构AI时代获客逻辑?
大数据·运维·人工智能·科技·chatgpt
小邓的技术笔记7 小时前
DeepSeek 大模型落地应用与场景实战指南
大数据·人工智能