Hadoop 之 Uber 模式

zhixingheyi_tian2025-12-09 9:54

参考链接

https://blog.csdn.net/beliefer/article/details/51160494/

mapper 端任务报错

复制代码

org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find output/file.out in any of the configured local directories
	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:509)
	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:179)
	at org.apache.hadoop.mapred.MROutputFiles.getOutputFile(MROutputFiles.java:57)
	at org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:1365)
	at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:1345)
	at org.apache.hadoop.mapred.Task.done(Task.java:1273)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:352)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:442)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:356)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:217)
	at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:260)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

具体代码在这里

src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java

复制代码

      //no path found
      throw new DiskErrorException ("Could not find " + pathStr +" in any of" +
      " the configured local directories");