springboot远程链接Hadoop

2.3、springboot远程链接Hadoop

1、依赖

<!--        Hadoop依赖-->
        <!-- Hadoop Client -->
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>3.3.1</version>
        </dependency>

2、yaml文件

#HDFS配置
hdfs:
  path: hdfs://192.168.44.128:9000
  user: hadoop

3、安装配置winutils.exe

• https://gitee.com/bochangguan/winutils/tree/master

• 配置系统环境变量：HADOOP_HOME，和path

• 

• path

  • 

4、示例代码

• controller层

  @RestController
  @RequestMapping("/test/hdfs")
  public class HdfsController {
   
      @Autowired
      private HdfsService service;
   
   
      @GetMapping("/mkdir")
      public String mkdir(String path){
          try {
              service.mkdir(path);
              return "1";
          } catch (Exception e) {
              System.out.print(e.toString());
              return "2";
          }
      }
  }

• service层

      public boolean mkdir(String dir) throws Exception{
          if(StringUtils.isBlank(dir)){
              return false;
          }
          if(exist(dir)){
              return true;
          }
          FileSystem fileSystem = getFileSystem();
          boolean isOk = fileSystem.mkdirs(new Path(dir));
          fileSystem.close();
          return isOk;
      }