springboot远程链接Hadoop

念言-ny2025-01-06 16:58

2.3、springboot远程链接Hadoop

1、依赖

xml 复制代码

<!--        Hadoop依赖-->
        <!-- Hadoop Client -->
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>3.3.1</version>
        </dependency>

2、yaml文件

yaml 复制代码

#HDFS配置
hdfs:
  path: hdfs://192.168.44.128:9000
  user: hadoop

3、安装配置winutils.exe

https://gitee.com/bochangguan/winutils/tree/master
配置系统环境变量：HADOOP_HOME，和path
path

4、示例代码

controller层

java 复制代码

@RestController
@RequestMapping("/test/hdfs")
public class HdfsController {
 
    @Autowired
    private HdfsService service;
 
 
    @GetMapping("/mkdir")
    public String mkdir(String path){
        try {
            service.mkdir(path);
            return "1";
        } catch (Exception e) {
            System.out.print(e.toString());
            return "2";
        }
    }
}

service层

java 复制代码

    public boolean mkdir(String dir) throws Exception{
        if(StringUtils.isBlank(dir)){
            return false;
        }
        if(exist(dir)){
            return true;
        }
        FileSystem fileSystem = getFileSystem();
        boolean isOk = fileSystem.mkdirs(new Path(dir));
        fileSystem.close();
        return isOk;
    }