AWS sdk for s3 - S3 client

rookiesx2024-12-13 13:02

背景

在产品环境上通过 http 的方式访问 aws s3 是不安全的，需要使用aws sdk 提供的接口来访问

技术实现

项目中使用的是java

1. 在gradel 中引用对应的aws 包

implementation 'software.amazon.awssdk:s3:2.20.80' // aws sdk

implementation 'software.amazon.awssdk:sts:2.20.0'

2. 在gradel 中引用对应的aws 包

需要使用S3 Client 来向aws s3 发起请求，为了避免重复创建Client 冗余对象，创建工厂类来管理对应的 S3 Client 对象。扩展性也大幅提高。

此外，由于每一个Client 对象是一个单一的Region，需要不同的aws cluster_region 。

java 复制代码

public class S3ClientFactory {
    private static final Map<String, S3Client> clients = new ConcurrentHashMap<>();

    public static S3Client getClient(Region region) {
        return clients.computeIfAbsent(region.toString(), r -> S3Client.builder()
                .region(Region.of(r))
                .build());
    }

    public static String getCsvContent(String bucketName, String objectKey, String configKey) {
        // Create S3 Clients
        S3Client s3Stg = S3ClientFactory.getClient(Region.US_WEST_2);    // stage
        S3Client s3ProdNa = S3ClientFactory.getClient(Region.US_EAST_1); // prod NA

        // Select the appropriate S3 client based on configKey
        S3Client s3 = selectClient(configKey, s3Stg, s3ProdNa);

        // Prepare the GetObject request
        GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                .bucket(bucketName)
                .key(objectKey)
                .build();

        // Fetch the object and read the content into a String
        try (ResponseInputStream<?> response = s3.getObject(getObjectRequest);
             BufferedReader reader = new BufferedReader(new InputStreamReader(response))) {

            // Collect all lines into a single string
            return reader.lines().collect(Collectors.joining("\n"));

        } catch (S3Exception e) {
            throw new CustomException("Failed to get AWS CSV by bucket and objectKey: " + e.awsErrorDetails().errorMessage(), e.getMessage());
        } catch (IOException e) {
            throw new CustomException("Error processing CSV content from AWS S3: " + e.getMessage(), e.getMessage());
        }
    }

    private static S3Client selectClient(String configKey, S3Client s3Stg, S3Client s3ProdNa) {
        if (configKey.contains("prod")) {
            return s3ProdNa; // Assuming s3ProdNa is used for NA and SA
        } else {
            return s3Stg; // Default to stage
        }
    }
}

展望

搞清楚 aws sdk 权限，access_api key，访问通信原理