S3部署oss前端静态资源

什么是oss和s3？为什么会用到oss和s3？

对象存储（OSS，Object Storage Service）是一种云端数据存储服务，一般的云服务厂商（阿里云，京东云...）都会提供对应的oss存储服务。而s3一般是指Amazon S3(Simple Storage Service)对象存储，它出现得比较早且有一套简单的RESTful API，于是成为了oss业内的标准接口规范。

至于为什么要用？先简单说下以前的部署流程吧：前端这里单独申请一台服务器，线上域名挂到前端服务器上，通过nginx访问后端服务器的接口，这样就会导致测试和线上环境都会多占用几台服务器专门搞前端的静态资源，所以为了省成本就考虑把静态资源放到oss，然后通过后端来处理去加载最新的资源文件返回。

该怎么做？

我们这里打算要用到的是京东云，但是发现网上似乎没有现成的前端部署京东云oss的插件，而京东云推荐我们直接调用aws的sdk，而sdk的文档内容太多了，我们需要的只有一个在项目在构建之后，把构建产物部署到oss服务器上就可以啦。那接下来我们就开始搞一个吧

开搞开搞！

既然是个上传的插件，那就应该和项目的构建工具无关，我们可以先封装一个deploy方法，然后套一个vite plugin 和 webpack plugin的壳子，这样不管什么样式就可以用到啦！

1. 先把准备工作搞下，既然要多发多个包的话，就必须要用下pnpm workspace了，目录结构如下

// deploy函数
const deploy = () => {
    // 做一些上传的操作
}
export default deploy

// webpack插件
import deploy from "deploy-oss";
export class DeployWebpackPlugin {
    private options
    constructor(config) {
        this.options = config;
    }
    apply(compiler: any) {
        const afterEmit = (compilation, callback) => {
            deploy()
        };

        if (compiler.hooks) {
            compiler.hooks.afterEmit.tapAsync("DeployOssPlugin", afterEmit);
        } else {
            compiler.plugin("DeployOssPlugin", afterEmit);
        }
    }
}

// vite 插件
import deploy from "deploy-oss";
export  function DeployWebpackPlugin(options)  {
    let buildConfig: any = {};
    return {
      name: "deploy-oss-plugin",
      enforce: "post",
      apply: "build",
      configResolved(config: any) {
        buildConfig = config.build;
      },
      async closeBundle() {
        deploy()
      },
    };
  }

2. 看下官方文档，我们都需要哪些参数？
   docs.jdcloud.com/cn/object-s...
   docs.aws.amazon.com/AWSJavaScri...

3. 根据上面文档我们可以知道我们需要的参数是什么？（oss的密钥，bucket名称，文件存储的地址，还有文件数据流，还有oss服务器地址，本文默认用的京东云的）

// 先定义下参数类型
export interface OptionInterface {
    pathName: string,  // 文件的路径
    bucketName: string, // bucket名称
    accessKey: string, // 密钥对
    secretKey: string, // 密钥对
    endpoint?:string // 服务器地址
}

// outDir 打包文件的地址，方便后面去遍历找文件
export type deployType=(outDir:string,options:OptionInterface)=>void

4. 根据outDir找到所有的文件，根据pathName,我们直接返回一个文件list，其中包含本地路径和oss的路径

import * as fs from "fs";
import { globSync } from "glob";

export type FileItemType = {
  localPath: string,
  remotePath: string
}

const dealPath = (dir, file) => {
  dir = `${dir}/`
  return file.replace(/\\/g, "/").replace(dir.replace(/\\/g, "/"), "");
};

export const getFilePath = function (dir) {
  return new Promise(function (resolve, rejects) {
    try {
      console.log("正在查询将上传的静态资源文件，请稍等...");
      const stat = fs.statSync(dir);
      let fileList = [];
      if (stat.isFile()) {
        console.log(`请将要上传的文件放到打包目录下！！！！！！`)
      } else {
        fileList = globSync
          (dir + "/**", {
            dot: true,
          })
          .filter(function (file) {
            const stat = fs.statSync(file);
            return stat.isFile();
          })
          .map(function (file) {
            const uploadTarget: FileItemType = {
              localPath: "",
              remotePath: ""
            };
            uploadTarget.localPath = file;
            uploadTarget.remotePath = dealPath(dir, file)
            return uploadTarget;
          }) as any;
      }
      resolve(fileList);
    } catch (e) {
      rejects(e);
    }
  });
};

5. 开始写下上传逻辑

import { OptionInterface } from "./type";
import AWS from 'aws-sdk'

// 初始化s3的sdk
export const getClient = (ossOptions: OptionInterface["oss"]) => {
    // oss-京东云： https://docs.jdcloud.com/cn/object-storage-service/sdk-nodejs
    const s3: any = new AWS.S3({ apiVersion: '2006-03-01' });
    s3.endpoint = ossOptions.endpoint;
    s3.config.update({
      endpoint: ossOptions.endpoint,
      accessKeyId: ossOptions.accessKey,
      secretAccessKey: ossOptions.secretKey,
      s3ForcePathStyle: true,
      signatureVersion: "v4"
    })
    return s3
  }
  
  
 // 上传文件
 export  const uploadFile = (client, params) => {
    return new Promise((resolve, reject) => {
      client.upload(params, (err, data) => {
        if (err) {
          reject(`${params.key}：${err}`);
        } else {
          resolve(data)
        }
      });
    })
  }
  
// 获取全部遍历文件，单个文件进行upload
const deploy: deployType = (outDir: string, options: OptionInterface) => {
    const ossConfig = options
    console.log("*************************");
    console.log("部署流程准备中...");
    console.log("*************************");
    const startTime = new Date().getTime();
    const client = getClient(ossConfig || {})
    getFilePath(outDir).then(async (fileList: FileItemType[]) => {
        console.log(fileList,'fileList')
        if (fileList.length) {
            console.log(`共${fileList.length}个文件，将被上传，目标为${ossConfig?.bucketName}/${ossConfig.pathName}`);
            console.log('-------------------------------------------------------------')
            await Promise.all(fileList.map((item) => {
                const fileData = fs.readFileSync(item.localPath)
                const params = { Bucket: ossConfig.bucketName, Key: `${ossConfig.pathName}/${item.remotePath}`, Body: fileData };
                return uploadFile(client, params)
            })).then(async () => {
                console.log('文件上传成功')
            }).catch(err => {
                console.log("文件上传失败：", err.red)
            })
            const duration = (new Date().getTime() - startTime) / 1000;
            console.log("*************************");
            console.log("\x1b[32m%s\x1b[0m", `已完成上线 ^_^, cost ${duration.toFixed(2)}s`);
            console.log("*************************");

        } else {
            console.log("暂无部署文件，上线中断！！！！！",)
        }
    });
}

5. 最后优化下，把文件拆分下,vite和webpack的插件参数补齐一下

// vite plugin
import deploy from "deploy-oss";
import { OptionInterface, PluginRes } from "../lib/type";;
export  function DeployWebpackPlugin(options: OptionInterface): PluginRes | undefined {
    let buildConfig: any = {};
    return {
      name: "upload-oss-to-ducc",
      enforce: "post",
      apply: "build",
      configResolved(config: any) {
        buildConfig = config.build;
      },
      async closeBundle() {
        const outDir = buildConfig.outDir
        deploy(outDir, options)
      },
    };
  }
  
// webpack plugin
import deploy from "deploy-oss";
import { OptionInterface } from "../lib/type";;
export class DeployWebpackPlugin {
    private options: OptionInterface
    constructor(config: OptionInterface) {
        this.options = config;
    }
    apply(compiler: any) {
        const path = compiler.options.output.path || "";
        const afterEmit = (compilation, callback) => {
            deploy(path, this.options)
        };

        if (compiler.hooks) {
            compiler.hooks.afterEmit.tapAsync("UploadOssToDuccPlugin", afterEmit);
        } else {
            compiler.plugin("UploadOssToDuccPlugin", afterEmit);
        }
    }
}

6. 添加一个测试包，测试下效果，这里推荐一个s3的谷歌浏览器插件，配置下密钥就可以查看oss下面的所有文件，这样差不多就都完成了

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import { DeployWebpackPlugin } from "deploy-oss-vite-plugin";

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [react(), DeployWebpackPlugin({
    accessKey: "",
    secretKey: "",
    bucketName: "",
    pathName: "test"
  })],
})

7. 后面的逻辑可以根据自己项目使用情况自己加工，我这里因为打包出来的文件是带hash值的，所以需要进行一个请求把文件的根目录和我们文件存放的地址告诉后端。

/* eslint-disable @typescript-eslint/no-explicit-any */
import * as fs from "fs";
import { getFilePath } from './util'
import { FileItemType, OptionInterface } from './type';
// 具体的业务代码就不多说了，大家可以根据自己的业务自身加工request请求
// getConfigDefaultRecord, publishConfig, queryConfig, updateConfig
import { getConfigDefaultRecord, publishConfig, queryConfig, updateConfig, } from './ducc';
import { getClient, uploadFile } from './oss';


const deploy = (outDir: string, options: OptionInterface) => {
  const { oss: ossConfig, config:sourceConfig } = options
  console.log("*************************");
  console.log("部署流程准备中...");
  console.log("*************************");
  const startTime = new Date().getTime();
  const client = getClient(ossConfig || {})
  getFilePath(outDir).then(async (fileList:  FileItemType[]) => {
    if (fileList.length) {
      console.log(`共${fileList.length}个文件，将被上传，目标为${ossConfig?.bucketName}/${ossConfig.pathName}`);
      console.log('-------------------------------------------------------------')
      await Promise.all(fileList.map((item) => {
        const fileData = fs.readFileSync(item.localPath)
        const params = { Bucket: ossConfig.bucketName,  
                         Key: `${ossConfig.pathName}/${item.remotePath}`,
                         Body: fileData  };
        return uploadFile(client, params)
      })).then(async () => {
        console.log('文件上传成功')
      }).catch(err => {
        console.log("文件上传失败：", err.red)
      })
      console.log('-------------------------------------------------------------')
      if (sourceConfig) {
        const oldConfigDataMap = await queryConfig(sourceConfig)
        const { getDataMap } = sourceConfig
        let dataMap = getConfigDefaultRecord()
        dataMap = {
          ...dataMap,
          ...(getDataMap ? getDataMap(options, fileList) : {})
        }
        await Promise.all(Object.entries(dataMap).map(item => {
          return updateConfig(item, sourceConfig, oldConfigDataMap)
        })).then(() => {
          console.log("配置已更新！")
        }).catch((err) => {
          console.log("配置更新失败", err.red)
        })
        console.log('-------------------------------------------------------------')
        if (sourceConfig.autoPublish) {
          await publishConfig(sourceConfig)
          console.log("配置已发布！")
          console.log('-------------------------------------------------------------')
        }
        const duration = (new Date().getTime() - startTime) / 1000;
        console.log("*************************");
        console.log("\x1b[32m%s\x1b[0m", `已完成上线 ^_^, cost ${duration.toFixed(2)}s`);
        console.log("*************************");
      }
    } else {
      console.log("暂无部署文件，上线中断！！！！！",)
    }
  });
}


export default deploy

8. 最喜欢的环节，发包，这里推荐一个工具npm-run-all 和 changeset

   github.com/waltiu/depl...

  "scripts": {
    "build": "npm-run-all -s build:*",
    "build:lib": "cd ./packages/lib && pnpm run build",
    "build:vite": "cd ./packages/vite-plugin && pnpm run build",
    "build:webpack": "cd  ./packages/webpack-plugin && pnpm run build",
    "test": "npm-run-all -s test:*",
    "test:vite": "cd ./packages/vite-project && pnpm run test",
    "test:webpack": "cd  ./packages/webpack-project && pnpm run test",
    "pub":"npx changeset && npx changeset version &&  npx changeset publish"
  },

汇总下遇到的问题吧

1. 构建报错，compilerOptions 为 undefined 。解决办法：主要在tsconfig加上下面的配置就好了

    {
      "compilerOptions": {
        "moduleResolution": "node"
      }
    }

2. 上传到oss，提示accessKeyId 不对。解决办法：一定要2个endpoint都设置

    s3.endpoint = ossOptions.endpoint||endpoint;
    s3.config.update({
      endpoint: ossOptions.endpoint||endpoint,
      accessKeyId: ossOptions.accessKey,
      secretAccessKey: ossOptions.secretKey,
      s3ForcePathStyle: true,
      signatureVersion: "v4"
    })

3. fs报错。解决办法： 导入换成 import * as fs from "fs";
4. glob.sync取不到值。解决办法：导入换成 import { globSync } from "glob";
5. 部署完成后，静态资源访问成功，但是界面空白。解决办法：s3上传文件默认的content_type为octet-stream，我们需要在上传文件的时候，主动设置下content_type，推荐一个库mime

   const params = { 
                       Bucket: ossConfig.bucketName, 
                       Key: `${ossConfig.pathName}/${item.remotePath}`,
                       Body: fileData, 
                       ContentType: mime.getType(item.remotePath) 
                   };