大文件上传服务实现(后端篇)

引言

本文从零构建一套初步完整的文件上传后端系统。内容覆盖项目框架搭建、统一接口出参规范、数据库实体设计(文件信息、分片数据、分片索引)、自定义依赖注入装饰器(InjectRepository),以及分片上传的核心逻辑(分片存在性校验、缺失分片检测、并发合并分片),最终实现 "支持断点续传、状态可追溯、接口标准化" 的文件上传服务。该文需搭配 《任务队列及大文件上传实现(前端篇)》一起食用。

核心技术栈

  • typeorm(基于 node 的 ORM 框架)
  • express(基于 node 的 Web 应用框架)
  • routing-controllers(基于装饰器的路由管理工具)
  • typedi(实现依赖注入)
  • zod(用于数据验证)

核心目录

vbscript 复制代码
src
 │  data-source.ts
 │  index.ts
 ├─constant
 │      index.ts
 ├─controller
 │      file.ts
 ├─decorators
 │      index.ts
 ├─entity
 │      file.ts
 │      fileChunk.ts
 │      fileChunkIndex.ts
 ├─interceptors
 │      response.ts
 ├─middlewares
 │      errorHandler.ts
 ├─migration
 ├─services
 │      file.ts
 └─utils
        file.ts
        response.ts

搭建过程

开始搭建

使用 typeORM 搭建一个项目框架

typescript 复制代码
// 全局安装typeorm
npm i typeorm -g

// 搭建一个名为 upload-file-be 的项目,并使用mysql数据库
typeorm init --name upload-file-be --database mysql --express

// 进入 upload-file-be 文件夹后执行,通过yarn管理包依赖
yarn install

调整 DateSource 配置

注意:数据库服务的搭建教程 不在该文章的讨论范围之内,大家可自行搜索相关文章。

arduino 复制代码
/** src/data-source.ts */
...
export const AppDataSource = new DataSource({
  // 更新数据库服务连接的配置
  type: "mysql",
  host: "localhost",
  port: 3306,
  username: "root",
  password: "12345678",
  database: "playground",

  synchronize: true,
  logging: false,
  // 自动导入所有实体定义
  entities: ["src/entity/**/*.ts"],
  migrations: [],
  subscribers: [],
});

安装以下依赖

  • routing-controllers(基于装饰器的路由管理工具) 及其 peer dependencies
  • typedi(实现依赖注入)
  • cors(实现跨域)
  • morgan(express 接口访问记录中间件)
  • multer(express 解析上传文件中间件)
  • zod(数据验证)
  • lodash(实用工具库)
bash 复制代码
yarn add routing-controllers class-transformer class-validator
yarn add typedi cors morgan multer zod lodash
yarn add @types/express @types/multer -D

FileController示例,路径:src/controller/file

typescript 复制代码
import { Get, JsonController } from "routing-controllers";

@JsonController("/file")
export class FileController {
  @Get("/all")
  async all() {
    return [
      {
        id: 0,
        name: "全部文件",
      },
    ];
  }
}

调整入口文件配置,路径:src/index.ts

typescript 复制代码
import "reflect-metadata";
import * as express from "express";
import * as bodyParser from "body-parser";
import * as logger from "morgan";
import { AppDataSource } from "./data-source";
import { useExpressServer } from "routing-controllers";

const PORT = 3010;

async function initialize() {
  try {
    // create express app
    let app = express();
    app.use(bodyParser.json());

    await AppDataSource.initialize();

    app = useExpressServer(app, {
      cors: true, // 支持跨域访问
      controllers: [`${__dirname}/controller/**/*.ts`], // 动态导入controller
      classTransformer: true,
      validation: true,
      defaultErrorHandler: false,
      routePrefix: "/api", // 所有路由增加前缀"/api"
      defaults: {
        paramOptions: {
          required: true,
        },
      },
    });

    // setup express app here
    app.use(logger("dev"));

    // start express server
    app.listen(PORT);
    console.log(`Express server has started on port ${PORT}`);
  } catch (error) {
    console.log(error);
  }
}

initialize();

浏览器访问:http://localhost:3010/api/file/all 返回以下数据代表项目搭建成功

统一接口出参规范

通过响应拦截器、错误处理中间件,统一将所有接口的出参规范

response 处理工具

路径:src/utils/response.ts

typescript 复制代码
export class JSON_Response {
  /**
   *  业务结果(成功或失败),更细粒度的区分可通过code实现。
   */
  success: boolean;
  /**
   *  业务结果的状态码,默认为成功(0)
   *
   *  成功区间:0-9999,默认:0
   *  失败区间:10000-19999,默认:10000
   */
  code!: number;
  /**
   * 响应的JSON数据,一般在 success 为 true 时才可能返回data
   */
  data?: any;
  /**
   * 业务请求提示, success 为 false 时返回失败原因。
   */
  msg: string;

  constructor() {}

  static success(r: Partial<JSON_Response> = {}) {
    const { code = 0, success = true, msg = "ok", data } = r;
    const intance = new JSON_Response();
    intance.code = code;
    intance.success = success;
    intance.msg = msg;
    intance.data = data;

    return intance;
  }

  static fail(r: Partial<JSON_Response> = {}) {
    const { code = 10000, success = false, msg = "发生未知错误" } = r;
    const intance = new JSON_Response();
    intance.code = code;
    intance.success = success;
    intance.msg = msg;

    return intance;
  }
}

/**
 * 将简单的数据转为标准的响应数据结构(JSON_Response)
 */
export function success(code?: number, data?: Record<string, unknown>): void;
export function success(msg?: string, data?: Record<string, unknown>): void;
export function success(data?: Record<string, unknown>, code?: number): void;
export function success(data?: Record<string, unknown>, msg?: string): void;
export function success(data?: Record<string, unknown>): void;
export function success() {
  const firstArg = arguments[0];
  const secondArg = arguments[1];
  const resParam = { success: true } as JSON_Response;

  if (typeof firstArg === "number" || typeof secondArg === "number") {
    resParam.code = typeof firstArg === "number" ? firstArg : secondArg;
  }

  if (typeof firstArg === "string" || typeof secondArg === "string") {
    resParam.msg = typeof firstArg === "string" ? firstArg : secondArg;
  }

  if (typeof firstArg === "object" || typeof secondArg === "object") {
    resParam.data = typeof firstArg === "object" ? firstArg : secondArg;
  }

  return JSON_Response.success(resParam);
}

export function fail(msg?: string, code?: number) {
  return JSON_Response.fail({ code, msg });
}

response 拦截器

路径:/src/interceptors/response.ts

typescript 复制代码
import {
  Action,
  Interceptor,
  InterceptorInterface,
  InternalServerError,
} from "routing-controllers";
import { Service } from "typedi";
import { fail, JSON_Response, success } from "../utils/response";

@Service()
@Interceptor()
export class ResponseInterceptor implements InterceptorInterface {
  // 将接口出参数据格式统一为 JSON_Response
  intercept(_: Action, result: any) {
    // 向前兼容
    if (result instanceof JSON_Response) {
      return result;
    }

    // 错误对象
    if (result instanceof Error) {
      return fail(result.message);
    }

    if (["object", "string", "number", "undefined"].includes(typeof result)) {
      return success(result);
    }

    return new InternalServerError("无效的返回值");
  }
}

errorHandler 中间件

路径:/src/middlewares/errorHandler.ts

typescript 复制代码
import {
  ExpressErrorMiddlewareInterface,
  Middleware,
} from "routing-controllers";
import { Service } from "typedi";
import { fail, JSON_Response } from "../utils/response";

@Middleware({ type: "after" })
@Service()
export class ErrorHandler implements ExpressErrorMiddlewareInterface {
  error(error: any, request: any, response: any, next: (err: any) => any) {
    let result: JSON_Response;

    // 处理已知的HTTP错误
    if (error.httpCode && error.httpCode >= 400) {
      result = fail(error.message || "请求错误");
      response.status(error.httpCode);
    }

    // 处理业务逻辑错误
    else if (error instanceof Error) {
      result = fail(error.message || "请求错误");
      response.status(500);
    }

    // 处理其他未知错误
    else {
      result = fail("服务器内部错误");
      response.status(500);
    }

    response.json(result);
  }
}

路径:/src/index.ts

diff 复制代码
+ import "reflect-metadata";
import * as express from "express";
import * as bodyParser from "body-parser";
import * as logger from "morgan";
import { AppDataSource } from "./data-source";
import { useExpressServer } from "routing-controllers";
+ import { ErrorHandler } from "./middlewares/errorHandler";
+ import { ResponseInterceptor } from "./interceptors/response";

const PORT = 3010;

async function initialize() {
  try {
    // create express app
    let app = express();
    app.use(bodyParser.json());

    await AppDataSource.initialize();

    app = useExpressServer(app, {
      cors: true, // 支持跨域访问
      controllers: [`${__dirname}/controller/**/*.ts`], // 动态导入controller
      classTransformer: true,
      validation: true,
      defaultErrorHandler: false,
      routePrefix: "/api", // 所有路由增加前缀"/api"
+     middlewares: [ErrorHandler],
+     interceptors: [ResponseInterceptor],
      defaults: {
        paramOptions: {
          required: true,
        },
      },
    });

    // setup express app here
    app.use(logger("dev"));

    // start express server
    app.listen(PORT);
    console.log(`Express server has started on port ${PORT}`);
  } catch (error) {
    console.log(error);
  }
}

initialize();

声明实体

FileInfo 实体

路径:/src/entity/file.ts

typescript 复制代码
import {
  Entity,
  Column,
  PrimaryColumn,
  CreateDateColumn,
  UpdateDateColumn,
  DeleteDateColumn,
  OneToMany,
  PrimaryGeneratedColumn,
} from "typeorm";
import { FileChunk } from "./fileChunk";

/**
 * 文件状态
 */
export enum FileStatus {
  Uploading = 0,
  Mergeing = 1,
  Success = 2,
  Failed = 3,
}

@Entity()
export class FileInfo {
  @PrimaryGeneratedColumn()
  id: number;

  @Column({
    type: "varchar",
    comment: "文件hash",
  })
  hash?: string;

  @Column({
    type: "varchar",
    comment: "文件名",
    default: "index",
  })
  name?: string;

  @Column({
    type: "varchar",
    comment: "文件目录",
    nullable: true,
  })
  path: string;

  @Column({
    type: "varchar",
    comment: "chunk目录",
    nullable: true,
  })
  chunkPath: string;

  @Column({
    type: "bigint",
    comment: "文件大小,单位字节(B)",
  })
  totalSize: number;

  @Column({
    type: "varchar",
    comment: "文件拓展名",
  })
  extension? : string;

  @Column({
    type: "int",
    comment: "文件chunk数量",
  })
  totalChunk: number;

  @Column({
    type: "int",
    comment: "文件chunk大小,单位字节(B)",
  })
  chunkSize: number;

  @Column({
    type: "tinyint",
    comment: "文件状态,0:上传中、1:合并中、2:成功、3:失败",
  })
  status: FileStatus;

  // 关联分片表
  @OneToMany(() => FileChunk, (chunk) => chunk.fileInfo)
  chunks: FileChunk[];

  @CreateDateColumn({ type: "timestamp" })
  createdAt!: Date;

  @UpdateDateColumn({ type: "timestamp", select: false })
  updatedAt!: Date;

  @DeleteDateColumn({ type: "timestamp", select: false })
  deletedAt!: Date;
}

FileChunk 实体

路径:/src/entity/fileChunk.ts

typescript 复制代码
import {
  Entity,
  Column,
  CreateDateColumn,
  UpdateDateColumn,
  DeleteDateColumn,
  PrimaryGeneratedColumn,
  ManyToOne,
} from "typeorm";
import { FileInfo } from "./file";

@Entity()
export class FileChunk {
  @PrimaryGeneratedColumn()
  id: number;

  @Column({ type: "bigint" })
  fileId: number; // 文件唯一标识

  @Column({ type: "int", name: "chunk_index" })
  chunkIndex: number; // 分片索引

  @Column({ type: "varchar", name: "file_path" })
  filePath: string; // 分片存储路径

  // 关联文件信息表
  @ManyToOne(() => FileInfo, (fileInfo) => fileInfo.chunks)
  fileInfo: FileInfo;

  @CreateDateColumn({ type: "timestamp" })
  createdAt!: Date;

  @UpdateDateColumn({ type: "timestamp", select: false })
  updatedAt!: Date;

  @DeleteDateColumn({ type: "timestamp", select: false })
  deletedAt!: Date;
}

FileChunkIndex 实体

路径:/src/entity/fileChunkIndex.ts

typescript 复制代码
import {
  Entity,
  Column,
  PrimaryColumn,
  CreateDateColumn,
  UpdateDateColumn,
  DeleteDateColumn,
  PrimaryGeneratedColumn,
} from "typeorm";

@Entity()
export class FileChunkIndex {
  @PrimaryGeneratedColumn()
  id: number;

  @Column({ type: "bigint" })
  fileId: number;

  @Column({ type: "int", comment: "文件chunk下标" })
  index: number;

  @CreateDateColumn({ type: "timestamp" })
  createdAt!: Date;

  @UpdateDateColumn({ type: "timestamp", select: false })
  updatedAt!: Date;

  @DeleteDateColumn({ type: "timestamp", select: false })
  deletedAt!: Date;
}

InjectRepository装饰器

实现一个 InjectRepository 装饰器,可以在 Controller/Service 中通过它注入Repository 路径:/src/constant/index.ts

typescript 复制代码
export const DATA_SOURCE_ID = "data_source";

定义 InjectRepository

路径:/src/decorators/index.ts

typescript 复制代码
import Container from "typedi";
import { DATA_SOURCE_ID } from "../constant";
import { DataSource } from "typeorm";

export function InjectRepository(name: string) {
  return (
    target: any,
    propertyKey: string,
    descriptor?: PropertyDescriptor
  ) => {
    const ds = Container.get<DataSource>(DATA_SOURCE_ID);
    const repo = ds.getRepository(name);

    Object.defineProperty(target, propertyKey, {
      value: repo,
      ...(descriptor || {}),
    });
  };
}

路径:/src/index.ts

diff 复制代码
import "reflect-metadata";
import * as express from "express";
import * as bodyParser from "body-parser";
import * as logger from "morgan";
import { AppDataSource } from "./data-source";
- import { useExpressServer } from "routing-controllers";
+ import { useContainer, useExpressServer } from "routing-controllers";
import { ErrorHandler } from "./middlewares/errorHandler";
import { ResponseInterceptor } from "./interceptors/response";
+ import Container from "typedi";
+ import { DATA_SOURCE_ID } from "./constant";

const PORT = 3010;

async function initialize() {
  try {
    // create express app
    let app = express();
    app.use(bodyParser.json());
    
-   await AppDataSource.initialize();
+    useContainer(Container);
+
+    const ds = await AppDataSource.initialize();
+    Container.set({
+      id: DATA_SOURCE_ID,
+      global: true,
+      value: ds,
+    });
    
    ......

FileService

路径:/src/services/file

typescript 复制代码
import { Service } from "typedi";
import { InjectRepository } from "../decorators";
import { In, Repository } from "typeorm";
import { FileInfo, FileStatus } from "../entity/file";
import { FileChunk } from "../entity/fileChunk";
import { FileChunkIndex } from "../entity/fileChunkIndex";

@Service()
export class FileService {
  @InjectRepository("FileInfo")
  private fileRepository: Repository<FileInfo>;
  @InjectRepository("FileChunk")
  private fileChunkRepository: Repository<FileChunk>;
  @InjectRepository("FileChunkIndex")
  private fileChunkIndexRepository: Repository<FileChunkIndex>;

  /**
   *  chunk 是否存在
   */
  async isExistChunk(fileId: number, index: number) {
    return await this.fileChunkRepository.exists({
      where: {
        fileId,
        chunkIndex: index,
      },
    });
  }

  /**
   *  获取 file
   */
  async getLiteFile(
    hash: string,
    totalSize: number,
    chunkSize: number,
    extension?: string
  ) {
    return await this.fileRepository.findOneBy({
      hash,
      totalSize,
      chunkSize,
      extension,
    });
  }

  /**
   *  获取 file
   */
  async getLiteFileById(id: number) {
    return await this.fileRepository.findOneBy({
      id,
    });
  }

  /**
   *  删除 file 及其相关联数据
   */
  async deleteFile(id: number) {
    await this.fileRepository.delete(id);
    await this.fileChunkRepository.delete({
      fileId: id,
    });
    await this.fileChunkIndexRepository.delete({
      fileId: id,
    });
  }

  /**
   *  确保 file、fileIndex 存在,不存在会自动创建
   */
  async ensureFileExistence(
    hash: string,
    totalSize: number,
    chunkSize: number,

    chunkPathPrefix: string,
    pathPrefix: string,
    extension?: string,
    name?: string
  ) {
    let file = await this.fileRepository.findOneBy({
      hash: hash,
      totalSize,
      chunkSize,
      extension,
      name,
    });

    if (!file) {
      file = await this.createFile(hash, totalSize, chunkSize, extension, name);

      await this.fileRepository.update(
        { id: file.id },
        {
          chunkPath: `${chunkPathPrefix}/${file.id}`,
          path: `${pathPrefix}/${file.id}`,
        }
      );

      const chunkIndex = Array.from({ length: file.totalChunk }).map((r, i) => {
        return {
          fileId: file.id,
          index: i,
        };
      });

      await this.fileChunkIndexRepository.save(chunkIndex);
    }

    return file;
  }

  /**
   *  新建 file
   */
  async createFile(
    hash: string,
    totalSize: number,
    chunkSize: number,
    extension?: string,
    chunkPath?: string,
    path?: string,
    name?: string
  ) {
    const file = new FileInfo();
    file.hash = hash;
    file.totalSize = totalSize;
    file.chunkSize = chunkSize;
    file.chunkPath = chunkPath;
    file.path = path;
    file.extension = extension;
    file.status = FileStatus.Uploading;
    file.totalChunk = Math.ceil(totalSize / chunkSize);
    if (name) {
      file.name = name;
    }

    return await this.fileRepository.save(file);
  }

  /**
   *  新建 file chunk
   */
  async createFileChunk(fileId: number, chunkIndex: number, filePath: string) {
    const chunk = new FileChunk();
    chunk.fileId = fileId;
    chunk.chunkIndex = chunkIndex;
    chunk.filePath = filePath;

    return await this.fileChunkRepository.save(chunk);
  }

  /**
   *  获取文件 chunk
   */
  async getFileChunks(fileId: number, chunkIndexs?: number[]) {
    return await this.fileChunkRepository.findBy({
      fileId,
      ...(chunkIndexs?.length ? { chunkIndex: In(chunkIndexs) } : {}),
    });
  }

  /**
   *  获取缺失的 chunk index 列表
   */
  async getMissingChunkIndexs(id: number) {
    const file = await this.getLiteFileById(id);
    if (!file) {
      return;
    }

    const { totalChunk } = file;
    const queryBuilder = this.fileChunkIndexRepository
      .createQueryBuilder("index")
      // 只取0到totalChunks-1范围内的数字
      .where("index.index < :totalChunk AND index.fileId = :fileId", {
        totalChunk,
        fileId: file.id,
      })
      .leftJoin(
        FileChunk,
        "c",
        "c.fileId = :fileId AND c.chunkIndex = index.index",
        {
          fileId: id,
        }
      )
      // 筛选出未匹配到的数字(即缺失的chunk索引)
      .andWhere("c.chunkIndex IS NULL")
      .select("index.index", "missing_chunk_index")
      .orderBy("index.index", "ASC");

    const result = await queryBuilder.getRawMany();
    return result.map((item) => item.missing_chunk_index);
  }

  /**
   *  删除fileChunks
   */
  async removeFileChunks(fileId: number, chunkIndexs?: number[]) {
    await this.fileChunkRepository.delete({
      fileId,
      ...(chunkIndexs?.length ? { chunkIndex: In(chunkIndexs) } : {}),
    });
  }
}

FileController

路径:/src/controller/file.ts

typescript 复制代码
import { Inject, Service } from "typedi";
import { FileInfo, FileStatus } from "../entity/file";
import { InjectRepository } from "../decorators";
import { DataSource, Repository } from "typeorm";
import {
  Body,
  Delete,
  Get,
  JsonController,
  Param,
  Post,
  Req,
  UploadedFile,
} from "routing-controllers";
import {
  ensureDirectoryExistence,
  getFileUploadOptions,
  processChunk,
  readDirSync,
} from "../utils/file";

import * as multer from "multer";
import * as z from "zod";
import { en } from "zod/locales";
import { fail, success } from "../utils/response";
import * as fs from "fs";
import { File_PREFIX_PATH, UPLOAD_CHUNK_PREFIX_PATH } from "../constant";
import { keyBy } from "lodash";
import { FileService } from "../services/file";

z.config(en());

const UploadFileChunk = z.object({
  hash: z.string().min(1),
  index: z.coerce.number().int(),
  fileSize: z.coerce.number().int(),
  chunkSize: z.coerce.number().int(),
  extension: z.string(),
});

export type UploadQuery = {
  hash: string;
  index: number;
  fileSize: number;
  chunkSize: number;
  extension?: string;
};

const fileUploadOptions = getFileUploadOptions(
  async (
    req: Express.Request & {
      _transformedQuery: UploadQuery;
      _chunkDirName: string;
    },
    file: Express.Multer.File
  ) => {
    const query = (req as unknown as { query: Record<string, string> }).query;

    let payload: UploadQuery;

    // 校验入参是否合法
    let errorMessage = "";
    const parseResult = UploadFileChunk.safeParse(query);
    if (!parseResult.success) {
      if (parseResult.error instanceof z.ZodError) {
        // 格式:【字段】、【字段】-错误信息
        errorMessage = parseResult.error.issues
          .map(
            (e) =>
              `${e.path.map((field) => `【${String(field)}】`).join("、")}-${
                e.message
              }`
          )
          .join(",");
      }
    } else {
      payload = parseResult.data;
    }

    if (errorMessage) throw new Error(errorMessage);

    const fileService = new FileService();

    const { hash, index, fileSize, chunkSize, extension } = payload;

    const fileInfo = await fileService.ensureFileExistence(
      hash,
      fileSize,
      chunkSize,
      `${UPLOAD_CHUNK_PREFIX_PATH}`,
      `${File_PREFIX_PATH}`,
      extension
    );

    const isExist = await fileService.isExistChunk(fileInfo.id, index);

    req._transformedQuery = payload;
    req._chunkDirName = String(fileInfo.id);

    return !isExist;
  }
);

@Service()
@JsonController("/file")
export class FileController {
  @InjectRepository("FileInfo")
  private fileRepository: Repository<FileInfo>;

  @Inject()
  private fileService: FileService;

  @Get("/all")
  async all() {
    return this.fileRepository.find();
  }

  @Post("/upload")
  async upload(
    @Req() request: { _transformedQuery?: UploadQuery },
    @Body({ required: false }) body: any,
    @UploadedFile("chunk", { options: fileUploadOptions, required: false })
    chunk: any
  ) {
    if (!request._transformedQuery) return fail("缺少必要参数");

    const { hash, fileSize, chunkSize, extension, index } =
      request._transformedQuery ?? {};

    const fileInfo = await this.fileService.getLiteFile(
      hash,
      fileSize,
      chunkSize,
      extension
    );

    // 文件保存成功
    if (chunk) {
      await this.fileService.createFileChunk(
        fileInfo.id,
        index,
        fileInfo.chunkPath
      );
    }

    // 上传成功
    if (fileInfo.status === FileStatus.Success) {
      return success("上传成功", {
        status: fileInfo.status,
        filePath: fileInfo.path,
      });
    } else if (fileInfo.status === FileStatus.Mergeing) {
      // 合并中
      return success("合并中", {
        status: fileInfo.status,
      });
    } else if (fileInfo.status === FileStatus.Uploading) {
      const missingChunk = await this.fileService.getMissingChunkIndexs(
        fileInfo.id
      );

      if (missingChunk.length)
        return success({
          status: fileInfo.status,
          pendingUploadChunks: missingChunk,
        });

      const chunkNames = readDirSync(fileInfo.chunkPath);
      const chunkNameMap = keyBy(chunkNames);

      const chunkIndexs = Array.from(
        { length: fileInfo.totalChunk },
        (_, i) => i
      );

      // 存在未上传chunk
      const losedChunks = chunkIndexs.filter((i) => !chunkNameMap[i]);
      if (losedChunks.length) {
        await this.fileService.removeFileChunks(fileInfo.id, losedChunks);
        return success({
          status: fileInfo.status,
          pendingUploadChunks: losedChunks,
        });
      }

      const fileName = `${fileInfo.path}/index${
        extension ? `.${extension}` : ""
      }`;

      ensureDirectoryExistence(fileInfo.path);
      const writeStream = fs.createWriteStream(fileName, {
        flags: "w",
        encoding: null,
      });

      // 开始尝试合并,使用并发控制组装切片
      try {
        await this.fileRepository.update(
          { id: fileInfo.id },
          {
            status: FileStatus.Mergeing,
          }
        );

        const chunkIndexes = Array.from(
          { length: fileInfo.totalChunk },
          (_, i) => i
        );

        for (let i = 0; i < chunkIndexes.length; i += 5) {
          const batch = chunkIndexes.slice(i, i + 5);

          await Promise.all(
            batch.map((index) =>
              processChunk(`${fileInfo.chunkPath}`, String(index), writeStream)
            )
          );
        }

        writeStream.end();

        // 验证文件大小
        await fs.stat(fileName, (err, stats) => {
          if (fileSize && stats.size !== fileSize) {
            throw new Error(
              `文件大小不匹配: 期望 ${fileSize}, 实际 ${stats.size}`
            );
          }
        });

        await this.fileRepository.update(
          { id: fileInfo.id },
          {
            status: FileStatus.Success,
          }
        );
        return success({
          status: FileStatus.Success,
        });
      } catch (error) {
        // 合并失败删除所有chunk
        writeStream.destroy();
        await this.fileService.removeFileChunks(fileInfo.id);

        // 返回失败结果
        await this.fileRepository.update(
          { id: fileInfo.id },
          {
            status: FileStatus.Failed,
          }
        );

        return success({
          status: FileStatus.Failed,
        });
      }
    } else {
      // 失败
      return success({
        status: FileStatus.Failed,
      });
    }
  }

  @Delete("/:id")
  async deleteFile(@Param("id") id: number) {
    const file = await this.fileService.getLiteFileById(id);
    if (!file) return fail("文件不存在");
    const { path, chunkPath } = file;

    await fs.rmdirSync(`${path}`);
    await fs.rmdirSync(`${chunkPath}`);

    await this.fileService.deleteFile(id);
  }
}

总结

  • 用 TypeORM 管数据库、routing-controllers 简化路由、typedi 实现依赖注入,搭配 multer/cors 等中间件,形成分层清晰(Controller→Service→Repository)的项目架构。

  • 通过ResponseInterceptor拦截器、自定义ErrorHandler中间件将接口响应统一为JSON_Response格式(包含 success、code、data、msg 字段)

  • 核心通过FileService封装关键能力:ensureFileExistence确保文件与分片索引初始化、getMissingChunkIndexs检测缺失分片(支持断点续传)、processChunk实现并发分片合并(提升效率);

  • 前端上传前先校验分片存在性,避免重复上传;合并前校验文件大小,确保传输完整性

相关推荐
今天没ID3 小时前
高阶函数
后端
初级程序员Kyle3 小时前
开始改变第三天 Java并发(1)
java·后端
无名之辈J3 小时前
GC Overhead 排查
后端
倚栏听风雨4 小时前
jackson @JsonAnyGetter @JsonAnySetter 使用说明
后端
一枚前端小能手4 小时前
🚀 Node.js 25重磅发布!快来看看吧
前端·javascript·node.js
impossible19947274 小时前
如何开发一个自己的包并发布到npm
node.js·1024程序员节
Mintopia4 小时前
🚀 Next.js 16 新特性深度解析:当框架开始思考人生
前端·后端·全栈
Good kid.4 小时前
一键部署 Deepseek网页聊天系统(基于 Spring Boot + HTML 的本地对话系统)
spring boot·后端·html