rust zip异步压缩与解压

在使用actix-web框架的时候,如果使用zip解压任务将会占用一个工作线程,因为zip库是同步阻塞的,想用异步非阻塞需要用另一个库,下面列出同步解压,跟异步解压的两个方法实现,异步解压不会占用工作线程。注意:debug模式下rust异步压缩会很慢,打包成release之后就非常快了。

压缩

依赖

复制代码
tokio = { version = "1.35.1", features = ["macros"] }
tokio-util = "0.7.10"
async_zip = { version = "0.0.17", features = ["tokio", "tokio-fs", "deflate"] }
futures-lite = "2.3.0"
anyhow = "1.0.44"

rust代码

复制代码
use anyhow::anyhow;
use async_zip::tokio::read::seek::ZipFileReader;
use async_zip::tokio::write::ZipFileWriter;
use async_zip::{Compression, DeflateOption, ZipEntryBuilder};
use futures_lite::AsyncWriteExt;
use std::path::{Path, PathBuf};
use tokio::fs::File;
use tokio::fs::{create_dir_all, OpenOptions};
use tokio::fs::{read_dir, remove_dir_all};
use tokio::io::{AsyncBufReadExt, AsyncReadExt, BufReader};
use tokio_util::compat::TokioAsyncWriteCompatExt;
use tokio_util::compat::{FuturesAsyncWriteCompatExt, TokioAsyncReadCompatExt};

//读取文件夹文件
async fn dirs(dir: PathBuf) -> Result<Vec<PathBuf>, anyhow::Error> {
    let mut dirs = vec![dir];
    let mut files = vec![];
    while !dirs.is_empty() {
        let mut dir_iter = read_dir(dirs.remove(0)).await?;
        while let Some(entry) = dir_iter.next_entry().await? {
            let entry_path_buf = entry.path();
            if entry_path_buf.is_dir() {
                dirs.push(entry_path_buf);
            } else {
                files.push(entry_path_buf);
            }
        }
    }
    Ok(files)
}

//压缩单个文件
async fn zip_entry(
    input_path: &Path,
    file_name: &str,
    zip_writer: &mut ZipFileWriter<File>,
) -> Result<(), anyhow::Error> {
    let mut input_file = File::open(input_path).await?;
    let builder = ZipEntryBuilder::new(file_name.into(), Compression::Deflate)
        .deflate_option(DeflateOption::Normal);
    let mut entry_writer = zip_writer.write_entry_stream(builder).await?;
    futures_lite::io::copy(&mut input_file.compat(), &mut entry_writer).await?;
    entry_writer.close().await?;
    return Ok(());
}

//压缩
pub async fn zip(input_path: &Path, out_path: &Path) -> Result<(), anyhow::Error> {
    let file = File::create(out_path).await?;
    let mut writer = ZipFileWriter::with_tokio(file);
    let input_dir_str = input_path
        .as_os_str()
        .to_str()
        .ok_or(anyhow!("Input path not valid UTF-8."))?;
    if input_path.is_file() {
        let file_name = input_path
            .file_name()
            .ok_or(anyhow!("File name not found.".to_string()))?
            .to_string_lossy();
        zip_entry(input_path, &file_name, &mut writer).await?;
    } else {
        let entries = dirs(input_path.into()).await?;
        for entry_path_buf in entries {
            let entry_path = entry_path_buf.as_path();
            let entry_str = entry_path
                .as_os_str()
                .to_str()
                .ok_or(anyhow!("Directory file path not valid UTF-8."))?;
            let file_name = &entry_str[(input_dir_str.len() + 1)..];
            zip_entry(entry_path, file_name, &mut writer).await?;
        }
    }
    writer.close().await?;
    Ok(())
}

//解压
pub async fn unzip<T: AsRef<Path>>(path: T, out_path: T) -> Result<(), anyhow::Error> {
    let out_path = out_path.as_ref();
    if out_path.exists() {
        remove_dir_all(out_path).await?;
    } else {
        create_dir_all(out_path).await?;
    }
    let path = path.as_ref();
    let file = File::open(path).await?;
    let reader = BufReader::new(file);
    let mut zip = ZipFileReader::with_tokio(reader).await?;
    for index in 0..zip.file().entries().len() {
        let entry = zip
            .file()
            .entries()
            .get(index)
            .ok_or(anyhow!("zip entry not found".to_string()))?;
        let raw = entry.filename().as_bytes();
        let mut file_name = &String::from_utf8_lossy(raw).to_string(); //必需转换为utf8,否则有乱码
        let zip_path = out_path.join(file_name);
        if file_name.ends_with("/") {
            create_dir_all(&zip_path).await?;
            continue;
        }
        if let Some(p) = zip_path.parent() {
            if !p.exists() {
                create_dir_all(p).await?;
            }
        }
        let mut entry_reader = zip.reader_without_entry(index).await?;
        let mut writer = OpenOptions::new()
            .write(true)
            .create_new(true)
            .open(&zip_path)
            .await?;
        futures_lite::io::copy(&mut entry_reader, &mut writer.compat_write()).await?;
    }
    Ok(())
}

测试

复制代码
#[cfg(test)]
mod tests {
    use super::*;
    #[tokio::test]
    async fn test_zip() -> Result<(), anyhow::Error> {
        let path = Path::new("file/tmp/test");
        zip(path, Path::new("file/tmp/out.zip")).await?;
        Ok(())
    }

    #[tokio::test]
    async fn test_unzip() -> Result<(), anyhow::Error> {
        let path = Path::new("file/tmp/a/out.zip");
        unzip(path, Path::new("file/tmp")).await?;
        Ok(())
    }
}
相关推荐
ftpeak4 小时前
从零开始使用 axum-server 构建 HTTP/HTTPS 服务
网络·http·https·rust·web·web app
咸甜适中5 小时前
rust语言 (1.88) 学习笔记:客户端和服务器端同在一个项目中
笔记·学习·rust
咸甜适中9 小时前
rust语言 (1.88) egui (0.32.2) 学习笔记(逐行注释)(二十八)使用图片控件显示图片
笔记·学习·rust·egui
huli332012 小时前
pingora_web:首款基于 Cloudflare Pingora 的企业级 Rust Web 框架
rust
Pomelo_刘金13 小时前
如何优雅地抽离 Rust 子工程:以 rumqttd 为例
rust
几颗流星15 小时前
Rust 常用语法速记 - 错误处理
后端·rust
向上的车轮19 小时前
如何用 Rust 重写 SQLite 数据库(二):是否有市场空间?
数据库·rust·sqlite
烈风1 天前
004 Rust控制台打印输出
开发语言·后端·rust
a7360151 天前
二十二、包管理与发布 (Cargo 进阶)
开发语言·rust
编码浪子1 天前
趣味学RUST基础篇(异步补充)
开发语言·后端·rust