编译测试兼容gzip并行压缩解压工具pigz

从存储库 https://github.com/madler/pigz 拉取源代码

直接make即可

复制代码
root@6ae32a5ffcde:/# cd /par/pigz-2.8
root@6ae32a5ffcde:/par/pigz-2.8# make
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual   -c -o pigz.o pigz.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual   -c -o yarn.o yarn.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual   -c -o try.o try.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/deflate.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/blocksplitter.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/tree.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/lz77.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/cache.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/hash.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/util.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/squeeze.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/katajainen.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/symbols.c
gcc  -o pigz pigz.o yarn.o try.o deflate.o blocksplitter.o tree.o lz77.o cache.o hash.o util.o squeeze.o katajainen.o symbols.o -lm -lpthread -lz
ln -f pigz unpigz

命令行选项如下

复制代码
root@6ae32a5ffcde:/par/pigz-2.8# ./pigz
Usage: pigz [options] [files ...]
  will compress files in place, adding the suffix '.gz'. If no files are
  specified, stdin will be compressed to stdout. pigz does what gzip does,
  but spreads the work over multiple processors and cores when compressing.

Options:
  -0 to -9, -11        Compression level (level 11, zopfli, is much slower)
  --fast, --best       Compression levels 1 and 9 respectively
  -A, --alias xxx      Use xxx as the name for any --zip entry from stdin
  -b, --blocksize mmm  Set compression block size to mmmK (default 128K)
  -c, --stdout         Write all processed output to stdout (won't delete)
  -C, --comment ccc    Put comment ccc in the gzip or zip header
  -d, --decompress     Decompress the compressed input
  -f, --force          Force overwrite, compress .gz, links, and to terminal
  -F  --first          Do iterations first, before block split for -11
  -h, --help           Display a help screen and quit
  -H, --huffman        Use only Huffman coding for compression
  -i, --independent    Compress blocks independently for damage recovery
  -I, --iterations n   Number of iterations for -11 optimization
  -J, --maxsplits n    Maximum number of split blocks for -11
  -k, --keep           Do not delete original file after processing
  -K, --zip            Compress to PKWare zip (.zip) single entry format
  -l, --list           List the contents of the compressed input
  -L, --license        Display the pigz license and quit
  -m, --no-time        Do not store or restore mod time
  -M, --time           Store or restore mod time
  -n, --no-name        Do not store or restore file name or mod time
  -N, --name           Store or restore file name and mod time
  -O  --oneblock       Do not split into smaller blocks for -11
  -p, --processes n    Allow up to n compression threads (default is the
                       number of online processors, or 8 if unknown)
  -q, --quiet          Print no messages, even on error
  -r, --recursive      Process the contents of all subdirectories
  -R, --rsyncable      Input-determined block locations for rsync
  -S, --suffix .sss    Use suffix .sss instead of .gz (for compression)
  -t, --test           Test the integrity of the compressed input
  -U, --rle            Use run-length encoding for compression
  -v, --verbose        Provide more verbose output
  -V  --version        Show the version of pigz
  -Y  --synchronous    Force output file write to permanent storage
  -z, --zlib           Compress to zlib (.zz) instead of gzip format
  --                   All arguments after "--" are treated as files

用文本文件测试压缩和解压

复制代码
root@6ae32a5ffcde:/par# pigz-2.8/pigz 1y.csv -k
root@6ae32a5ffcde:/par# time pigz-2.8/pigz 1y.csv -k -f

real    0m5.845s
user    0m13.396s
sys     0m4.133s
root@6ae32a5ffcde:/par# time pigz-2.8/pigz -9 1y.csv -k -f

real    0m9.410s
user    0m27.705s
sys     0m3.622s

root@6ae32a5ffcde:/par#
root@6ae32a5ffcde:/par# time pigz-2.8/pigz -d 1y.csv.gz -k -f

real    0m5.899s
user    0m1.192s
sys     0m2.313s

用二进制可执行文件测试

复制代码
root@6ae32a5ffcde:/par# time pigz-2.8/pigz -9 duckdb0904 -k -f

real    0m0.713s
user    0m4.438s
sys     0m0.306s
root@6ae32a5ffcde:/par# time pigz-2.8/pigz -d duckdb0904.gz -k -f

real    0m0.481s
user    0m0.172s
sys     0m0.182s

与pixz对比测试

复制代码
root@6ae32a5ffcde:/par# time pixz-1.0.7/src/pixz -1 1y.csv 1y1.pxz

real    0m26.298s
user    0m12.014s
sys     0m2.234s

root@6ae32a5ffcde:/par# time pixz-1.0.7/src/pixz -d 1y1.pxz 1y1.csv

real    0m3.822s
user    0m2.377s
sys     0m0.297s

root@6ae32a5ffcde:/par# time pixz-1.0.7/src/pixz duckdb0904 duckdb0904.pxz

real    0m6.575s
user    0m17.106s
sys     0m1.210s


root@6ae32a5ffcde:/par# time pixz-1.0.7/src/pixz -d duckdb0904.pxz duckdb0904.old

real    0m3.831s
user    0m0.535s
sys     0m0.178s

root@6ae32a5ffcde:/par# ls -l 1y.csv*
-rwxrwxrwx 1 root root 888888900 Sep 20 04:13 1y.csv
-rwxrwxrwx 1 root root 220501865 Sep 20 04:13 1y.csv.gz
-rwxrwxrwx 1 root root  22042116 Sep 26 10:35 1y1.pxz

root@6ae32a5ffcde:/par# ls -l duckdb0904*
-rwxrwxrwx 1 root root 68540512 Sep  4 12:46 duckdb0904
-rwxrwxrwx 1 root root 20959531 Sep  4 12:46 duckdb0904.gz
-rwxrwxrwx 1 root root 14170880 Sep 26 12:55 duckdb0904.pxz
相关推荐
青少儿编程课堂15 分钟前
蓝桥杯+全国青少年信息素养大赛+电子学会PythonC/C++考级:最长公共子序列LCS解析
c++·python·算法·bfs·信息学竞赛
倚楼听风语2 小时前
Kotlin 协程与 suspend 原理详解
java·算法·kotlin
程序员清风2 小时前
双指针与滑动窗口:高效处理连续区间
开发语言·算法
代钦塔拉2 小时前
【C++/Qt 进阶】设计模式系列(2):工厂模式——从游戏角色到披萨店
c++·算法·设计模式
思茂信息2 小时前
电磁仿真学涉及哪些算法?
人工智能·算法·数学建模·matlab·emc·电磁仿真·仿真辐射
糖果豆豆2 小时前
让 AI 自查三遍,它只会把错的说得更像对的——检查的人和被检查的,是同一个
算法·架构
把我格子衫拿来3 小时前
VTK 与 Qt 的 CMake 工程怎么组织?模块、工具链与二进制兼容排错
c++·qt·vtk·cmake·工程实践
总裁余(余登武)4 小时前
K-means算法改进(Python版)
python·算法·kmeans
灯澜忆梦4 小时前
【面向对象编程c++】| 指针与自定义数据类型
开发语言·c++·算法
xxwl5854 小时前
区域赛算法模版第八章---贪心(Greedy)
c++·笔记·学习·算法