windows下使用PowerShell切割大数据文件

测试文件为24.4G文件

打开PowerShell窗口，使用以下命令

$filePath 为指向文件路径 $outputPath 输出到指定文件夹 $chunkSize 单个文件控制切割大小

将命令修改完后，直接粘贴到powershell窗口，点击回车即可进行切割

java 复制代码

$filePath = "D:\test\zb_content_1.txt"
$outputPath = "D:\test\"
$chunkSize = 1GB
$bufferSize = 1MB
$fileStream = [System.IO.File]::OpenRead($filePath)
$reader = New-Object System.IO.BinaryReader($fileStream)
$chunkIndex = 1

try {
    while ($true) {
        $chunkFilePath = Join-Path $outputPath "zb_content_1_$chunkIndex.txt"
        $chunkIndex++
        $writer = New-Object System.IO.BinaryWriter([System.IO.File]::OpenWrite($chunkFilePath))

        try {
            $bytesLeft = $chunkSize
            while ($bytesLeft -gt 0) {
                $buffer = $reader.ReadBytes([Math]::Min($bytesLeft, $bufferSize))
                if ($buffer.Length -eq 0) {
                    break
                }
                $writer.Write($buffer)
                $bytesLeft -= $buffer.Length
            }
        } finally {
            $writer.Close()
        }

        if ($bytesLeft -gt 0) {
            break
        }
    }
} finally {
    $reader.Close()
}