Python自学:使用多进程处理 multiprocessing

1. 使用多进程执行函数

以下代码没有使用多进程。

python 复制代码
import time

start = time.perf_counter()

def do_something():
    print('Sleeping 1 second...')
    time.sleep(1)
    print('Done Sleep...')

do_something()
do_something()

finish = time.perf_counter()

print(f'Finished in {round(finish-start, 2)} second(s)')

输出为:

Sleeping 1 second...

Done Sleep...

Sleeping 1 second...

Done Sleep...

Finished in 2.03 second(s)

以下代码使用了多进程。

python 复制代码
import time
import multiprocessing


def do_something():
    print('Sleeping 1 second...')
    time.sleep(1)
    print('Done Sleep...')


if __name__ == '__main__':

    start = time.perf_counter()

    p1 = multiprocessing.Process(target=do_something)
    p2 = multiprocessing.Process(target=do_something)

    p1.start()
    p2.start()

    p1.join()
    p2.join()

    finish = time.perf_counter()

    print(f'Finished in {round(finish-start, 2)} second(s)')

输出为:

Sleeping 1 second...

Sleeping 1 second...

Done Sleep...

Done Sleep...

Finished in 1.07 second(s)

2. 使用loop创建多个进程,并在函数中传入参数。

python 复制代码
import time
import multiprocessing


def do_something(seconds):
    print(f'Sleeping {seconds} second(s)...')
    time.sleep(seconds)
    print('Done Sleep...')


if __name__ == '__main__':

    start = time.perf_counter()

    processes = []

    for _ in range(10):
        p = multiprocessing.Process(target=do_something, args=[1.5])
        p.start()
        processes.append(p)

    for process in processes:
        process.join()

    finish = time.perf_counter()

    print(f'Finished in {round(finish-start, 2)} second(s)')

输出为:

Sleeping 1.5 second(s)...

Sleeping 1.5 second(s)...

Sleeping 1.5 second(s)...

Sleeping 1.5 second(s)...

Sleeping 1.5 second(s)...

Sleeping 1.5 second(s)...

Sleeping 1.5 second(s)...

Sleeping 1.5 second(s)...

Sleeping 1.5 second(s)...

Sleeping 1.5 second(s)...

Done Sleep...

Done Sleep...

Done Sleep...

Done Sleep...

Done Sleep...

Done Sleep...

Done Sleep...

Done Sleep...

Done Sleep...

Done Sleep...

Finished in 1.62 second(s)

3. 使用进程池实现多进程

python 复制代码
import time
import concurrent.futures


def do_something(seconds):
    print(f'Sleeping {seconds} second(s)...')
    time.sleep(seconds)
    return f'Done Sleep...{seconds}'


if __name__ == '__main__':

    start = time.perf_counter()

    with concurrent.futures.ProcessPoolExecutor() as executor:
        secs = [5, 4, 3, 2, 1]
        results = executor.map(do_something, secs)

        for result in results:
            print(result)

    finish = time.perf_counter()

    print(f'Finished in {round(finish-start, 2)} second(s)')

输出为:

Sleeping 5 second(s)...

Sleeping 4 second(s)...

Sleeping 3 second(s)...

Sleeping 2 second(s)...

Sleeping 1 second(s)...

Done Sleep...5

Done Sleep...4

Done Sleep...3

Done Sleep...2

Done Sleep...1

Finished in 5.14 second(s)

4. 使用多进程处理图片

以下代码展示了没有使用多进程处理图片

python 复制代码
import time
from PIL import Image, ImageFilter

img_names = [
    'photo-1516117172878-fd2c41f4a759.jpg',
    'photo-1532009324734-20a7a5813719.jpg',
    'photo-1524429656589-6633a470097c.jpg',
    'photo-1530224264768-7ff8c1789d79.jpg',
    'photo-1564135624576-c5c88640f235.jpg',
    'photo-1541698444083-023c97d3f4b6.jpg',
    'photo-1522364723953-452d3431c267.jpg',
    'photo-1493976040374-85c8e12f0c0e.jpg',
    'photo-1504198453319-5ce911bafcde.jpg',
    'photo-1530122037265-a5f1f91d3b99.jpg',
    'photo-1516972810927-80185027ca84.jpg',
    'photo-1550439062-609e1531270e.jpg',
    'photo-1549692520-acc6669e2f0c.jpg'
]

t1 = time.perf_counter()

size = (1200, 1200)

for img_name in img_names:
    img = Image.open(img_name)

    img = img.filter(ImageFilter.GaussianBlur(15))

    img.thumbnail(size)

    img.save(f'processed/{img_name}')
    print(f'{img_name} was processed...')

t2 = time.perf_counter()

print(f'Finished in {t2-t1} seconds')

输出为:

photo-1516117172878-fd2c41f4a759.jpg was processed...

photo-1532009324734-20a7a5813719.jpg was processed...

photo-1524429656589-6633a470097c.jpg was processed...

photo-1530224264768-7ff8c1789d79.jpg was processed...

photo-1564135624576-c5c88640f235.jpg was processed...

photo-1541698444083-023c97d3f4b6.jpg was processed...

photo-1522364723953-452d3431c267.jpg was processed...

photo-1493976040374-85c8e12f0c0e.jpg was processed...

photo-1504198453319-5ce911bafcde.jpg was processed...

photo-1530122037265-a5f1f91d3b99.jpg was processed...

photo-1516972810927-80185027ca84.jpg was processed...

photo-1550439062-609e1531270e.jpg was processed...

photo-1549692520-acc6669e2f0c.jpg was processed...

Finished in 13.196055100299418 seconds

使用多进程的方式处理图片

python 复制代码
import time
import concurrent.futures
from PIL import Image, ImageFilter

img_names = [
    'photo-1516117172878-fd2c41f4a759.jpg',
    'photo-1532009324734-20a7a5813719.jpg',
    'photo-1524429656589-6633a470097c.jpg',
    'photo-1530224264768-7ff8c1789d79.jpg',
    'photo-1564135624576-c5c88640f235.jpg',
    'photo-1541698444083-023c97d3f4b6.jpg',
    'photo-1522364723953-452d3431c267.jpg',
    'photo-1493976040374-85c8e12f0c0e.jpg',
    'photo-1504198453319-5ce911bafcde.jpg',
    'photo-1530122037265-a5f1f91d3b99.jpg',
    'photo-1516972810927-80185027ca84.jpg',
    'photo-1550439062-609e1531270e.jpg',
    'photo-1549692520-acc6669e2f0c.jpg'
]


def process_image(img_name):
    
    img = Image.open(img_name)

    img = img.filter(ImageFilter.GaussianBlur(15))

    img.thumbnail((1200, 1200))

    img.save(f'processed/{img_name}')
    print(f'{img_name} was processed...')

if __name__ == '__main__':
    t1 = time.perf_counter()

    with concurrent.futures.ProcessPoolExecutor() as executor:
        executor.map(process_image, img_names)

    t2 = time.perf_counter()

    print(f'Finished in {t2-t1} seconds')

输出为:

photo-1516117172878-fd2c41f4a759.jpg was processed...

photo-1516972810927-80185027ca84.jpg was processed...

photo-1524429656589-6633a470097c.jpg was processed...

photo-1522364723953-452d3431c267.jpg was processed...

photo-1532009324734-20a7a5813719.jpg was processed...

photo-1530122037265-a5f1f91d3b99.jpg was processed...

photo-1530224264768-7ff8c1789d79.jpg was processed...

photo-1564135624576-c5c88640f235.jpg was processed...

photo-1550439062-609e1531270e.jpg was processed...

photo-1541698444083-023c97d3f4b6.jpg was processed...

photo-1549692520-acc6669e2f0c.jpg was processed...

photo-1504198453319-5ce911bafcde.jpg was processed...

photo-1493976040374-85c8e12f0c0e.jpg was processed...

Finished in 2.651644399855286 seconds

我们可以看到,处理时间缩短为原来的1/5,大大提高了图片处理的速度。

相关推荐
presenttttt2 分钟前
用Python和OpenCV从零搭建一个完整的双目视觉系统(四)
开发语言·python·opencv·计算机视觉
每日出拳老爷子8 分钟前
[C#] 使用TextBox换行失败的原因与解决方案:换用RichTextBox的实战经验
开发语言·c#
半桔12 分钟前
【Linux手册】从接口到管理:Linux文件系统的核心操作指南
android·java·linux·开发语言·面试·系统架构
nightunderblackcat21 分钟前
新手向:实现ATM模拟系统
java·开发语言·spring boot·spring cloud·tomcat·maven·intellij-idea
开开心心就好23 分钟前
电脑息屏工具,一键黑屏超方便
开发语言·javascript·电脑·scala·erlang·perl
笑衬人心。31 分钟前
Java 17 新特性笔记
java·开发语言·笔记
序属秋秋秋1 小时前
《C++初阶之内存管理》【内存分布 + operator new/delete + 定位new】
开发语言·c++·笔记·学习
木头左2 小时前
逻辑回归的Python实现与优化
python·算法·逻辑回归
ruan1145142 小时前
MySQL4种隔离级别
java·开发语言·mysql
quant_19863 小时前
R语言如何接入实时行情接口
开发语言·经验分享·笔记·python·websocket·金融·r语言