Multiprocessing Event Object In Python

Need for an Event Object

A process is a running instance of a computer program.

Every Python program is executed in a Process, which is a new instance of the Python interpreter. This process has the name MainProcess and has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.

Sometimes we may need to create new child processes in our program in order to execute code concurrently.

Python provides the ability to create and manage new processes via the multiprocessing.Process class.

You can learn more about multiprocessing in the tutorial:

In concurrent programming, sometimes we need to coordinate processes with a boolean variable. This might be to trigger an action or signal some result.

This could be achieved with a mutual exclusion lock (mutex) and a boolean variable, but provides no way for processes to wait for the variable to be set True.

Instead, this can be achieved using an event object.

What is an event object and how can we use it with processes in Python?

Example of Using a Shared Event with Processes

We can explore how to use a multiprocessing.Event object.

In this example we will create a suite of processes that each will perform some processing and report a message. All processes will use an event to wait to be set before starting their work. The main process will set the event and trigger the child processes to start work.

First, we can define a target task function that takes the shared multiprocessing.Event instance and a unique integer to identify the process.

|-------|-------------------------------------------------------|
| 1 2 3 | # target task function def task(event, number): # ... |

Next, the function will wait for the event to be set before starting the processing work.

|---------|---------------------------------------------------------------------------------------------------|
| 1 2 3 4 | ... # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() |

Once triggered, the process will generate a random number, block for a moment and report a message.

|-----------|---------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 | ... # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) |

Tying this together, the complete target task function is listed below.

|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 6 7 8 9 | # target task function def task(event, number): # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) |

The main process will first create the shared multiprocessing.Event instance, which will be in the "not set" state by default.

|-------|----------------------------------------------------|
| 1 2 3 | ... # create a shared event object event = Event() |

Next, we can create and configure five new processes specifying the target task() function with the event object and a unique integer as arguments.

This can be achieved in a list comprehension.

|-------|-----------------------------------------------------------------------------------------------------------|
| 1 2 3 | ... # create a suite of processes processes = [Process(target=task, args=(event, i)) for i in range(5)] |

We can then start all child processes.

|---------|---------------------------------------------------------------------|
| 1 2 3 4 | ... # start all processes for process in processes: process.start() |

Next, the main process will block for a moment, then trigger the processing in all of the child processes via the event object.

|-------------|---------------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 6 | ... # block for a moment print('Main process blocking...') sleep(2) # trigger all child processes event.set() |

The main process will then wait for all child processes to terminate.

|---------|------------------------------------------------------------------------------------------|
| 1 2 3 4 | ... # wait for all child processes to terminate for process in processes: process.join() |

Tying this all together, the complete example is listed below.

|----------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | # SuperFastPython.com # example of using an event object with processes from time import sleep from random import random from multiprocessing import Process from multiprocessing import Event # target task function def task(event, number): # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) # entry point if name == 'main': # create a shared event object event = Event() # create a suite of processes processes = [Process(target=task, args=(event, i)) for i in range(5)] # start all processes for process in processes: process.start() # block for a moment print('Main process blocking...') sleep(2) # trigger all child processes event.set() # wait for all child processes to terminate for process in processes: process.join() |

Running the example first creates and starts five child processes.

Each child process waits on the event before it starts its work, reporting a message that it is waiting.

The main process blocks for a moment, allowing all child processes to begin and start waiting on the event.

The main process then sets the event. This triggers all five child processes that perform their simulated work and report a message.

Note, your specific results will differ given the use of random numbers.

|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 6 7 8 9 10 11 | Main process blocking... Process 0 waiting... Process 1 waiting... Process 2 waiting... Process 3 waiting... Process 4 waiting... Process 0 got 0.06198821143561384 Process 4 got 0.219334069761699 Process 3 got 0.7335552378594119 Process 1 got 0.7948771419640999 Process 2 got 0.8713839353896263 |

Further Reading

This section provides additional resources that you may find helpful.

Python Multiprocessing Books

I would also recommend specific chapters in the books:

Guides

APIs

References

Overwhelmed by the python concurrency APIs?

Find relief, download my FREE Python Concurrency Mind Maps

Takeaways

You now know how to use a multiprocessing.Event Object in Python

Do you have any questions?

Ask your questions in the comments below and I will do my best to answer


相关推荐
钱钱钱端6 分钟前
【压力测试】如何确定系统最大并发用户数?
自动化测试·软件测试·python·职场和发展·压力测试·postman
慕卿扬6 分钟前
基于python的机器学习(二)—— 使用Scikit-learn库
笔记·python·学习·机器学习·scikit-learn
Json____12 分钟前
python的安装环境Miniconda(Conda 命令管理依赖配置)
开发语言·python·conda·miniconda
斌斌_____13 分钟前
通过反射机制,比较两个对象的字段值的差异
java
cooldream200917 分钟前
Spring Boot中集成MyBatis操作数据库详细教程
java·数据库·spring boot·mybatis
阑梦清川26 分钟前
JavaEE进阶---第一个SprintBoot项目创建过程&&&我的感受
java·java-ee·springboot
程序员清风32 分钟前
浅析Web实时通信技术!
java·后端·面试
wyh要好好学习39 分钟前
SSM— spring,springMVC,mybatis整合
java·spring
2401_8582861139 分钟前
C6.【C++ Cont】cout的格式输出
开发语言·c++
小袁在上班40 分钟前
Python 单元测试中的 Mocking 与 Stubbing:提高测试效率的关键技术
python·单元测试·log4j