Multiprocessing Event Object In Python

Need for an Event Object

A process is a running instance of a computer program.

Every Python program is executed in a Process, which is a new instance of the Python interpreter. This process has the name MainProcess and has one thread used to execute the program instructions called the MainThread. Both processes and threads are created and managed by the underlying operating system.

Sometimes we may need to create new child processes in our program in order to execute code concurrently.

Python provides the ability to create and manage new processes via the multiprocessing.Process class.

You can learn more about multiprocessing in the tutorial:

In concurrent programming, sometimes we need to coordinate processes with a boolean variable. This might be to trigger an action or signal some result.

This could be achieved with a mutual exclusion lock (mutex) and a boolean variable, but provides no way for processes to wait for the variable to be set True.

Instead, this can be achieved using an event object.

What is an event object and how can we use it with processes in Python?

Example of Using a Shared Event with Processes

We can explore how to use a multiprocessing.Event object.

In this example we will create a suite of processes that each will perform some processing and report a message. All processes will use an event to wait to be set before starting their work. The main process will set the event and trigger the child processes to start work.

First, we can define a target task function that takes the shared multiprocessing.Event instance and a unique integer to identify the process.

|-------|-------------------------------------------------------|
| 1 2 3 | # target task function def task(event, number): # ... |

Next, the function will wait for the event to be set before starting the processing work.

|---------|---------------------------------------------------------------------------------------------------|
| 1 2 3 4 | ... # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() |

Once triggered, the process will generate a random number, block for a moment and report a message.

|-----------|---------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 | ... # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) |

Tying this together, the complete target task function is listed below.

|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 6 7 8 9 | # target task function def task(event, number): # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) |

The main process will first create the shared multiprocessing.Event instance, which will be in the "not set" state by default.

|-------|----------------------------------------------------|
| 1 2 3 | ... # create a shared event object event = Event() |

Next, we can create and configure five new processes specifying the target task() function with the event object and a unique integer as arguments.

This can be achieved in a list comprehension.

|-------|-----------------------------------------------------------------------------------------------------------|
| 1 2 3 | ... # create a suite of processes processes = [Process(target=task, args=(event, i)) for i in range(5)] |

We can then start all child processes.

|---------|---------------------------------------------------------------------|
| 1 2 3 4 | ... # start all processes for process in processes: process.start() |

Next, the main process will block for a moment, then trigger the processing in all of the child processes via the event object.

|-------------|---------------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 6 | ... # block for a moment print('Main process blocking...') sleep(2) # trigger all child processes event.set() |

The main process will then wait for all child processes to terminate.

|---------|------------------------------------------------------------------------------------------|
| 1 2 3 4 | ... # wait for all child processes to terminate for process in processes: process.join() |

Tying this all together, the complete example is listed below.

|----------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | # SuperFastPython.com # example of using an event object with processes from time import sleep from random import random from multiprocessing import Process from multiprocessing import Event # target task function def task(event, number): # wait for the event to be set print(f'Process {number} waiting...', flush=True) event.wait() # begin processing value = random() sleep(value) print(f'Process {number} got {value}', flush=True) # entry point if name == 'main': # create a shared event object event = Event() # create a suite of processes processes = [Process(target=task, args=(event, i)) for i in range(5)] # start all processes for process in processes: process.start() # block for a moment print('Main process blocking...') sleep(2) # trigger all child processes event.set() # wait for all child processes to terminate for process in processes: process.join() |

Running the example first creates and starts five child processes.

Each child process waits on the event before it starts its work, reporting a message that it is waiting.

The main process blocks for a moment, allowing all child processes to begin and start waiting on the event.

The main process then sets the event. This triggers all five child processes that perform their simulated work and report a message.

Note, your specific results will differ given the use of random numbers.

|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 2 3 4 5 6 7 8 9 10 11 | Main process blocking... Process 0 waiting... Process 1 waiting... Process 2 waiting... Process 3 waiting... Process 4 waiting... Process 0 got 0.06198821143561384 Process 4 got 0.219334069761699 Process 3 got 0.7335552378594119 Process 1 got 0.7948771419640999 Process 2 got 0.8713839353896263 |

Further Reading

This section provides additional resources that you may find helpful.

Python Multiprocessing Books

I would also recommend specific chapters in the books:

Guides

APIs

References

Overwhelmed by the python concurrency APIs?

Find relief, download my FREE Python Concurrency Mind Maps

Takeaways

You now know how to use a multiprocessing.Event Object in Python

Do you have any questions?

Ask your questions in the comments below and I will do my best to answer


相关推荐
何大春1 分钟前
【弱监督语义分割】Self-supervised Image-specific Prototype Exploration for WSSS 论文阅读
论文阅读·人工智能·python·深度学习·论文笔记·原型模式
在下不上天10 分钟前
Flume日志采集系统的部署,实现flume负载均衡,flume故障恢复
大数据·开发语言·python
SEVEN-YEARS13 分钟前
深入理解TensorFlow中的形状处理函数
人工智能·python·tensorflow
EterNity_TiMe_18 分钟前
【论文复现】(CLIP)文本也能和图像配对
python·学习·算法·性能优化·数据分析·clip
陌小呆^O^23 分钟前
Cmakelist.txt之win-c-udp-client
c语言·开发语言·udp
Suyuoa29 分钟前
附录2-pytorch yolov5目标检测
python·深度学习·yolo
儿时可乖了32 分钟前
使用 Java 操作 SQLite 数据库
java·数据库·sqlite
ruleslol33 分钟前
java基础概念37:正则表达式2-爬虫
java