Python异步----------信号量

定义

sem = asyncio.Semaphore(100)

使用

python 复制代码

sem = asyncio.Semaphore(100)
async def fetch(url):
    async with sem:
        async with session.get(url) as resp:
            return await resp.txt()

原理

信号量机制=计数器+等待队列

1，初始值100表示有100个令牌

2，async with sem 等价于先await sem.acquire() 再sem.release()

并发上限的实现

1，前100个进入acquire()拿到令牌，继续执行并在io上await

2，第101个协程在await acquire()处被挂起，被放进semphore的等待队列，事件循环会去跑别的可运行任务

3，当某个已经进入临界区的协程执行完毕退出时 sem.release()发生，信号量从等待队列里唤醒一个协程，让他拿到令牌进入临界区

4，semaphore 限的是包住的那段代码的""在途数量"" 如把session.get包住，则限制了同时在发/等响应的请求数。

源码

python 复制代码

class Semaphore(_ContextManagerMixin, mixins._LoopBoundMixin):
    """A Semaphore implementation.

    A semaphore manages an internal counter which is decremented by each
    acquire() call and incremented by each release() call. The counter
    can never go below zero; when acquire() finds that it is zero, it blocks,
    waiting until some other thread calls release().

    Semaphores also support the context management protocol.

    The optional argument gives the initial value for the internal
    counter; it defaults to 1. If the value given is less than 0,
    ValueError is raised.
    """

    def __init__(self, value=1):
        if value < 0:
            raise ValueError("Semaphore initial value must be >= 0")
        self._waiters = None    #等待队列
        self._value = value     #令牌个数

python 复制代码

    async def acquire(self):
        """Acquire a semaphore.

        If the internal counter is larger than zero on entry,
        decrement it by one and return True immediately.  If it is
        zero on entry, block, waiting until some other coroutine has
        called release() to make it larger than 0, and then return
        True.
        """
        if not self.locked():  #消耗令牌  
        #令牌在"唤醒等待者"的那一刻就被扣掉了，而不是等协程真正恢复执行时才扣
            self._value -= 1
            return True

        if self._waiters is None:
            self._waiters = collections.deque()  #创建等待队列
        fut = self._get_loop().create_future()
        self._waiters.append(fut)

        # Finally block should be called before the CancelledError
        # handling as we don't want CancelledError to call
        # _wake_up_first() and attempt to wake up itself.
        try:
            try:
                await fut
            finally:
                self._waiters.remove(fut)
        except exceptions.CancelledError:
            if not fut.cancelled():
                self._value += 1
                self._wake_up_next()
            raise

        if self._value > 0:
            self._wake_up_next()
        return True

python 复制代码

    def release(self):
        """Release a semaphore, incrementing the internal counter by one.

        When it was zero on entry and another coroutine is waiting for it to
        become larger than zero again, wake up that coroutine.
        """
        self._value += 1
        self._wake_up_next()