python - ProcessPoolExecutor锁定不必要的更多 future

我正在使用ProcessPoolExecutor生成子流程。
目前，我正在尝试在KeyboardInterrupt / Ctrl + C上正常退出脚本。

我正在创建由2名员工组成的池，并提交5个期货。在中断时，我试图取消尚未执行的所有将来。
如果在执行前两个期货期间中断，则该池只能取消两个期货，这意味着当前正在运行三个。但是我只有两个工人，每个过程运行5秒钟。我的未来执行什么或为什么执行？

import subprocess
from concurrent.futures import ProcessPoolExecutor
import signal
import sys


def executecommands(commands):
    # Replace signal handler of parent process, so child processes will ignore terminate signals
    original_sigint_handler = signal.signal(signal.SIGINT, signal.SIG_IGN)
    processpool = ProcessPoolExecutor(1)
    # Restore original handler, so the parent process can handle terminate signals
    signal.signal(signal.SIGINT, original_sigint_handler)
    futures = []
    try:
        for command in commands:
            futures.append(processpool.submit(executecommand, command))

        processpool.shutdown()
        print("Executed commands without interruption")
    except KeyboardInterrupt:
        print("\nAttempting to cancel pending commands..")
        for future in futures:
            if future.cancel():
                print("Cancelled one command")
            else:
                print("One command was already running, couldn't cancel")
        print("Waiting for running processes to finish..")
        processpool.shutdown()
        print("Shutdown complete")
        sys.exit(0)


def executecommand(command):
    # create a subprocess and run it
    print("running command")
    process = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    print("finished command")
    return process

if __name__ == '__main__':
    print("lets go")
    commandlist = [['/bin/sleep', '5'], ['/bin/sleep', '5'], ['/bin/sleep', '5'], ['/bin/sleep', '5'], ['/bin/sleep', '5']]
    executecommands(commandlist)

最佳答案

这是CPython实现的详细信息，但是唯一可以取消的期货是不在“调用队列”中的期货。通话队列包含接下来将要执行的所有期货。其大小为max_workers + EXTRA_QUEUED_CALLS。 （EXTRA_QUEUED_CALLS is currently set to 1。）

在您的情况下，当您开始执行前两个期货时，呼叫队列将填充3个下一个期货（max_workers为2，EXTRA_QUEUED_CALLS为1）。由于您总共只有5个期货，因此您无法取消其中的任何一个。

如果您用10个具有2个工作人员的期货填充命令列表，则可以取消最近的5个期货：

lets go
running command
running command
^C
Attempting to cancel pending commands..
One command was already running, couldn't cancel
One command was already running, couldn't cancel
One command was already running, couldn't cancel
One command was already running, couldn't cancel
One command was already running, couldn't cancel
Cancelled one command
Cancelled one command
Cancelled one command
Cancelled one command
Cancelled one command
Waiting for running processes to finish..
running command
running command
finished command
finished command
running command
finished command
Shutdown complete

关于python - ProcessPoolExecutor锁定不必要的更多 future ，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/35700273/