使用 asyncio (Python 3.4+) 异步接收长时间运行的 shell 命令的输出?

问题描述

我试图弄清楚如何以非阻塞方式简单地启动一些长时间运行的 shell 命令，并在它们完成时异步处理它们的输出，按照它们完成的顺序，即使这是他们开始的另一个顺序，使用 Python 3.4 及更高版本中可用的 asyncio python 库.

I'm trying to figure out how to simply start a number of long running shell commands in a non-blocking way, and asynchronously handle their output when they finish, in the order they finish, even if that is another order than they started, using the asyncio python library available in Python 3.4 and forward.

我找不到这样做的简单示例，即使在 asyncio 文档本身中，这似乎也是相当低级的.

I couldn't find a simple example of doing this, even in the asyncio documentation itself, which also seems to be quite low-level.

推荐答案

使用 get_lines() 协程，以异步方式获取 shell 命令输出并将协程传递给 asyncio.as_completed()，按照它们完成的顺序获取结果:

Use get_lines() coroutines, to get shell commands output asynchronously and pass the coroutines to asyncio.as_completed(), to get the results in the order they finish:

#!/usr/bin/env python3.5
import asyncio
import sys
from asyncio.subprocess import PIPE, STDOUT

async def get_lines(shell_command):
    p = await asyncio.create_subprocess_shell(shell_command,
            stdin=PIPE, stdout=PIPE, stderr=STDOUT)
    return (await p.communicate())[0].splitlines()

async def main():
    # get commands output concurrently
    coros = [get_lines('"{e}" -c "print({i:d}); import time; time.sleep({i:d})"'
                       .format(i=i, e=sys.executable))
             for i in reversed(range(5))]
    for f in asyncio.as_completed(coros): # print in the order they finish
        print(await f)


if sys.platform.startswith('win'):
    loop = asyncio.ProactorEventLoop() # for subprocess' pipes on Windows
    asyncio.set_event_loop(loop)
else:
    loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()

这篇关于使用 asyncio (Python 3.4+) 异步接收长时间运行的 shell 命令的输出?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！