Aiohttp，Asyncio:RuntimeError:事件循环已关闭

问题描述

我有两个脚本，scraper.py 和 db_control.py.在scraper.py中，我有这样的东西:

I have two scripts, scraper.py and db_control.py. In scraper.py I have something like this:

...
def scrape(category, field, pages, search, use_proxy, proxy_file):
    ...
    loop = asyncio.get_event_loop()

    to_do = [ get_pages(url, params, conngen) for url in urls ]
    wait_coro = asyncio.wait(to_do)
    res, _ = loop.run_until_complete(wait_coro)
    ...
    loop.close()

    return [ x.result() for x in res ]

...

在 db_control.py 中:

And in db_control.py:

from scraper import scrape
...
while new < 15:
    data = scrape(category, field, pages, search, use_proxy, proxy_file)
    ...
...

理论上，刮板应该在未知时间启动，直到获得足够的数据.但是当 new 不是 imidiately >15 然后出现这个错误:

Theoretically, scraper should be started unknown-times until enough of data have been obtained. But when new is not imidiatelly > 15 then this error occurs:

  File "/usr/lib/python3.4/asyncio/base_events.py", line 293, in run_until_complete
self._check_closed()
  File "/usr/lib/python3.4/asyncio/base_events.py", line 265, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

但是如果我只运行一次 scrape() 脚本就可以正常工作.所以我想重新创建 loop = asyncio.get_event_loop() 存在一些问题，我尝试过 this 但没有任何改变.我该如何解决这个问题?当然，这些只是我的代码片段，如果您认为问题可能出在其他地方，完整代码可用 此处.

But scripts works just fine if I run scrape() only once. So I guess there is some problem with recreating loop = asyncio.get_event_loop(), I have tried this but nothing changed. How I can fix this? Of course those are just snippets of my code, if you think problem can be elsewhere, full code is available here.

推荐答案

方法 run_until_complete, run_forever, run_in_executor、create_task, call_at 明确检查循环并在关闭时抛出异常.

Methods run_until_complete, run_forever, run_in_executor, create_task, call_at explicitly checkthe loop and throw exception if it's closed.

引用自文档 - BaseEvenLoop.关闭:

Quote from docs - BaseEvenLoop.close:

这是幂等且不可逆的

---

除非你有一些(好的)理由，你可以简单地省略关闭线:

---

Unless you have some(good) reasons, you might simply omit the close line:

def scrape(category, field, pages, search, use_proxy, proxy_file):
    #...
    loop = asyncio.get_event_loop()

    to_do = [ get_pages(url, params, conngen) for url in urls ]
    wait_coro = asyncio.wait(to_do)
    res, _ = loop.run_until_complete(wait_coro)
    #...
    # loop.close()
    return [ x.result() for x in res ]

如果你想每次都有一个全新的循环，你不必手动创建它并设置为默认值:

If you want to have each time a brand new loop, you have t create it manually and set as default:

def scrape(category, field, pages, search, use_proxy, proxy_file):
    #...
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    to_do = [ get_pages(url, params, conngen) for url in urls ]
    wait_coro = asyncio.wait(to_do)
    res, _ = loop.run_until_complete(wait_coro)
    #...
    return [ x.result() for x in res ]

这篇关于Aiohttp，Asyncio:RuntimeError:事件循环已关闭的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！