在父进程中捕获子进程异常

问题描述

我正在创建多个分别运行爬虫的进程.我想确保如果爬虫进程中有一些异常，我能够在父进程中捕获它.这是流程创建代码:

I am creating multiple processes which are running crawlers separately. I want to ensure that if there is some exception in crawler process, I am able to catch it in Parent Process.Here is the process creation code:

try:
    caching_process = Process(target=run_crawler_process, args=(Config.CRAWLER_NAME, locations,
                                                               city_payloads_map, cycle_count))
    caching_process.start()
except Exception as processException:
    raise processException

推荐答案

你不能用 Process 对象做到这一点.multiprocessing.Pool 或 concurrent.futures.ProcessPoolExecutor 允许这样做.

You cannot do that with Process objects. The multiprocessing.Pool or the concurrent.futures.ProcessPoolExecutor allow to do that.

pool = multiprocessing.Pool()
task = pool.apply_async(run_crawler_process, (Config.CRAWLER_NAME, locations,c ity_payloads_map, cycle_count))

try:
    task.get()
except Exception as error:
    print("Error while processing task: %s" % error)

这篇关于在父进程中捕获子进程异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！