如何在python中通过多进程使用tqdm?

问题描述

我正在尝试通过多个进程使用tqdm.并且行为不符合预期.我认为关键是pbar的值不会在整个过程中更新.那么如何处理这个问题呢?我也尝试使用Value手动更新pbar.n，但仍然失败.似乎tqdm不支持更新值和手动呈现.

I'm trying to use tqdm through multi processes. And the behavior is not as expected. I think the point is that the value of pbar doesn't update through the processes. So how to deal with this problem? I have also tried to use Value to update pbar.n manually, but still failed. It seems tqdm doesn't support update value and render manually.

def test(lock, pbar):
    for i in range(10000):
        sleep(0.1)
        lock.acquire()
        pbar.update()
        lock.release()

pbar = tqdm(total = 10000)
lock = Lock()
for i in range(5):
    Process(target = test, args = (lock, pbar))

推荐答案

通常，每个进程都有自己的数据，彼此独立.产生一个新进程(在Unix上调用os.fork)将创建该进程的副本.当前的过程.每个进程都获得自己的所有全局值的副本(例如pbar).每个进程的全局变量可以与变量共享相同的名称在其他过程中，但每个过程可以拥有一个独立的值.

Generally, each process has its own data, independent of every other process.Spawning a new process (which calls os.fork on Unix) creates a copy of thecurrent process. Each process obtains its own copy of all global values (such aspbar). Each process's global variables may share the same names as variablesin the other processes, but each can hold an independent value.

在您的情况下，您似乎只希望有一个pbar存在，并且所有对update应该更新那个pbar.因此，只需一个过程即可创建pbar，并使用Queue向该进程发送信号以更新pbar:

In your case it looks like you want just one pbar to exist, and all calls toupdate should update that one pbar. So create pbar in only one process,and use a Queue to send signals to that process to update pbar:

import multiprocessing as mp

SENTINEL = 1

def test(q):
    for i in range(10000):
        sleep(0.1)
        q.put(SENTINEL)

def listener(q):
    pbar = tqdm(total = 10000)
    for item in iter(q.get, None):
        pbar.update()

if __name__ == '__main__':
    q = mp.Queue()
    proc = mp.Process(target=listener, args=(q,))
    proc.start()
    workers = [mp.Process(target=test, args=(q,)) for i in range(5)]
    for worker in workers:
        worker.start()
    for worker in workers:
        worker.join()
    q.put(None)
    proc.join()

这篇关于如何在python中通过多进程使用tqdm?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！