请教一下大佬多进程的超时问题

import time import random from multiprocessing import Pool def fun(name): print(f'{name} 运行') time.sleep(random.randrange(5,15)) print(f'{name} 运行结束') if __name__ == '__main__': pool = Pool(processes=2) names = ['anne','alice','biantai','haha'] for name in names: pool.apply_async(fun,(name,)) pool.close() pool.join() print('主线程')

我想要如果进程时间超过 10s 就结束掉，打印超时提示，然后让新进程执行，超时的进程放后面再执行，最后让所有的进程都在 10s 内成功运行完成，有什么优雅的方法吗？

Supplement 1 Jun 17, 2021

import multiprocessing from multiprocessing.dummy import Pool as ThreadPool from functools import partial import time import random def worker(name): begin = time.time() print(f'{name} 运行') time.sleep(random.randrange(5, 15)) end = time.time() print(f'{name} 运行结束,运行了{end - begin}秒') return end - begin def collectMyResult(result): pass # print("Got result {}".format(result)) def abortable_worker(func, *args, **kwargs): timeout = kwargs.get('timeout', None) name = kwargs.get("name",None) p = ThreadPool(1) res = p.apply_async(func, args=args) try: out = res.get(timeout) # Wait timeout seconds for func to complete. return out except multiprocessing.TimeoutError: print(f"{name}运行超时，已终止") raise if __name__ == "__main__": pool = multiprocessing.Pool(maxtasksperchild=1,processes=2) names = ["赵","钱","孙","李"] # list of arguments for name in names: abortable_func = partial(abortable_worker, worker, timeout=10,name=name) pool.apply_async(abortable_func, args=nae, callback=collectMyResult) pool.close() pool.join()

解决了超时就终止的问题了

name

pool

import

2 replies 2021-06-11 09:48:29 +08:00

zhengxiaowai

Jun 11, 2021

https://docs.python.org/zh-cn/3/library/multiprocessing.html#using-a-pool-of-workers

有 timeout 参数，会抛出异常，另外 "最后让所有的进程都在 10s 内成功运行完成" 你得有足够的进程数量

fmdxx1991

Jun 11, 2021

@zhengxiaowai 这个用 get 尝试过，发现同时运行 2 个子进程的时候，用 get(timeout)会堵塞主进程，直到超时才会执行下一步，这个就有问题了

``` python
for name in names:
p = pool.apply_async(fun,(name,))
p.get(timeout=10)
```

这样写的话，就是每次运行一个进程，然后等待超时，那就没必要用多进程了

放外面的话, 也有很大的问题....