受限於 GIL(全域解譯器鎖),CPython 單進程只能佔用一顆 CPU 核心。multiprocessing 模組以進程級並行繞過 GIL,API 設計極似 threading,讓並發遷移極為平滑。本文由淺入深示範三種常用模式。
import multiprocessing, time
def worker(msg):
for i in range(3):
print(msg)
time.sleep(1)
if __name__ == "__main__":
p = multiprocessing.Process(target=worker, args=("hello",))
p.start()
p.join()
print("Sub-process done.")
⚠️ 必須置於 if __name__ == "__main__": 保護區塊,否則 Windows 會無限遞迴生成進程。
if __name__ == "__main__":
pool = multiprocessing.Pool(processes=4) # 最多 4 進程並發
for i in range(10):
pool.apply_async(worker, (f"hello {i}",))
pool.close()
pool.join()
print("Sub-process(es) done.")
apply_async:非阻塞提交,立即返回 AsyncResultpool.close():拒絕新任務pool.join():等待所有任務完成def worker(msg):
for i in range(3):
print(msg)
time.sleep(1)
return "done " + msg
if __name__ == "__main__":
pool = multiprocessing.Pool(processes=4)
results = []
for i in range(10):
results.append(pool.apply_async(worker, (f"hello {i}",)))
pool.close()
pool.join()
for res in results:
print(res.get()) # 取得返回值,若進程拋出異常會在此重拋
print("Sub-process(es) done.")
| 場景 | 解決方案 |
|---|---|
| 需共享狀態 | multiprocessing.Manager 提供 dict/list/Value/Array 代理物件 |
| 大量資料傳遞 | 使用 multiprocessing.Queue 或共享記憶體 Array/RawArray |
| Windows 無 fork | 確保進入點在 if __name__ == "__main__":,避免重複 import 執行 |
| 異常傳遞 | AsyncResult.get() 會重拋子進程異常,配合 traceback 診斷 |
| 進程數調優 | CPU 密集型設為核心數;I/O 密集型可設 2-4 倍核心數 |