]> code.ossystems Code Review - openembedded-core.git/commitdiff
lib/oe/utils: Fix hang in multiprocess_launch()
authorRichard Purdie <richard.purdie@linuxfoundation.org>
Sat, 16 Feb 2019 14:01:56 +0000 (14:01 +0000)
committerRichard Purdie <richard.purdie@linuxfoundation.org>
Sun, 17 Feb 2019 10:18:48 +0000 (10:18 +0000)
If large results values are returned by the subprocesses, we can hit a deadlock
where the subprocess is trying to write data back to the parent, the pipe is full
and the parent is waiting for the child to exit.

Avoid this by calling the update() method which would trigger reading a result
from the child, avoiding the deadlock. The issue is described in
https://bugs.python.org/issue8426

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meta/lib/oe/utils.py

index 0c1d48a209eab3ed977503d1ff72f87afe852d6b..3a496090f3af0a7a360a061648ce93d926bcc84f 100644 (file)
@@ -307,6 +307,10 @@ def multiprocess_launch(target, items, d, extraargs=None):
             p.start()
             launched.append(p)
         for q in launched:
+            # Have to manually call update() to avoid deadlocks. The pipe can be full and
+            # transfer stalled until we try and read the results object but the subprocess won't exit
+            # as it still has data to write (https://bugs.python.org/issue8426)
+            q.update()
             # The finished processes are joined when calling is_alive()
             if not q.is_alive():
                 if q.exception: