DevOps Zone is brought to you in partnership with:

Software developer specializing in MongoDB, Python, Tornado, and Javascript, with particular interests in real-time web and new tools that get the job done with grace and alacrity. A. Jesse Jiryu is a DZone MVB and is not an employee of DZone and has posted 67 posts at DZone. You can read more from them at their website. View Full User Profile

Dawn Of The Thread

10.28.2013
| 2343 views |
  • submit to reddit

In my previous post, Night of the Living Thread, I described how a dead Python thread may think it's still alive after a fork, and how I fixed this bug in the Python standard library. But what if you need to save your code from the ravenous undead in Python 2.6? The bug will never be fixed in Python 2.6's standard library. You're on your own. In Python 2.6, no one can hear you scream.

Recall from my last post that I can create a zombie thread like this:

t = threading.Thread()
t.start()
if os.fork() == 0:
    # We're in the child process.
    print t.isAlive()

isAlive() should always be False, but sometimes it's True. The problem is, I might fork before the thread has added itself to the global _active list, so Python doesn't mark the thread as "stopped" after the fork. To fix this, I need t.start() to wait until the thread is completely initialized, so that I can't fork too soon.

Here's my solution, a SafeThread that will never rise from its grave:

class SafeThread(threading.Thread):
    def __init__(self, *args, **kwargs):
        super(SafeThread, self).__init__(*args, **kwargs)
        self.really_started = threading.Event()

    def start(self):
        super(SafeThread, self).start()
        self.really_started.wait()

    def run(self):
        self.really_started.set()
        super(SafeThread, self).run()

I create an event in the SafeThread's constructor. Then in SafeThread.start(), I call the standard Thread's start method, but I wait for the event before returning. Finally, I trigger the event in run(), which is executed in the new thread. By the time the standard Thread executes run(), it has added itself to _active: thus, we know it's safe to set the event and unblock the thread that called start(). Even if I fork immediately after that, the Safethread is in _active and can't become zombified.

You can imagine more complex scenarios that will still defeat me. For example, Thread A could start Thread B, and Thread C could fork at the wrong moment and leave Thread B zombified. For absolute safety from zombies, upgrade to Python 2.7.6 or 3.3.3 when they're released with my bugfix. Meanwhile, SafeThread is good enough for the common case where the main thread creates the background thread and then forks.

(Get ready for the gory finale, Day of the Thread, in which humanity's last hope is to figure out the quirky code review system used by the Python Software Foundation.)

Published at DZone with permission of A. Jesse Jiryu Davis, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)