Sphinx doctests and the execution namespace

The mock documentation is built with the excellent Sphinx (of course!) and as many as possible of the examples in the documentation are doctests, to ensure that the examples are still up to date for new releases.

doctests mimic executing your examples at the interactive interpreter, but they aren't exactly the same. One big difference is that the execution context for a doctest is not a real module, but a dictionary. This is particularly important for mock examples, because the following code will work at the interactive interpreter but not in a doctest:

>>> from mock import patch
>>> class Foo(object):
...      pass
...
>>> with patch('__main__.Foo') as mock_foo:
...   assert Foo is mock_foo
...

The name (__name__) of the doctest execution namespace is __builtin__, but this is a lie. The namespace is a dictionary, internal to the DocTest. Whilst executing doctests under Sphinx, the real __main__ module is the sphinx-builder script.

To get the example code above working I either need to rewrite it (and make it less readable - probably by shoving the class object I'm patching out into a module), or I need to somehow make the current execution context into __main__.

Fortunately the Sphinx doctest extension provides the doctest_global_setup config option. This allows me to put a string into my conf.py, which will be executed before the doctests of every page in my documentation (doctests from each page share an execution context).

I solved the problem by creating a proxy object that delegates attribute access to the current globals() dictionary. I shove this into sys.modules as the __main__ module (remembering to store a reference to the real __main__ so that it doesn't get garbage collected). When patch accesses or changes an attribute on __main__ it actually uses the current execution context.

Here's the code from conf.py:

doctest_global_setup = """
import sys
import __main__

# keep a reference to __main__
sys.modules['__main'] = __main__

class ProxyModule(object):
    def __getattr__(self, name):
        return globals()[name]
    def __setattr__(self, name, value):
        globals()[name] = value
    def __delattr__(self, name):
        del globals()[name]

sys.modules['__main__'] = ProxyModule()
"""

doctest_global_cleanup = """
sys.modules['__main__'] = sys.modules['__main']
"""

The corresponding doctest_global_cleanup option restores the real __main__ when the test completes.

Note

In the comments Nick Coghlan suggests a simplification for the ProxyModule:

class ProxyModule(object):
    def __init__(self):
        self.__dict__ = globals()