Python Zone is brought to you in partnership with:

Mitch Pronschinske is the Lead Research Analyst at DZone. Researching and compiling content for DZone's research guides is his primary job. He likes to make his own ringtones, watches cartoons/anime, enjoys card and board games, and plays the accordion. Mitch is a DZone Zone Leader and has posted 2576 posts at DZone. You can read more from them at their website. View Full User Profile

PyPy Opens Up a "Blackhole"

06.28.2010
| 8312 views |
  • submit to reddit
Over the weekend, the PyPy blog announced the release of PyPy 1.3, which features a much faster JIT compiler and support for C-coded CPython extension modules.  The JIT compiler in PyPy has also been stabilized and many user issues and bugs have been addressed.  As PyPy slithers into mainstream attention, the main theme of this release is stability.

While the vast majority of Python developers currently use CPython, which is downloaded at python.org, another Python implementation written in Python - PyPy - has gained wider support from developers.  Some see PyPy as an even better platform for Python's growth than CPython.  One of the main reasons why PyPy is so well suited for growth is because changes can be made to the language without having to update the JIT.  The JIT backend is abstracted so that it is almost totally separate from the interpreter.  You can maintain the speed of the JIT while making updates to the language, therefore you don't need to be an assembly expert to write new bytecode.  

CPython modules
The CPython extension modules (written in C) are the only feature that breaks with the stability theme.  This support is considered alpha quality because it's unlikely that unmodified C extensions will work out of box (due to missing functions or refcounting details).  This support is disabled by default, so to turn it on you need to type:
import cpyext
Then you can try to import a .so file.  The libraries are source-compatible but not binary-compatible, so you'll need to recompile binaries using this, for example (details will vary depending on your build system):
pypy setup.py build
Be sure to include the above line at the beginning of setup.py or put it in your PYTHONSTARTUP.

The Blackhole Compiler
The secret behind the JIT compiler's speedup starts with the fact that its executable contains three interpreters.  They include a regular Python interpreter, for code that can't be JIT-compiled, a tracing interpreter, for JIT-compiling the most executed loops, and the "blackhole" interpreter.  The speed of compilation has increased substantially thanks to a redesigning of the blackhole interpreter as a time-critical component.  It is designed to finished the execution of the current Python opcode and it essentially works like the JIT compiler, but without the overhead of tracing (it doesn't need to box all values).  This post explains the blackhole compiler more thoroughly.  

PyPy has a number of benchmarks showing its speed compared to CPython.  PyPy is also able to automatically generate C code along with JVM and .NET versions of the interpreter (no need to write in Jython or IronPython).  See what PyPy projects are currently in progress during Google's Summer of Code.