Python Closures and Decorators (Pt. 1)

Since I, in retrospect, made the wrong choice when cutting down a Python course to four hours and messed up the decorator exercise, I promised the attendants that I'd make a post about closures and decorators and explain it better - this is my attempt to do so.

Functions are objects, too. In fact, in Python they are First Class Objects - that is, they can be handled like any other object with no special restrictions. This gives us some interesting options, and I'll try to move through them from the bottom up.

A very basic case of using the fact that functions are objects is to use them as you would a function pointer in C; pass it into another function that will use it. To illustrate this, we'll take a look at the implementation of a repeat function - that is, a function that accepts another function as argument together with a number, and then calls the passed function the specified number of times:

>>> #A very simple function
>>> def greeter():
...   print("Hello")
...
>>> #An implementation of a repeat function
>>> def repeat(fn, times):
...   for i in range(times):
...     fn()
...
>>> repeat(greeter, 3)
Hello
Hello
Hello
>>>

This pattern is used in a large number of ways - passing a comparison function to a sorting algorithm, passing a decoder function to a parser, and in general specializing the behaviour of a function, or passing a specific parts of a job to be done into a function that abstracts the work flow (i.e. sort() knows how to sort lists, compare() knows how to compare elements).

Functions can also be declared in the body of another function, which gives us another important tool. In the most basic case, this can be used to "hide" utility functions in the scope of the function that uses them:

>>> def print_integers(values):
...   def is_integer(value):
...     try:
...       return value == int(value)
...     except:
...       return False
...   for v in values:
...     if is_integer(v):
...       print(v)
...
>>> print_integers([1,2,3,"4", "parrot", 3.14])
1
2
3

 This may be useful, but is hardly in itself a very powerful tool. Compared with the fact that functions can be passed as arguments however, we can add behaviours to function after they are constructed, by wrapping them in another function. A simple example would be to add a trace output to a function:

>>> def print_call(fn):
...   def fn_wrap(*args, **kwargs): #take any arguments
...     print("Calling %s" % (fn.func_name))
...     return fn(*args, **kwargs) #pass any arguments to fn()
...   return fn_wrap
...
>>> greeter = print_call(greeter) #wrap greeter
>>> repeat(greeter, 3)
Calling fn_wrap
Hello
Calling fn_wrap
Hello
Calling fn_wrap
Hello
>>>
>>> greeter.func_name
'fn_wrap'

 As you can see, we can replace the greeter function with a new function that uses print to log the call, and then calls the original function. As seen on the last two rows of the example, the function name of the function reflects that it has been replaced, which may or may not be what we wanted. If we want to wrap a function while keeping the original name, we can do so by adding a row to our print_call function:

>>> def print_call(fn):
...   def fn_wrap(*args, **kwargs): #take any arguments
...     print("Calling %s" % (fn.func_name))
...     return fn(*args, **kwargs) #pass any arguments to fn()
...   fn_wrap.func_name = fn.func_name #Copy the original name
...   return fn_wrap

Since this is rapidly turning into a very long post, I'll stop here and return tomorrow with part two, where we'll look at closures, partials, and (finally) decorators.

Until then, if this is all new to you, use print_call as a base to create a function that will print function name and arguments passed before calling the wrapped function, and the function name and return value before returning.