Top-rated recipes by Eric Snow

Smarter Default Arguments (Python)

2011-08-12T23:06:57-07:00

Python recipe 577786 by Eric Snow (default_arguments, deferred, function, functions). Revision 3.

Improved handling of mutable and deferred default arguments. After the recipe you'll find an explanation of what that means.

Works for 2.7 with minor tweaks (getfullargspec --> getargspec).

A class decorator for creating named tuples (Python)

2013-02-14T19:58:13-08:00

Python recipe 578456 by Eric Snow (metaprogramming, namedtuple). Revision 3.

This class decorator factory is useful for replacing the following:

class MyTuple(namedtuple('MyTuple', "a b c")):
    """Something special."""
    @classmethod
    def from_defaults(cls, a, b=None, c=5):
        return cls(a, b, c)

or even:

class MyTuple(namedtuple('MyTuple', "a b c")):
    """Something special."""
    def __new__(cls, a, b=None, c=5):
        return super().__new__(cls, a, b, c)

with this:

@as_namedtuple("a b c", None, c=5)
class MyTuple:
    """Something special."""

I found that I often subclass named tuples to add on some functionality or even just a nice docstring. Plus with the class syntax there's no missing that a class is bound to the name (and it's a little easier to search for the definition). When you subclass a named tuple the boilerplate involved really jumps out.

One of the main reasons Adding support for defaults to namedtuple would mitigate the need for that functionality here, but I'm not going to hold my breath on that.

One nice (though minor) thing is that you don't have to repeat the name when defining the namedtuple.

A Simpler Namespace Class (Python)

2013-02-14T17:42:24-08:00

Python recipe 578141 by Eric Snow (namespaces, object). Revision 3.

A very simple, attribute-based namespace type (and one offspring). Everyone's written one of these...

Make a Class Available in its Own Definition Body! (Python)

2011-09-15T05:36:36-07:00

Python recipe 577867 by Eric Snow (class, metaclass).

This may sound too good to be true, but see for yourself. No time machines involved. Just plug in this metaclass and look for __newclass__ in your class body. It's not perfect, but the problematic corner cases are more bark than bite.

Make a Class's Name Available in its Definition Body (Python)

2011-08-04T21:38:40-07:00

Python recipe 577813 by Eric Snow (classes).

Since a class object is created after the body is executed, it can't be available to the class body. Even the name is unavailable, at least by default. However, you can use the __prepare__() method in a metaclass to stick it in there. This recipe is a simple demonstration of how.

Simple but Powerful Grouped Constants -- A Step Toward NamedConstants (Python)

2011-08-12T23:21:32-07:00

Python recipe 577755 by Eric Snow (constants). Revision 6.

The recipe provides an easy-to-use class for a group of "constants". It provides helper functions for getting the constant values just right. It also exposes the mechanism it uses to bind the contents of an iterable into the namespace of another object.

For this binding and for the grouped constants, this recipe makes it easy to dynamically generate the mapped values you want to expose.

The next step is to make the values aware of the context in which they are bound and to strengthen their association with the group they are in. And that is one of the recipes I'm working on next!

Inherit Method Docstrings Using Only Function Decorators (Python)

2011-06-26T02:28:55-07:00

Python recipe 577746 by Eric Snow (decorator, decorators, docstring). Revision 3.

This recipe provides a descriptor and a decorator. The decorator will be used on any method in your class to indicate that you want that method to inherit its docstring.

This is useful when you are using abstract bases classes and want a method to have the same docstring as the abstract method it implements.

This recipe uses recipe #577745, the deferred_binder module.

Mixin and Overlay (Python)

2011-06-01T05:16:47-07:00

Python recipe 577730 by Eric Snow (classes, inheritance, mixins).

The Two Sides of Classes

Class inheritance either adds new attributes to a class, or changes existing attributes. However, this distinction is not made in Python's class system. This recipe is a stab at providing that separation. It takes advantage of metaclasses and class decorators to do it.

For this recipe the two sides of class inheritance are called mixins and overlays. Mixins add new attributes to a class. Overlays change existing attributes, which in the case of methods means changing the behavior of the methods. However, overlays do not add any attributes.

Separation of Concerns

Another issue with class inheritance is that abstraction rarely breaks down into perfect trees with a clear separation of concerns. Python allows multiple inheritance, which can help, but requires cooperation between classes in the diamond hierarchy. Mixins and overlays help with this problem. Hopefully, that will be evidenced by the recipe and subsequent examples.

With this approach, the main single inheritance line can focus on the core abstraction and mixins/overlays can be used to extend the classes in other directions. This does not solve all the problems regarding separation of concerns, but it solves some. I hope to address the rest in another recipe that centers around delegation through a component architecture on instances.

Interfaces

Since Python 2.6 we have had Abstract Base Classes providing a mechanism for promising what interfaces an object provides. I'll show in one of the examples how an ABC can be split into a interface portion and a mixin portion. The recipe takes advantage of the ABC functionality in the Python type system. I expect that mixins and overlays would be a good fit with one of my other recipes (Recipe 577711).

The Recipe Classes

This recipe provides a standard approach to applying mixin classes without using inheritance. This is done through a metaclass. The metaclass builds a __mixins__ attribute on the class and provides a mixes_in class decorator. Applying that decorator to the class will add the attributes in __mixins__ to the decorated class. However, if any of those names are already bound on the name then it will fail. The decorator returns the modified class.

Note: Traditional mixins are typically done through multiple inheritance, as opposed to class decorators.

This recipe also provides a companion to mixin classes, called overlays. This is done through a metaclass in exactly the same way as the mixins, but provides an __overlays__ attribute and a class decorator called overlays. In contrast to mixins, if an overlay attribute is missing on the decorated class, it will fail. This is because the decorator will return a new class that inherits from the decorated class, and overrides the attributes in __overlays__.

Note: Metaclasses are used here because we need to pull from the class namespace there. A class decorator does not afford us the same functionality without more complexity.

Note: This recipe should work fine in 2.7 with a switch to the __metaclass__ syntax.

Class-only Methods (Python)

2013-03-08T17:00:53-08:00

Python recipe 578486 by Eric Snow (metaprogramming).

We use the classmethod builtin to tie methods to the class rather than the instance. This is really useful for a variety of things, like alternate contructors. However, sometimes you don't want to expose the method on the instances, just on the class.

For instance, it usually doesn't make a lot of sense to have direct access to alternate constructors from instances; the main constructor, via the __call__ method, is only available on the class. Why? So that instances call implement their own call semantics. This is similar to why the methods (and other class attributes) on namedtuples all have names starting with an underscore.

To restrict a method just to the class, currently you must use a metaclass. Most people consider this black magic so they just use @classmethod. Furthermore, implementing a metaclass is simply more verbose than decorating a method with classmethod. Plus you have to deal with things like metaclass conflicts and the fact that metaclasses are inherited.

So...here is a sibling to classmethod that makes a method class-only without introducing metaclass complexity. You use it the same way as you would classmethod.

A Class Decorator that Adds a copy() Method (Python)

2013-02-14T20:43:20-08:00

Python recipe 578457 by Eric Snow (metaprogramming).

Here's a class decorator that adds a rudimentary copy() method onto the decorated class. Use it like this:

@copiable
class SomethingDifferent:
    def __init__(self, a, b, c):
        self.a = a
        self.b = b
        self.c = c

or like this:

@copiable("a b c")
class SomethingDifferent:
    def __init__(self, a, b, c):
        self.a = a
        self.b = b
        self.c = c

s = SomethingDifferent(1,2,3)
sc = s.copy()
assert vars(s) == vars(sc)

(Python 3.3)

A simple enum type (Python)

2013-02-13T06:53:38-08:00

Python recipe 578455 by Eric Snow (enum, metaclass). Revision 2.

Inspired by various threads[1] on the python-ideas list, here's a simple enum type. Enums are generated either via the class syntax or via Enum.make().

class Breakfast(Enum):
    SPAM, HAM, EGGS
    BACON, SAUSAGE

Breakfast = Enum.make("SPAM HAM EGGS BACON SAUSAGE")

Here are some of the features:

Enum:

inheriting from an enum inherits copies of its values.
the export() method allows for exposing an enum's values in another namespace.

Enum Values:

the underlying values within an enum are essentially useless, diminishing the temptation to rely on them.
identity is equality, like with None, True, and False.
enum values support bitwise operations within the same enum.
the result of a bitwise operation is always the same object given the same inputs.

[1] see http://mail.python.org/pipermail/python-ideas/2013-January/019003.html

ChainedList and ChainedListView: Exposing Multiple Lists as a Single Sequence (Python)

2012-07-03T21:00:02-07:00

Python recipe 578190 by Eric Snow (lists, wrapper).

Handy for composing lists without losing the underlying indepedence. The "lists" don't actually have to be lists, though for ChainedList they must be mutable.

A generic factory factory. (Python)

2012-06-11T23:31:11-07:00

Python recipe 578165 by Eric Snow (factory).

Factories are useful for producing objects without having to store certain data on the objects' type. Instances of this factory class are factories that can produce

Tracking and Manipulating the Python Import State (Python)

2012-04-22T07:27:29-07:00

Python recipe 578107 by Eric Snow (import).

This is a rough analog to the import engine described in PEP 406. Here I've called it ImportState.The focus here is on using it as a context manager to limit changes to the import state to a block of code (in a with statement). Differences from PEP 406 are described below.

A MutableMapping that Can Use Unhashable Objects as Keys (Python)

2012-04-04T20:36:59-07:00

Python recipe 578096 by Eric Snow .

The catch is that the unhashable objects aren't actually stored in the mapping. Only their IDs are. Thus you must also store the actual objects somewhere else.

A Script that Adds a Directory to sys.path Permanently (Python)

2012-02-16T23:15:11-08:00

Python recipe 578043 by Eric Snow (script, sys_path).

This script takes advantage of PEP 370, "Per user site-packages directory". It manages .pth files, which are are non-volatile (unlike manually adding to sys.path). See http://docs.python.org/library/site.html.

Emulate Keyword-Only Arguments in Python 2 (Python)

2011-11-03T18:31:52-07:00

Python recipe 577940 by Eric Snow (argument, keyword_only, kwonly, parameter).

Python 3 introduced a useful feature: keyword-only arguments. In order to get the same effect in Python 2, you must use **kwargs in your parameter list. Then, at the beginning of your function body you must manually extract what would be the keyword-only arguments once you upgrade to 3.

This recipe helps reduce the boilerplate to a single function call. You still don't get those parameters in your "def" clause (where they are more obvious), but at least it reduces the clutter.

Simple Lock Classes Based on threading.Lock() (Python)

2011-10-19T21:26:39-07:00

Python recipe 577914 by Eric Snow (lock, locking).

Nothing fancy here. Locks can take many forms. Here are a few.

A Protocol for Making Objects Immutable (Python)

2011-10-07T03:59:45-07:00

Python recipe 577895 by Eric Snow (freeze, immutable, mutable, unfreeze).

Python already provides immutable versions of many of the mutable built-in types. Dict is the notable exception. Regardless, here is a protocol that objects may implement that facilitates turning immutable object mutable and vice-versa.

See What the Builtins Are (Python)

2011-11-04T22:44:09-07:00

Python recipe 577888 by Eric Snow (builtins). Revision 3.

The built-ins are all those functions, constants, and types that are there when you need them most, like list, any, and None. This recipe is a simple way of looking at the built-ins. Check below for a thorough explanation.