Welcome, guest | Sign In | My Account | Store | Cart

Let's say you're at a company and you're deploying a package called "tools" on all production boxes. Normally, code on these boxes could "import tools" to use this package.

However, over time the API to tools will evolve as you release new versions that add functionality and fix bugs. If there's lots of company code that "imports tools", then you're stuck with backward compatibility forever.

This recipe presents a method for letting client code specify on one line which version of "tools" they wish to use -- and then import from the tools package as normal. Behind the scenes, the recipe is making sure that the client works with the version of the package that they requested. If the client ever wants to change versions, it's a one-line change at the top of their code.

Python, 88 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
# In tools/, make a subdirectory _stable_1_0_2 containing the 1.0.2 stable 
# version of the "tools" package.  Make _stable_1_0 and _stable_1 symlinks to that directory,
# assuming you don't have any newer 1.0.x or 1.x releases. Also make a _latest_1_0_2 containing
# the latest (possibly unstable) 1.0.2.x version, and symlink _latest_1_0 and _latest_1 to it.
# Finally, create a _exact_1_0_2_4 if you have, for example, a 1.0.2.4 release of "tools".
#
# Each of those subdirectories is a package that can be loaded as "tools" by the client.



# File: tools/__init__.py

if '_original__import__' not in locals():
    _original__import__ = __import__

def myimport(name, theglobals=None, thelocals=None, fromlist=None, level=-1):
    if name.split('.')[0] != "tools":
        return _original__import__(name, theglobals, thelocals,
                fromlist, level)

    if not currentversion:
        raise Exception("After importing tools, you must "
                "load a specific version by typing something like "
                "'tools = tools.loadstable(\"0.1\")' .")

    def withversion(name):
        """
        Turn "tools[.anything]" into "<versionname>[.anything]" , where
        <versionname> is something like 'tools._stable_0_1' .
        """
        parts = name.split('.')
        parts[0] = currentversion.__name__ # eg 'tools._stable_0_1'
        return '.'.join(parts)

    # "import tools[.whatever.whatever]"
    if not fromlist:
        # use <versionname> instead of "tools", but otherwise execute the 
        # import as expected
        _original__import__(withversion(name), theglobals, thelocals,
                fromlist, level)
        # but return <currentversion> as the top-level package instead of
        # returning tools as expected
        return currentversion

    # "from tools[.whatever.whatever] import thing": instead,
    # "from <versionname>[.whatever.whatever] import thing", and 
    # return thing as expected
    return _original__import__(withversion(name), theglobals, thelocals,
            fromlist, level)
__builtins__["__import__"] = myimport

def loadstable(ver):
    return _loadversion(ver, prefix="_stable_")

def loadunstable(ver):
    return _loadversion(ver, prefix="_latest_")

def loadexact(ver):
    return _loadversion(ver, prefix="_exact_")

def _loadversion(ver, prefix):
    targetname = prefix + ver.replace('.', '_')
    mainpackage = _original__import__("tools", globals(), locals(),
        [targetname])
    global currentversion
    currentversion = getattr(mainpackage, targetname)

    # Let users change versions after choosing this one
    currentversion.loadstable = loadstable
    currentversion.loadunstable = loadunstable
    currentversion.loadexact = loadexact

    return currentversion

currentversion = None




# In the user's code:

# This line makes "tools" be the stable 1.0.2 release.
import tools; tools = tools.loadstable('1')

# Now the user can work with tools as normal
from tools import foo
import tools.bar.bim
from tools.baz import bonk

I have put this in production at the company that I work for. For more stuff I've built, see http://www.sorryrobot.com.

How it works

The magic line in the client's code is:

import tools; tools = tools.loadstable('1')

loadstable() imports the actual package "tools._stable_1" and returns it, assigning it to the variable "tools". importing tools also installs a new version of __import__.

Then when the user does

import tools.bar.bim

the new __import__ loads the actual package "tools._stable_1.bar.bim" instead; the "tools" variable is still pointing to the actual package "tools._stable_1", but has bar.bim loaded in its namespace.

Some benefits:

You can make non-backwards-compatible changes with impunity. If you don't provide _stable_1, but only provide _stable_1_2, then users have to loadstable('1.2') -- and when you want to break compatibility, you release version 1.3. (If there were a _stable_1, then they could loadstable('1') and your 1.3 release would break their code.)

The user can loadstable('1.2') and when you release bugfix 1.2.0.1, their code automatically gets it.

Users can switch between stable and unstable versions of the code with a single line.

If the user decides he wants to use 1.3 instead, it's a single one-line change at the top of the file. Even if way down in his code he dynamically decides to "import tools.specialtything", it will load the 1.3 version with no change to that line of code.

Thoughts

The current implementation lets you specify your version once in your main file; all other files that "import tools" will get the version you originally specified. This might not be the best way to go about this -- maybe it's better to make each file have to specify its version, and to barf if two files mismatch. That way, the coder can always be sure of what version he's running by looking at the top of the file.