"Design Patterns" [GoF] prescribes two commandments to live by:
1) Program to an interface, not an implementation. 2) Favor object composition over class inheritance.
This implies that every class should have a defined interface, and that every class should be a composition of objects. However, there is often a large barrier to object composition - sometimes a class needs to have the interface of its components. You don't want to extend the functionality of the component, you just want its interface, and you want all calls to that interface to be sent to the component object. In this case, it is so much easier to inherit even though you are not really extending the functionality of the parent class (this especially happens when writing GUI applications, where the components have huge APIs). If you bite the bullet and make a composite, you end up writing a lot of wrapper functions (see Figure 1). (A wrapper function has the same function signature as the function it is wrapping, and it simply calls the same function in the target object.) No one likes to write wrapper functions - its dull, boring, menial work. I've thrown out plenty of my own designs because they would have required me to write wrappers for a huge number of functions (for example, the wxWindow class has over 130 member functions. See http://www.wxWindows.org), so I end up inheriting even though I know it is "wrong".
There is no way around this problem in C++ and Java (though maybe an IDE vendor could put in a tool to automatically generate wrapper functions). Amazingly enough, however, Python allows you to modify classes at run time. This means it is possible to make Python automatically write these wrapper (or proxy) functions for you.
Download
Copy to clipboard1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | **********************************************************************
Begin Figure 1
**********************************************************************
// Written in Pseudo-Java/C++
interface InterfaceB {
functionB1();
functionB2();
}
class B implements InterfaceB {
public:
functionB1() {
doStuff;
}
functionB2() {
doMoreStuff;
}
}
class A implements InterfaceB {
private:
B m_b;
public:
functionB1() {
m_b->functionB1();
}
functionB2() {
m_b->functionB2();
}
}
**********************************************************************
End Figure 1
**********************************************************************
**********************************************************************
Begin Figure 2
**********************************************************************
#
# File: ProxyInterfaceOf.py
#
import types
def createProxyFunction(proxyObject):
proxyObjectClassDict = proxyObject.__class__.__dict__
# Create a function to forward requests to TargetObject.TargetFunction
def ProxyFunction(self, *moreArgs, **evenMoreArgs):
TargetObject, TargetFunction = proxyObjectClassDict["__functionMap"][ProxyFunction]
return TargetFunction(TargetObject, *moreArgs, **evenMoreArgs)
return ProxyFunction
def proxyInterfaceOf(ProxyObject, TargetObject):
proxyObjectClassDict = ProxyObject.__class__.__dict__
targetObjectClassDict = TargetObject.__class__.__dict__
# Go through all the class attributes of the TargetObject
for attribute in targetObjectClassDict:
# If the attribute is a user defined function and not private.
if ( (type(targetObjectClassDict[attribute]) is types.FunctionType )
and (not attribute.startswith("__"))):
# Create a function that forwards a function call to the TargetObject
ProxyFunction = createProxyFunction(ProxyObject)
# We need to be able to figure out the name of the function
# the user is trying to call, so we create a mapping of
# [ ProxyFunction : ( TargetObject, TargetFunction ) ]
# Here we create the function map if it doesnt exist.
if (not proxyObjectClassDict.has_key("__functionMap")):
proxyObjectClassDict["__functionMap"] = dict()
# Remember the TargetObject and the TargetFunction.
TargetFunction = targetObjectClassDict[attribute]
proxyObjectClassDict["__functionMap"][ProxyFunction] = (TargetObject, TargetFunction)
# Create the same function in the ProxyObject that is in the TargetObject.
proxyObjectClassDict[attribute] = ProxyFunction
**********************************************************************
End Figure 2
**********************************************************************
**********************************************************************
Begin Figure 3
**********************************************************************
from ProxyInterfaceOf import proxyInterfaceOf
#
# Simple Example of using proxyInterfaceOf()
#
class bar:
def __init__(self):
myFoo = foo(23)
proxyInterfaceOf(self, myFoo) # I proxy all public foo functions
class foo:
def __init__(self, appSpec):
self.__appSpec = appSpec
def func1(self):
print "func1 :: internal data member self.__appSpec = " + str(self.__appSpec)
def func2(self, arg1, arg2):
print "func2 :: arg1 = " + str(arg1) + ", arg2 = " + str(arg2)
myBar = bar()
#print bar.__dict__
myBar.func1()
myBar.func2(34, "woo")
**********************************************************************
End Figure 3
**********************************************************************
Output from Figure 3:
func1 :: internal data member self.__appSpec = 23
func2 :: arg1 = 34, arg2 = woo
|
The code takes advantage of the meta-programming aspects of Python. A Python class can be modified at run-time. For example, member functions or member variables can be added or removed on the fly. Most of the time this isn't very useful and is usually extremely dangerous. However, if used in the right way, it allows us to develop new programming patterns that may not otherwise be possible.
Every object in python has a .__class__.__dict__ variable which contains a dictionary of the object's namespace, which includes member functions and member variables. We can add a member function or member variable to the class by adding it to this dictionary.
Before we start on the code, let's define some terms so that we are all on the same page. The container class will be called ProxyClass, and an instantiated object of that class will be called ProxyObject. The component object inside the ProxyClass that we want to wrap will be called the TargetObject and its class will be called TargetClass.
The API for this code is just one function named proxyInterfaceOf(ProxyObject, TargetObject). The result of calling this function is that all public functions of the TargetObject are wrapped by the ProxyObject. For example, if an object Foo started with functions named f1() and f2(), and an object Bar had functions named f3() and f4(), after calling proxyInterfaceOf(Foo, Bar), Foo would then have functions f1(), f2(), f3(), and f4(). The functions f3() and f4() in the Foo object would simply be wrappers which call the function of the same name in object Bar.
The proxyInterfaceOf() function works by iterating through each public member function in the TargetObject, creating ProxyFunction()s in the ProxyObject that will forward requests to the TargetObject. Note that a ProxyFunction() is a dynamically created function - you don't encounter this sort of thing every day. The code for these functions is written when the program executes. A function named createProxyFunction() generates the code for each ProxyFunction(), and every ProxyFunction() contains exactly the same code. If the code is the same every time, how does each ProxyFunction() know which function in the TargetObject to call? The answer is that we use a private data member in the ProxyObject to keep a map of:
[ ProxyFunction : ( TargetObject, TargetFunction ) ].
Let's call this the __functionMap. Now when the ProxyFunction() is called, it looks up itself in this map (it can do this because each ProxyFunction() has its own unique memory address), finds the cooresponding TargetObject and TargetFunction, and makes the call with all the arguments that it (the ProxyFunction()) received.
How is this __functionMap created? This is where things can be a little confusing (as if they werent already?) The __functionMap is created in proxyInterfaceOf(), and it is added as a data member to the ProxyObject (once again using dynamic programming). The weird thing is that ProxyFunction must exist before an entry can be placed in the __functionMap. Thus, the ProxyFunction() is generated by createProxyFunction(), and then the address of that ProxyFunction() is placed in the __functionMap, along with its targets. This is a little clearer when you think that the code for the ProxyFunction() is being written, not executed, in createProxyFunction(). The final step in this whole process is to add ProxyFunction() to the class dictionary of the ProxyObject.
To summarize, there are two separate run-time stages to this paradigm: Initialization and Invocation.
Initialization Input: ProxyObject, TargetObject 1) Create __functionMap as private data member in ProxyObject 2) Create a ProxyFunction() 3) Map the ProxyFunction() to (TargetObject,TargetFunction) in the __functionMap 4) Add the ProxyFunction() to ProxyObject's class dictionary, using the TargetFunction as the dictionary key
Invocation When a ProxyFunction() is called, it does the following: 1) Get the class dictionary for the ProxyObject 2) From the class dictionary, get the private data member __functionMap 3) From the __functionMap, look up ProxyFunction()'s own address to get the corresponding TargetObject and TargetFunction 4) Call TargetObject.TargetFunction with the arguments given to the ProxyFunction() 5) Return whatever TargetObject.TargetFunction returns
Usage
(See Figure 3 for a usage example) Make use of proxyInterfaceOf() with extreme caution. You should only use this function in the __init__ stage of an object. You have to make sure that none of the functions you write clash with the names of the TargetFunctions, otherwise the ones in your class will be replaced by the one in the TargetObject.
One of the problems with using the proxyInterfaceOf() function is that it leaves no traces of documentation. You must make sure to document that a class implements the interface of another object.
Possible Improvements
The code could be expanded to optionally proxy all the TargetObject's parent class functions as well, or proxy just a subset of the TargetObject's member functions instead of the entire set. There should also be warnings or better default behavior when there is a namespace clash - that is, when a function being added has the same name as a function that already exists in the class.
There also may be better ways to code proxyInterfaceOf(), perhaps using Python's "compile/exec" functions so that the mapping problem does not occur. As some of you may have noted, doing lookups for each function call will slow things down. If the ProxyFunction()s could be generated so that this lookup did not have to occur, this would speed things up.
Conclusion
We have seen one way in which meta-programming can be used for the force of good. We can uphold the Gang of Four's two main tenets without causing ourselves undo misery. Python has allowed us this freedom - it gives us the ability to explore options like these. This new functionality allows us to rid ourselves of inheritance forever. May our sins be absolved and may we always favor object composition over inheritance.
All suggestions, comments, and bugs are welcome.
References
[Gof] "Design Patterns", Gamma, et al., Addison-Wesley, 1995.
Java Dynamic Proxies. In Java this is called a Dynamic Proxy. They were demonstrated in 1999 and added to the language core with Java 1.3.
http://java.sun.com/j2se/1.3/docs/guide/reflection/proxy.html#api http://www.javaworld.com/javaworld/jw-11-2000/jw-1110-proxy.html http://www.google.com/search?hl=en&lr=&ie=ISO-8859-1&q=java+dynamic+proxy&btnG=Google+Search
Easier way. Why not just use __getattr__ (and __setattr__ if you write to members of the target):
Easier way. Oops - embarassing error corrected!
Why not just use __getattr__ (and __setattr__ if you write to members of the target):
Another embarrassing error... Running your code, I get a segmentation fault.
It really should be something like:
But, then you don't get the nice listing of function names when you do "dir(foo)" like to originally posted proxy code...
BUT, the original proxy code doesn't account for property objects, just function objects. If you're going to program to an interface, the property objects are just as important as the functions...
Response to "Easier ways" Hi guys - Thanks for the alternative code! Perhaps both ways should be implemented, for as Tracy Ruggles points out - the function names should appear in the class dict, and they should also be part of the property objects.
-Paul
Java Dynamic Proxies. Hi Ry4an -
Thanks for the links...I've never heard of Dynamic Proxies before, as usual I am 3 years behind and had to reinvent the wheel. The material on the "Dynamic Proxy Classes" web page will be great for improving the design for python. I've also noticed that the Python Cookbook recipe "Synchronizing All Methods on an Object" (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65202) has a good function for recursively finding all the functions in the superclasses of an object, which is what this recipe also needs. Anyhow, thanks a bunch for the feedback!
-Paul
Why not use metaclass? Have you considered using __metaclass__ instead? Suppose you want to create a large number of these proxy objects. Your method would mean all the wrapping happens for each instance, true? If you used a metaclass, you could do the wrapping before your wrapping class itself is created.