Welcome, guest | Sign In | My Account | Store | Cart
1

Use self.super(p, *kw) instead of super(cls, self).func(p, *kw).

Latest version (much faster, with docs, tests, Pyrex version) is available at: http://members.optusnet.com.au/tcdelaney/python.html#autosuper

Python, 127 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
import sys

class _super (object):
    """
    Wrapper for the super object.

    If called, a base class method of the same name as the current method
    will be called. Otherwise, attributes will be accessed.
    """

    __name__ = 'super'

    # We want the lowest overhead possible - both speed and memory
    # We need to put the mangled name in as Python 2.2.x didn't work
    # with private names specified in __slots__.
    __slots__ = ('_super__super', '_super__method')

    def __init__(self, super, name):
        object.__setattr__(self, '_super__super', super)

        try:
            object.__setattr__(self, '_super__method', getattr(super, name))
        except AttributeError:
            object.__setattr__(self, '_super__method', name)

    def __call__(self, *p, **kw):
        """
        Calls the base class method with the passed parameters.
        """

        # We want fastest performance in the normal case - i.e. calling
        # self.super(*p, **kw). We don't care as much about how long it
        # takes to fail.
        method = object.__getattribute__(self, '_super__method')

        try:
            return method(*p, **kw)
        except TypeError:
            if type(method) is not str:
                raise

        # This should throw an AttributeError, but they could have modified
        # the base class
        super = object.__getattribute__(self, '_super__super')
        method = getattr(super, method)
        object.__setattr__(self, '_super__method', method)
        return method(*p, **kw)

    def __getattribute__ (self, name):
        """
        Gets a base class attribute.
        """

        super = object.__getattribute__(self, '_super__super')

        try:
            return getattr(super, name)
        except (TypeError, AttributeError):
            # Cannot call like self.super.super - produces inconsistent results.
            if name == 'super':
                raise TypeError("Cannot get 'super' object of 'super' object")

            raise

    def __setattr__(self, name, value):
        """
        All we want to do here is make it look the same as if we called
        setattr() on a real `super` object.
        """
        super = object.__getattribute__(self, '_super__super')
        object.__setattr__(super, name, value)

def _getSuper (self):
    """
    Gets the `super` object for the class of the currently-executing method.
    """
    frame = sys._getframe().f_back
    code = frame.f_code
    name = code.co_name

    # Find the method we're currently running by scanning the MRO and comparing
    # the code objects. When we find a match, that *might* be the class we're
    # currently in - however, we need to keep searching until we fail to find
    # a match. This is due to the way that methods are created - if you have
    #
    # class A (autosuper):
    #     def test (self):
    #         pass
    #
    # class B (A):
    #     pass
    #
    # then calling getattr(B, 'test') will return A.test, with no way to
    # determine that it's A.test. We only want to use this after calling 
    # getattr(A, 'test') otherwise we will determine the wrong class.

    cur_class = None

    for c in type(self).__mro__:
        try:
            m = getattr(c, name)
            func_code = m.func_code
        except AttributeError:
            func_code = None

        if func_code is code:
            cur_class = c
        elif cur_class is not None:
            break

    if cur_class is None:
        # We could fail to find the class if we're called from a function
        # nested in a method
        raise TypeError, "Can only call 'super' in a bound method"

    return _super(super(cur_class, self), name)

class autosuper (object):
    """
    Automatically determine the correct super object and use it.
    """

    # We want the lowest overhead possible - both speed and memory
    __slots__ = ()

    super = property(fget=_getSuper,
                     doc=_getSuper.__doc__.strip())

Just call self.super(params) or self.super.<attr> from inside a method and everything works. No name mangling, etc. Now works with Python 2.2 and higher.

This method suffers from a fair bit of overhead - the version on my website does some bytecode hacking to improve performance, at a slight cost in memory usage.

Works with psyco - but strangely, if you import psyco, autosuper slows down a lot (i.e. just the line 'import psyco'). The version on my web site appears to benefit from psyco (for the Python version) although the Pyrex version still has some issues.

The original code used a __metaclass__ - this is not necessary, and in fact a mixin autosuper class is more useful than a metaclass.

Modified to take into account Guido's comments - in particular, AttributeError will now always be raised if no class later in the MRO has that attribute.

Note that it is possible to cause problems if code objects for methods become shared somehow (e.g. the code object for one method is assigned to another). The way psyco replaces code objects (and Raymond Hettinger's constant binding recipe) do not conflict with this recipe.

7 comments

Michele Simionato 12 years, 11 months ago  # | flag

very interesting. super is very hairy indeed. Your recipe goes in a direction (using sys._getframe, inspecting the MRO by hand, etc) which I discarded, but seems to be working (I should really find the time to write down a comprehensive suite of corner cases, anyway). It would be interesting to try it with Jython (not sure about the status of Jython, but they should have added super and Python 2.2 now, at least at some alpha level).

After a lot of thinking I have decided that the simpler solution is to be very explicit and to pass the super object as an additional argument to the cooperative methods. I posted the idea here http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/284528 but that recipe is by no means complete. It takes a lot of work to get a robust solution and I am kind of lazy ;)

Anyway this would work with my recipe

class B(Cooperative):
    def __init__(self):
        print "B.__init__"

class C(B):
    def __init__(self,super):
        print "C.__init__"
        def helper():
            print "C - helper"
            super.__init__()
        helper()

whereas it cannot work with your approach. There are dark hacks that would allow me to pass the super argument implicitly, but they are too ugly to be even mentioned and I feel kind of fine to have an additional argument. In this way users are prevented just from the argument line that they are going to read a cooperative method and they can prepare themselves ;)

Nice work, but I would not want to be autosuper maintainer ! ;)

      Michele Simionato
Tim Delaney (author) 12 years, 11 months ago  # | flag

It deals with the most common case. Whilst it doesn't allow calling self.super from inside the nested helper function, you can pass self.super explicitly to the helper function.

class B (autosuper):
    def __init__(self):
        print "B.__init__"

class C (B):
    def __init__(self):
        print "C.__init__"

        def helper (super):
            print "C - helper"
            super()

        helper(self.super)

The most common case is to not use a helper function, so I don't see that it's an excessive amount of work to pass the bound super proxy explicitly in those cases that you want to.

I also don't want to go down the track of the hacking required to determine which function is nesting this one, etc.

Tim Delaney (author) 12 years, 10 months ago  # | flag

Doesn't rely on "magic" parameter names. Another advantage of my recipe over Cooperative is that it doesn't give magic meaning to parameter names. The recent discussions on c.l.py over Paul Morrow's suggestion for using parameter names to determine static and class methods prompted me to add this.

Changing parameter names should not cause change in behaviour.

Paul McNett 12 years, 8 months ago  # | flag

Great, but 2 common cases don't work. Tim, I'm really excited to see your improved autosuper. I tried to do exactly this back in April, and wound up giving up with a second-best solution of making it a classmethod instead and accepting the class as the parm instead of the self reference. My recipe (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/279604) works really well but a solution that allows calling self.super() instead of class.super() is certainly superior.

I've run into a couple problems with your recipe:

if __name__ == "__main__":
        class A(autosuper):
                def fun1(self):
                        print "A"
                        self.super()   #- TypeError

        class B(A):
                def fun1(self):
                        print "B"
                        self.super()

        class C(B): pass    #- recursion

        # class C(B):
        #       def fun1(self):
        #               print "C"
        #               self.super()
        t = C()
        t.fun1()

Problem #1 is a TypeError that occurs when trying to call super() from A.fun1(). The caller shouldn't have to know when/where to use self.super().

Problem #2 is the real bugger: if class C doesn't provide fun1, and fun1 is called, infinite recursion will occur with:

Traceback (most recent call last):
  File "autosuper.py", line 129, in ?
    t.fun1()
  File "autosuper.py", line 120, in fun1
    self.super()
  File "autosuper.py", line 38, in __call__
    return method(*p, **kw)
  File "autosuper.py", line 120, in fun1
    self.super()
  File "autosuper.py", line 38, in __call__
    return method(*p, **kw)
  File "autosuper.py", line 120, in fun1
    self.super()
  ...
RuntimeError: maximum recursion depth exceeded

I'm not enough of a guru to figure out why this recursion is occuring, but I do know that the use case that causes it is not uncommon. If autosuper can be fixed to account for this case, it looks to be quite useful.

Paul McNett 12 years, 8 months ago  # | flag

Great, but 2 common cases don't work. Tim, I'm really excited to see your improved autosuper. I tried to do exactly this back in April, and wound up giving up with a second-best solution of making it a classmethod instead and accepting the class as the parm instead of the self reference. My recipe (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/279604) works really well but a solution that allows calling self.super() instead of class.super() is certainly superior.

I've run into a couple problems with your recipe:

if __name__ == "__main__":
        class A(autosuper):
                def fun1(self):
                        print "A"
                        self.super()   #- TypeError

        class B(A):
                def fun1(self):
                        print "B"
                        self.super()

        class C(B): pass    #- recursion

        # class C(B):
        #       def fun1(self):
        #               print "C"
        #               self.super()
        t = C()
        t.fun1()

Problem #1 is a TypeError that occurs when trying to call super() from A.fun1(). The caller shouldn't have to know when/where to use self.super().

Problem #2 is the real bugger: if class C doesn't provide fun1, and fun1 is called, infinite recursion will occur with:

Traceback (most recent call last):
  File "autosuper.py", line 129, in ?
    t.fun1()
  File "autosuper.py", line 120, in fun1
    self.super()
  File "autosuper.py", line 38, in __call__
    return method(*p, **kw)
  File "autosuper.py", line 120, in fun1
    self.super()
  File "autosuper.py", line 38, in __call__
    return method(*p, **kw)
  File "autosuper.py", line 120, in fun1
    self.super()
  ...
RuntimeError: maximum recursion depth exceeded

I'm not enough of a guru to figure out why this recursion is occuring, but I do know that the use case that causes it is not uncommon. If autosuper can be fixed to account for this case, it looks to be quite useful.

Tim Delaney (author) 12 years, 8 months ago  # | flag

I think you mean AttributeError. The first case causes an AttributeError, and this is expected. I originally allowed missing attributes to pass silently, but Guido convinced me that this was the wrong way to go.

Traceback (most recent call last):
  File "D:\Development\modules\autosuper\test.py", line 16, in ?
    B().fun1()
  File "D:\Development\modules\autosuper\test.py", line 12, in fun1
    self.super()
  File "D:\Development\modules\autosuper\autosuper.py", line 108, in __call__
    return method(*p, **kw)
  File "D:\Development\modules\autosuper\test.py", line 7, in fun1
    self.super()   #- TypeError
  File "D:\Development\modules\autosuper\autosuper.py", line 116, in __call__
    method = getattr(super, method)
AttributeError: 'super' object has no attribute 'fun1'

I'm not sure what cases the second problem yet (it's got to be scanning the MRO) but I'll post an updated recipe when I've worked it out.

Tim Delaney (author) 12 years, 8 months ago  # | flag

Recursion problem solved. Comments are inline - basically, I was getting the base class method (and hence code object) at the subclass level, and so never getting past there.

The solution is to continue searching the MRO until I fail to find a matching code object - that means that the previous class was where the code object was defined i.e. the real class I want to use.

Add a comment

Sign in to comment

Created by Tim Delaney on Sun, 4 Jul 2004 (PSF)
Python recipes (4587)
Tim Delaney's recipes (1)

Required Modules

Other Information and Tasks