Welcome, guest | Sign In | My Account | Store | Cart

When inspecting new code (or when debugging), it can be handy to know exactly where a given attribute on an object or class comes from.

As a simple example, if you have a class MyClass, you might want to know where MyClass().myMethod is defined.

However, things can get tricky when things like __getattr__, __getattribute__, or even compiled objects come into play. That's where this function comes in. It returns what class a given attribute comes from, and what method was used to define it - ie, '__dict__' ('normal' definitions), '__slots__', '__getattr__', '__getattribute__', '(BUILTIN)'.

(Note - this function should't be relied on to be 100% accurate - rather, it's a best guess, for where to look to find it. It takes some pretty infrequent edge cases for it to be wrong, though...)

Python, 244 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
import inspect
import types

def get_class(obj):
    '''Retrives the class of the given object.

    unfortunately, it seems there's no single way to query class that works in
    all cases - .__class__ doesn't work on some builtin-types, like
    re._pattern_type instances, and type() doesn't work on old-style
    classes... 
    '''
    cls = type(obj)
    if cls == types.InstanceType:
        cls = obj.__class__
    return cls

def where_attr_defined(obj, attr_name):
    '''Given an instance or class and an attribute name, returns the class
    (or instance) where it was defined, and 'how'

    If the given attribute does not exist on the object, it raises an AttributeError

    If the given attribute exists in the given instance's __dict__, it is
    simply a data attribute on the instance, and (obj, '__dict__') is returned.

    If the given attribute exists, but the method can't figure out how (perhaps
    it's some sort of builtin/func/method/attribute??), then it returns
    (obj, '(UNKNOWN)')

    Otherwise, it returns (cls, how), where cls is the class where it was defined,
    and how is one of '__dict__', '__slots__', '__getattribute__', or '__getattr__',
    or '(COMPILED)'.

    Note that in the case of __getattribute__, and __getattr__, since these
    are methods which can call super implementations, the determination of the
    exact class is tricky/imprecise.

    Details on __getattr__/__getattribute__ resolution
    --------------------------------------------------

    First of all, __getattr__/__getattribute__ are only checked if what was
    passed in was an instance, and not just a class... (because if you do
    MyClass.foo, __geattr__/__getattribute__ are not invoked!)

    If we did have an instance, and we find that one of these methods is
    defined on a class, then we call that class's implementation with the
    given attribute. If we ever get a result, we are not immediately done -
    though that class's method returned a result, it may have called a parent
    super implemenation, and that parent class's implementation is what
    "really" provided the result. So, one we have A result, we go up the
    parent chain, calling the method and getting the result for any parent
    classes which also define the method; if the attribute is ever not found,
    or the result is different than the previous, we assume the previous
    successful was the "lowest" in the chain to add support for this attribute
    name, and return that previous class.
    '''
    if not hasattr(obj, attr_name):
        raise AttributeError('%r has no attribute %r' % (obj, attr_name))
        
    if inspect.isclass(obj):
        cls = obj
        inst = None
    else:
        inst = obj
        cls = get_class(obj)
        # first try the inst's __dict__
        inst_dict = getattr(inst, '__dict__', None)
        if inst_dict is not None and attr_name in inst_dict:
            return inst, '__dict__'

    # with __dict__ and slots, we know that as soon as we find an entry,
    # that's the one that defined it...
    #
    # ...with __getattribute__ and __getattr__, it's trickier, since they're
    # classmethods, and could call the super... so even though
    # myCls.__getattr__('myAttr') could return a result, it could be calling
    # the super .__getattr__, and 'myAttr' is "acutally" added by
    # parentCls.__getattr__
    #
    # To try to resolve this, if we get a result back from one of these, we
    # don't immediately return it... instead, we continue checking up the
    # parent chain until we either fail to get a result, or get a different
    # result... Note that this isn't perferct, as (among other things), it
    # would mean that there would need to be a valid equality comparison for
    # the type of object returned...
    def find_orig_getter(start_cls, start_result, attr_name, mro, meth_name):
        # Note that we have to be fed in the mro - we can't just use
        # start_cls.mro(), as, due to diamon inheritance, it could be
        # different from the "remaining" mro of a child class...
        result_pair = (start_cls, meth_name)
        for cls in mro:
            cls_dict = getattr(cls, '__dict__', None)
            if cls_dict is not None and meth_name in cls_dict:
                getter = getattr(cls, meth_name)
                try:
                    result = getter(inst, attr_name)
                except Exception:
                    # if we didn't get a result, then whatever result
                    # we already have must be the one to use...
                    return result_pair
                else:
                    # it's possible equality comparison can return an error...
                    # so first try 'is', and then try equality in a try/except...
                    were_equal = result is start_result
                    if not were_equal:
                        try:
                            were_equal = (result == start_result)
                        except Exception:
                            # if there was an error in comparing, consider them
                            # not equal...
                            were_equal = false
                    if were_equal:
                        result_pair = (cls, meth_name)
                    else:
                        return result_pair
                        
        return result_pair

    # Go up the mro chain, checking for __dicts__, __slots__, __getattribute__, and __gettattr__
    getattr_cls = None
    # .mro attribute only exists on new-style classes, so use inspect.getmro
    mro = inspect.getmro(cls)
    for i, parent in enumerate(mro):
        cls_dict = getattr(parent, '__dict__', None)
        cls_slot = cls_dict.get('__slots__', None)
        if cls_slot is not None and attr_name in cls_slot:
            return parent, '__slots__'
        if cls_dict is not None and attr_name in cls_dict:
            return parent, '__dict__'
        for get_method in ('__getattribute__', '__getattr__'):
            if (cls_dict is not None and get_method in cls_dict
                    or cls_slot is not None and get_method in cls_slot):
                if inst is not None:
                    getter = getattr(parent, get_method)
                    try:
                        result = getter(inst, attr_name)
                    except Exception:
                        continue
                    else:
                        # we got a result - check up parent chain
                        result = find_orig_getter(parent, result, attr_name, mro[i + 1:], get_method)
                        # check that we're not just returning object.__getattribute__...
                        if result != (object, '__getattribute__'):
                            return result
    # we went all the way up the chain, and couldn't find it... 

    # if we stored a getattr result, return that...
    if getattr_cls is not None:
        return getattr_cls, '__getattr__'

    # check if the object's class is from a compiled module - if so, assume that's the source
    # of our mystery attribute
    for cls in mro:
        module = inspect.getmodule(cls)
        if module and not hasattr(module, '__file__'):
            return (cls, '(COMPILED)')

    # otherwise, we have no clue...perhaps it's some weird builtin?
    return obj, '(UNKNOWN)'


############################
# Some examples / tests... #
############################


# Helper func to print the results of where_attr_defined
def print_attr_sources(instOrCls, attrs):
    if inspect.isclass(instOrCls):
        cls = instOrCls
        inst = cls()
    else:
        inst = instOrCls 
        cls = get_class(inst)

    for attr in attrs:
        print
        for obj in (cls, inst):
            objStr = cls.__name__
            if isinstance(obj, cls):
                objStr += '()'
            print "where_attr_defined(%s, %r):" % (objStr, attr),
            try:
                print where_attr_defined(obj, attr)
            except AttributeError:
                print "!! %s does not exist !!" % attr

# Check it's behavior on new-style clases, with inheritance, 
# __slots__, and __getattr__
class MyClass(object):
    __slots__ = ['foo', 'bar']
    def __init__(self):
        self.foo = 'things'
        
    @property
    def stuff(self):
        return 'whaev'

    def __getattr__(self, attr):
        if attr == 'numberOfThyCounting':
            return 3
        raise AttributeError


class SubClass(MyClass):
    def stuff(self):
        return 'an override!'

class GrandChild(SubClass):
    def __getattr__(self, attr):
        if attr == 'deadParrot':
            return 'pining for the fjords'
        return super(GrandChild, self).__getattr__(attr)



print_attr_sources(GrandChild, ('foo', 'bar', 'stuff', '__init__', 'numberOfThyCounting',
             'deadParrot', 'santaClaus'))


# check it's behavior on old-style classes..
class OldMyClass:
    def __init__(self):
        self.foo = 'things'
        
    @property
    def stuff(self):
        return 'whaev'

    def __getattr__(self, attr):
        if attr == 'numberOfThyCounting':
            return 3
        raise AttributeError

class OldSub(OldMyClass):
    def __init__(self):
        OldMyClass.__init__(self)

print_attr_sources(OldSub, ('foo', 'stuff', 'numberOfThyCounting', '__init__'))

# Check that it works on c-compiled types...
import re
compiled = re.compile('foo')
print_attr_sources(compiled, ('pattern', 'sub'))

If anyone knows of a better way to handle attributes supplied by compiled objects, let me know... right now, I'm just trying all other methods, and then if I can't find it any other way, I check to see if any of the classes in it's mro are compiled... and if so, assume that class is the source of the attribute. Obviously, this can be error prone - though, in most cases, it ends up being right...