Automating simple property creation (Python recipe) by Sean Ross

ActiveState Code (http://code.activestate.com/recipes/157768/)

correction. should have been:

def set_baz(self, value): self.__baz = value

baz = property(fset=set_baz)

not "bar"

sorry for any confusion,

Sean

correction. should have been:

def set_baz(self, value): self.__baz = value

baz = property(fset=set_baz)

not "bar"

sorry for any confusion,

Sean

re: correction. The corrections noted above have been made.

Breaks on inheritance. mangle's reliance on obj.__class__ means that it'll be unable to find the private variable if someone attempts to fetch a property via a subclass. To fix it, we need to get the class name somehow. Fortunately, that's in the stack, too. :)

'''
written by: Sean Ross
created: 10/21/02

modified to cope with inheritance by: Garth Kidd
when: 1/9/03
'''
from traceback import extract_stack

def __classAndLeftOperands__():
    stack = extract_stack()
    argStack = stack[-3:][0][-1]
    className = stack[1][2]

    args = argStack.split('=')
    args.pop()
    args = args[0].split(',') # handle csv's
    leftOperands = [arg.strip() for arg in args] # and, remove whitespace

    return className, leftOperands

def mangle(obj, name, className):
    '''mangles name according to python name-mangling
    conventions for private variables'''
    return '_' + className + '__' + name

# based on code by Guido van Rossum, comp.lang.python 2001-07-31
def readonly():
    "returns a list of read-only properties, one for each left operand"
    className, args = __classAndLeftOperands__()
    def __readonly(name, className):
        def get(obj):
            return getattr(obj, mangle(obj, name, className))
        return property(get)
    props = [__readonly(name, className) for name in args]
    if len(props) == 1: props = props[0]
    return props

def writeonly():
    "returns a list of write-only properties, one for each left operand"
    args = __leftoperands__()
    def __writeonly(name):
        def set(obj, value):
            return setattr(obj, mangle(obj, name), value)
        return property(fset=set)
    props = [__writeonly(name) for name in args]
    if len(props) == 1: props = props[0]
    return props

Posted updated module as source distribution. I've posted a source distribution for this [1], with an explanation on my blog [2]. To avoid treading on Sean's code, I've used distutils to give appropriate credit: him as the author, with me in as a maintainer. Sean, if you want to take back over and maintain at your site, let me know. 1: http://www.deadlybloodyserious.com/Python/files/prop-1.5.zip

2: http://www.deadlybloodyserious.com/Python/2003/01/09.html#a1047

Small Problem. Hi.

First, thank you for pointing out this algorithms limitation with

regard to inheritance. I've taken your suggestions and incorporated

them into the code(as you'll notice). I've made some changes; mostly

for personal taste. For instance, I changed __classAndLeftOperands__()

to __outside__() (yours is more descriptive, but it is quite long)

Anyway, there was one change I made that was not purely cosmetic:

className = stack[1][2]

became

cls = stack[-3:][0][-2]

This is important. On my system, for instance, to gain access to the appropriate stack frame from the front(?) of the stack, I need to call:

className = stack[6][2]

to get the intended results. My system has more overhead, seemingly.

Anyway, accessing the data from the back of the stack seems more reliable, as is evidenced by the original __leftoperands__() having

worked on your system.

Other than that, this is a fine improvement. Thanks.

Now, if you can figure out how to remove the requirement for having to provide the private instance variables for each property, that'd be great.

Aha! I'd noticed that my code was extremely brittle -- readonly() would fail if I compiled my application with py2exe, and intermittently if I tried to reload a module. I'll try it with your modifications!

No go. Py2exe strips source lines... ... so __outside__ can't get the source line. I'm exploring aloud on my blog, and will report back if I make any progress. Or, not. :)

http://www.deadlybloodyserious.com/Python/2003/01/10.html#a1050

Got it! Try something along these lines:

def readonly2(*propList):
    "inserts definitions for local properties in the parent frame"
    stack = traceback.extract_stack()
    cls = stack[-2:][0][-2]
    targetFrame = sys._getframe(1)
    def __readonly(name):
        def get(self):
            return getattr(self, mangle(cls, name))
        return property(fget=get)
    for prop in propList:
        targetFrame.f_locals[prop] = __readonly(prop)

... and then...

readonly2('name', 'colour', 'eyes')

... to create the read-only attributes. Not depending on the source, it works properly under py2exe. That said, the syntax leaves a lot to be desired.

Better yet! This works under python, python -OO, and py2exe:

def __outside__():
    '''retrieves list of names of left operands for calling function
       and the name of the class where that function was called.'''
    code = sys._getframe(2).f_code
    cls, names = code.co_name, list(code.co_names)
    names.reverse()
    caller = sys._getframe(1).f_code.co_name
    args = names[:names.index(caller)]
    return cls, args

When you update the code, set __version__ to "1.7"; using a float also breaks py2exe. :)

Sorry. Unfortunately, the new __outside__() doesn't work on my system.

Using the test code I provide in the comments, but with all code

in __main__ below test = Test() commented out and the new

__outside__() with print statements, i.e.,

def __outside__():

&nbsp...

&nbspcls, names = code.co_name, list(code.co_names)

&nbspprint "class: ", cls

&nbspprint "names: ", names

&nbsp...

&nbspargs = names[:names.index(caller)]

&nbspprint "args: ", args

&nbsp...

I get the following output:

class: Test

names:['__name__', '__module__', 'readonly', 'foo', 'bar', 'writeonly', 'fro', 'boz', '__init__']

caller: readonly

args: ['__init__', 'boz', 'fro', 'writeonly', 'bar', 'foo']

class: Test

names: ['__name__', '__module__', 'readonly', 'foo', 'bar', 'writeonly', 'fro', 'boz', '__init__']

caller: writeonly

args: ['__init__', 'boz', 'fro']

You'll notice that the args for readonly is _not_ ['foo', 'bar'].

So, unfortunately, __outside__() remains brittle, so long as

your intent is to use py2exe.

Version 1.6 seems ...ok..., otherwise.

deprecate __outside__(). For the moment __outside__() remains brittle. I propose that it be deprecated and that readonly2() replace readonly().

readonly2() appears to be stable. The syntax is ok, for now.

I do have one minor change:

def __readonly(name):

&nbspname = mangle(cls, name)

&nbspdef get(obj):

   &nbspreturn getattr(obj, name)

&nbspreturn property(fget=get)

I realized that mangle() was being called every time a property was used, not just when the property is created, which is, of course, unnecessary.

Until __outside__() can be made stable, I recommend that it not be used.

Let me know what you think.

If you agree, I will update the current version to reflect these decisions.

Just one more idea:

What I'd like to see is

def readonly(**kwds): ...

so that the usage would be

readonly(foo=1, bar=2)

More importantly, I'd like to remove the requirement for providing the private instance variables '__foo', '__bar',

i.e., I'd prefer not to have to do this:

readonly('foo', 'bar')

...

def __init__(self):

&nbspself.__foo = 1

&nbspself.__bar = 2

&nbsp...

In the meantime, readonly2() will suffice.

Let me know your thoughts.

Sean

This works. Try this:

def readonly(**kwds):

"inserts definitions for local properties in the parent frame"

&nbspstack = traceback.extract_stack()

&nbspcls = stack[-2:][0][-2]

&nbspframe = sys._getframe(1)

&nbspdef __readonly(name, default):

   &nbspname = mangle(cls, name)

   &nbspdef get(obj):

       &nbspsetattr(obj, name, default)

       &nbspreturn getattr(obj, name)

   &nbspreturn property(fget=get)

&nbspfor name,value in kwds.items():

   &nbspframe.f_locals[name] = __readonly(name, value)

usage: readonly(foo=1, bar=2)

Note: does not require you to provide private instance variables __foo and __bar (!)

It's a bit of a bodge, because you have to set the private instance variable every time the property is called. It's nicer in writeonly, where this bodge is not required.

I like this. Let me know what you think before I update the recipe.

fixed bodge. def __readonly(name, default):

&nbspname = mangle(cls, name)

&nbspdef get(obj):

   &nbspvalue = default

   &nbsptry:

       &nbspvalue = getattr(obj, name)

   &nbspexcept AttributeError:

       &nbspsetattr(obj, name, default)

   &nbspreturn value

&nbspreturn property(fget=get)

This avoids calling setattr() every time property is called. It is only called if getattr() fails, i.e., on the first call. All subsequent calls to getattr() will succeed, and setattr() will not be called.

Version 1.8. I've decided to go ahead and post version 1.8. I've changed __outside__() so that now it simply takes care of the repetitive code that was in both readonly() and writeonly().

Let me know if this latest version gives you any problems. For now, I'm very happy with this version. This is what I was hoping to make all along. Thanks for your help.

Sean

Aah. I think I get it, now. The problem with my __outside__ is that it assumes that readonly() or writeonly() are the LAST items called. In your test code, __init__ is defined lastish, hence the problem. I guess we could search through the frame above's locals to check to see whether the names are defined.

The new keyword argument style certainly saves us having to grope around for the names, but can we stick with the use of sys._getframe()? Its results seem a little more deterministic than wading through the trace.

co_stacksize? code.co_stacksize might have the number of lvars we need.

def __outside__():
    '''STILL A LITTLE BROKEN.'''
    code = sys._getframe(2).f_code
    cls, names = code.co_name, list(code.co_names)
    names.reverse()
    caller = sys._getframe(1).f_code.co_name
    args = names[:names.index(caller)][-code.co_stacksize:]
    args.reverse()
    return cls, args

... but it's still breaking if I use readonly() twice, so I'm not quite there yet.

Note also that I missed that my earlier code was accidentally reversing the arguments. Whups.

I'm not so allergic to having to define the instance variables myself in __init__, especially as I'd be using self.__varname inside my class for performance reasons. I'm also worried about readability; with the first syntax, it's immediately obvious to the reader that variables are being defined, even if they're not sure exactly how.

I've removed the traceback code, and have replaced it with sys._getframe() code.

You can still initialize the corresponding private instance variables(CPIVs) yourself, and you can still use them as self.__foo inside your class. But, the thing I like about how readonly() and writeonly() work now, is that you don't have to initialize those variables. What does this accomplish? Well, if you look at the test code I include in the discussion, you'll see a reduction from 18 lines to 5 lines of code(from defining read/writeonly props. the old way to defining them our way). In fact, you save 3 lines of code for each read/writeonly prop. If you init your CPIVs, you still save 2 lines/prop. It's up to you whether you want to declare them yourself or not.

Regarding readability, I agree that

foo, bar = readonly()

more obviously appears to be assigning readonly properties to the class. Still,

readonly(foo=1, bar=2)

has the benefit of creating self.__foo and self.__bar, "automagically", with default values. The previous syntax would continue to require that CPIVs be hand-coded. That is not what I want.

Explicit may be better than implicit, but in this case, I would say:

Is it more obvious from

foo, bar = readonly()

that you must make CPIVs by hand, or

is it more obvious from

readonly(foo=1, bar=2)

that CPIVs have been created for you?

When I think about it, neither is overly obvious, but I think the former is less so than the latter.

You could combine them, I suppose.

foo, bar = readonly(foo=1, bar=2)

This does appear to be more explicit, if not entirely more obvious. And it allows you to supply default or initial values for your readonly properties.

Note: foo, bar, fro, boz = readonly(1,2,3,4), is just wrong

foo, bar, fro, boz = readonly(foo=1, bar=2, fro=3, boz=4),

I think, is better. While

readonly(foo=1, bar=2, fro=3, boz=4)

alone, without assignment, is acceptable to me. Or, perhaps

__readonly__(foo=1, bar=2, fro=3, boz=4)

to let people know there's "automagic" happening...

I wish ASPN had stuck with linear, non-heirachial discussion threads. I think we're vectoring in on a choice between foo, bar = readonly(foo=2) (foo defaulted, bar requiring manual attribute creation) and something like __readonly__('bar', foo=2). I like the double-underscorage making it obvious that we're doing something a little different.

Aaaah. Of course. If we make initialisation mandatory, then we know how many items to kick out, and we don't need to grope through frames trying to figure out the names of our lvars...

It seems to me the ideas in this recipe could be accomplished more easily using a metaclass. First, suppose that __readonly__ creates one or more temporary instances of some class and stores them in its caller's locals dict. Assuming this dict is the locals of the class-defining function, it is the same dict which is passed to the metaclass's __new__ function, along with the new class's name. So in the __new__ function, you simply hunt through the dict to look for instances of your temporary class, and then replace them with properly formed property instances. E.g.:

class readonlypropmarker(object):
    def __init__(self, value):
        self.value = value

def __readonly__(**kwds):
    locals = sys._getframe(1).f_locals
    for name, val in kwds.items():
        locals[name] = readonlypropmarker(val)

def newreadonlyprop(clsname, name, marker):
    name = mangle(clsname, name)
    defvalue = marker.value
    def get(obj):
        #simplified version without setting attribute
        return getattr(obj, name, defvalue)
    return property(fget=get)

class ExtPropMeta(type):
    def __new__(meta, clsname, bases, attrs):
        for name, attr in attrs.items():
            if isinstance(attr, readonlypropmarker):
                attrs[name] = newreadonlyprop(clsname, name, attr)
        return super(ExtPropMeta, meta).__new__(meta, clsname,
                                                bases, attrs)

class ExtProp(object):
    __metaclass__ = ExtPropMeta

Then real classes can use ExtProp as a mixin base class to get this special property support.

I like this solution. It's clever, and reading it was an education. The only advantage I see to Garth's and my solution over yours is the fact that our solution does not require users to make their classes subclass ExtProp to get the desired behaviour. If they put rowo.py in their path, they can just import the behaviour, e.g.

&nbspfrom rowo import *

But, that's not much of a difference. It's just a matter of taste, I suppose. I happen to prefer:

&nbspfrom rowo import *

&nbspclass MyClass(object):

    __readonly__(foo=1)

to

&nbspfrom ExtProp import *

&nbspclass MyClass(ExtProp):

    __readonly__(foo=1)

Probably because the latter gives me the impression that my class is dependent upon another class. The former does not leave me with this impression. Other than that, I really did enjoy reading your code. Thank you very much for your contribution.

Sean