Generator Attributes (Python recipe) by Raymond Hettinger

ActiveState Code (http://code.activestate.com/recipes/164044/)

a class version with almost the same semantics..... but quite a bit more flexible!!!

# would it not be easier to do the following:

class Generator(object):
    '''
    this is a generator class.
    '''
    def __call__(self):
        self.counter = 0
        while self.counter < 10:
            self.counter += 1
            yield self.counter

gen = Generator()

for i in gen():
    print i

# yes this adds the extra step of class instantiation but it removes
# the need for outside-class parameter definition. and the advantage
# is that this method allows for transparent way to create not only
# parameters but special methods to manipulate internal generator
# state.


# Imagine a singleton generator:

class SingletonGenerator(object):
    '''
    this generator singleton class counts to ten and then resets.
    '''
    __state = 0
    def __call__(self):
        while self.__state < 10:
            self.__state += 1
            yield self.__state
        self.__state = 0
    def rest(self):
        self.__state = 0
    def __getattr__(self, name):
        return getattr(self.__class__, name)
    def __setattr__(self, name, val):
        return setattr(self.__class__, name, val)

# usage:

gen_a = SingletonGenerator()
gen_b = SingletonGenerator()

for i, j in zip(gen_a(), gen_b())
    print i, j,

## output:
# 1 2 3 4 5 6 7 8 9 10

some might call what was done in the previous comment "the borg" pattern, I see them (borg and singleton) to be one and the same, they differ only in the side you chose to look from...

A comparative example is found in 'Hackers and Painters' Anyone interested in this recipe might also like to read a recent book, "Hackers and Painters: Big Ideas from the Computer Age", by Paul Graham (2004).

Though the book is mostly a non-technical discussion of programming principles and practices (generally lionizing Lisp) there is an appendix (pp. 195-99) which compares the __call__ version above to even simpler solutions in other languages.

Though his example is just a generated function with maintained state, rather than a "generator" (ie a function with a "yield" call), it explains exactly why some people appreciate other, more powerful languages.

I disagree with him: The Python example is only slightly more complicated, and it is vastly easier for a non-expert user to comprehend. The implicitly lexical variables do not exactly aid readability.

However, Graham is not unkind to Python, and this book is definitely worth reading for anyone with an interest in the utility of programming languages.

I'm pointing that out because this recipe has caused some big ideas to gel in my mind. After studying this complicated recipe, I suddenly "get" what Python is doing with functions. Tes, the version in the comment is definitely the way I would code it in practice, but I'm glad to see the complex one.

In case you don't have the book, here are his examples. Python first:

class foo:
  def __init__(self, n): self.n = n
  def __call__(self, i): self.n +=1; return self.n

Obviously, this could yield self.n instead.

The Java solution is a pain, but in Javascript:

function foo(n) {
  return function(i) {
    return n += 1 }}

Smalltalk has strange syntax, but at least has the transparency of the Python version:

foo: n
  |s|
  s := n.
  ^[:i| s := s+i. ]

Perl has an implicit return statement:

sub foo {
  my ($n) = @_;
  sub {$n += shift}
}

Ruby is the clearest, but with strange delimiters:

def foo (n)
  lambda {|i| n += i} end

And finally, Lisp is easily the least cluttered:

(defun foo(n)
  (lambda (i) (incf n i)))

Graham claims that this cannot be done in C++, but of course C++ has its own analogue of __call__, operator():

struct foo {
  int n;
  foo(int N): n(N) {}
  int operator()(int i) {
    return n += i;
  }
};

// BUT BEWARE:
#include
int main() {
  foo f(7);
  foo g(30);
  cerr << f(2)  << ' ' << g(2) << ' ' << f(2)
       << ' ' << g(2)  << " hike!" << endl;
}
// Prints: 11 34 9 32 hike!
// (if you replace the stream insertion operators,
//  which had formatting problems)
// The stream-operator associates left-to-right,
// but it is evaluated right to left!

and this demonstrates the inherent dangers of this sort of programming.

(comment continued...)

(...continued from previous comment)

The author also lists 2 other Python solutions which work, and 2 which do not work:

def foo(n):
  class accumulator:
    def __init__(self, s):
      self.s = s
    def inc(self, i):
      self.s += i
      return self.s
  return acc(n).inc

# and

def foo(n):
  s = [n] # The list holds a copy of n
  def bar(i):
    s[0] += i
    return s[0]
  return bar

And these 2 do not work:

def foo(n):
  return lambda i: return n += 1

# nor

def foo(n):
  lambda i: n += 1

The author guesses that Python will one day allow something like these, but only the Python class-based version allows easy access to n. So in fact, what to some is a flaw in Python can be a feature to others.

I hope somebody learns as much from these examples as I have. Functional programming suddenly makes much more sense to me.

Graham's examples are not 'statically typed' The C++ and Java versions work only on integers, which is what Graham meant to point out. But a template is possible. Anyway, the point of the present recipe is to have access into the function, and the Python version is quite nice that way.

no comments. very nice recipe!