Any Python programmer knows how extremely powerful generators are. Now (since version 2.5) Python generators can not only yield values but also receive them, so they can be used to build coroutines.
One drawback of the current implementation of generators is that you can only yield/receive values to/from the immediate caller. That means, basically, that you cannot easily refactor your code and write nested generators. PEP-380 is the most serious effort to overcome this issue, but until it gets approved we can still play around with pure Python implementations of yield from.
This recipe follows terminology used by others in the past (recipe566726, recipe576728), but I've tried to simplify the code as much as possible.
Download
Copy to clipboard1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | """Python implementation of PEP-380 (yield from)."""
from functools import wraps
class _from(object):
"""Wrap a generator function to be used from a supergenerator."""
def __init__(self, genfunc):
self.genfunc = genfunc
def supergenerator(genfunc):
"""
Decorate a generator so it can yield nested generators (using _from class).
@supergenerator
def mysupergen():
yield "normal yield"
yield "one more yield"
yield _from(othergen())
yield _from(yet_othergen())
yield "last yield"
Note that nested generators can, in turn, yield other generators.
"""
def _process(gen):
tosend = None
while 1:
yielded = gen.send(tosend)
if isinstance(yielded, _from):
nested_gen = _process(yielded.genfunc)
nested_tosend = None
while 1:
try:
nested_yielded = nested_gen.send(nested_tosend)
except StopIteration, exc:
new_tosend = (exc.args[0] if exc.args else None)
break
except Exception, exc:
yielded.genfunc.close()
yielded2 = gen.throw(exc)
new_tosend = (yield yielded2)
break
nested_tosend = (yield nested_yielded)
else:
new_tosend = (yield yielded)
tosend = new_tosend
@wraps(genfunc)
def _wrapper(*args, **kwargs):
return _process(genfunc(*args, **kwargs))
return _wrapper
|
Now we can write a generator that yields values produced in another generators:
@supergenerator
def gen1():
yield "1a"
yield "1b"
yield _from(gen2())
yield "1c"
def gen2():
yield "2a"
yield _from(gen3())
yield "2b"
def gen3():
yield "3a"
yield "3b"
print list(gen1())
# ['1a', '1b', '2a', '3a', '3b', '2b', '1c']
Big deal, that's nothing we couldn't have done with a simple flattenning. What's really interesting is how we can now nest coroutines (example taken from profjim recipes):
@supergenerator
def gen1funct():
for i in xrange(3):
sent = yield i
print "sent1: %r" % (sent,)
retval = yield _from(gen2funct())
print "return value: %r" % (retval,)
def gen2funct():
for i in xrange(3,6):
sent = yield i
print "sent2: %r" % (sent,)
retval = yield _from(gen3funct())
print "return value2: %r" % (retval,)
raise StopIteration(100)
def gen3funct():
for i in xrange(6,9):
sent = yield i
print "sent3: %r" % (sent,)
raise StopIteration(1000)
Note how we use raise StopIteration(value) as a sort of return statement for the coroutine. While a yield in any level travels all the way up to the original caller, this returned value goes just to its inmediate caller.
