Popular Python recipes tagged "coroutine"

Pipeline made of coroutines (Python)

2012-09-14T09:53:58-07:00

Python recipe 578265 by Chaobin Tang (唐超斌) (coroutine, pipelining, python). Revision 2.

A pipeline made of several coroutines that can be turned off gracefully.

Run asynchronous tasks using coroutines (Python)

2010-08-06T16:16:20-07:00

Python recipe 577129 by Arnau Sanchez (coroutine, event, generator, gobject, gtk, gui, network, nonblocking, pygtk). Revision 20.

This recipe shows a simple, transparent (and hopefully pythonic) way of running asynchronous tasks when writing a event-driven application (i.e. GUI). The aim is to allow a programmer to write time-consuming functions (usually IO-bound, but not only) with sequential-looking code, instead of scattering the logic over a bunch of callbacks. We will take advantage of the coroutines introduced in Python 2.5 (see http://www.python.org/dev/peps/pep-0342).

The goal: wouldn't it be great if we could write something like this?

def myjob(entry, arg1, arg2, arg3):
    result1 = function_that_takes_eons_to_complete(arg1, arg2)    
    result2 = another_function_that_downloads_a_big_really_big_file(result1, arg3)
    entry.set_text("The result is: %d" % result2)

def on_start_button___clicked(button, entry):
    myjob(entry, 1, 2, 3)

...    
gtk.main()

Indeed, but we can't! The GUI will hang until the job is done and the user will be rightfully angry. Coroutines to the rescue: the absolute minimal change we can make to this code is transforming myjob into a coroutine and yield every time we do blocking stuff:

def myjob(entry, arg1, arg2, arg3):
    result1 = yield some_task(arg1, arg2)    
    result2 = yield some_other_task(result1, arg3)
    entry.set_text("The result is: %d" % result2)

def on_start__clicked(button, entry):
    start_job(myjob(entry, 1, 2, 3))

some_task and some_other_task are here the asynchronous implementation of the sequential tasks used in the first fragment, and start_job the wrapper around the coroutine. Note that we still have to implement non-blocking versions of the tasks, but they are usually pretty generic (wait some time, download a file, ...) and can be re-used. If you happen to have a CPU-bound function or even a IO-bound code you cannot split (urllib2 anyone?), you can always use a generic threaded task (granted, the whole point of using co-routines should be avoiding threads, but there is no alternative here).

At the end, all the plumbing we need to make it work is just 1 function: start_job (wrapper around the job to manage the flow of the coroutine). The rest of the code -two asynchronous tasks (sleep_task, threaded_task) and a demo app- are shown solely as an example.

Yet another Python implementation of PEP 380 (yield from) (Python)

2010-04-25T23:08:07-07:00

Python recipe 577153 by Arnau Sanchez (coroutine, decorator, generator, refactor). Revision 3.

Any Python programmer knows how extremely powerful generators are. Now (since version 2.5) Python generators can not only yield values but also receive them, so they can be used to build coroutines.

One drawback of the current implementation of generators is that you can only yield/receive values to/from the immediate caller. That means, basically, that you cannot easily refactor your code and write nested generators. PEP-380 is the most serious effort to overcome this issue, but until it gets approved we can still play around with pure Python implementations of yield from.

This recipe follows terminology used by others in the past (recipe566726, recipe576728), but I've tried to simplify the code as much as possible.

Asynchronous subprocess using asyncore (Python)

2013-01-21T19:51:00-08:00

Python recipe 576957 by Glenn Eychaner (async, asynchronous, asyncore, coroutine, decorator, generator, ipc, subprocess). Revision 21.

A coroutine-based wrapper for subprocess.Popen that uses asyncore to communicate with child processes asynchronously. This allows subprocesses to be called from within socket servers or clients without needing a complicated event loop to check both. Uses recipe 576965 to provide the asynchronous coroutine framework, recipe 576967 to provide asynchronous pipes, and recipe 577600 to provide multiple alarms.

Multicontext (e.g. asynchronous) inline execution framework using coroutines (Python)

2012-12-06T19:32:20-08:00

Python recipe 576965 by Glenn Eychaner (asynchronous, coroutine, decorator, generator, inline, nonblocking, pattern, thread). Revision 14.

A framework for executing inline code, contained in a generator, across multiple execution contexts, by pairing it with an executor that handles the context switching at each yield. An example of a generator which executes some iterations synchronously and some asynchronously is provided. The framework is general enough to be applied to many different coroutine situations.