Run asynchronous tasks using coroutines (Python)

2010-08-06T16:16:20-07:00

Python recipe 577129 by Arnau Sanchez (coroutine, event, generator, gobject, gtk, gui, network, nonblocking, pygtk). Revision 20.

This recipe shows a simple, transparent (and hopefully pythonic) way of running asynchronous tasks when writing a event-driven application (i.e. GUI). The aim is to allow a programmer to write time-consuming functions (usually IO-bound, but not only) with sequential-looking code, instead of scattering the logic over a bunch of callbacks. We will take advantage of the coroutines introduced in Python 2.5 (see http://www.python.org/dev/peps/pep-0342).

The goal: wouldn't it be great if we could write something like this?

def myjob(entry, arg1, arg2, arg3):
    result1 = function_that_takes_eons_to_complete(arg1, arg2)    
    result2 = another_function_that_downloads_a_big_really_big_file(result1, arg3)
    entry.set_text("The result is: %d" % result2)

def on_start_button___clicked(button, entry):
    myjob(entry, 1, 2, 3)

...    
gtk.main()

Indeed, but we can't! The GUI will hang until the job is done and the user will be rightfully angry. Coroutines to the rescue: the absolute minimal change we can make to this code is transforming myjob into a coroutine and yield every time we do blocking stuff:

def myjob(entry, arg1, arg2, arg3):
    result1 = yield some_task(arg1, arg2)    
    result2 = yield some_other_task(result1, arg3)
    entry.set_text("The result is: %d" % result2)

def on_start__clicked(button, entry):
    start_job(myjob(entry, 1, 2, 3))

some_task and some_other_task are here the asynchronous implementation of the sequential tasks used in the first fragment, and start_job the wrapper around the coroutine. Note that we still have to implement non-blocking versions of the tasks, but they are usually pretty generic (wait some time, download a file, ...) and can be re-used. If you happen to have a CPU-bound function or even a IO-bound code you cannot split (urllib2 anyone?), you can always use a generic threaded task (granted, the whole point of using co-routines should be avoiding threads, but there is no alternative here).

At the end, all the plumbing we need to make it work is just 1 function: start_job (wrapper around the job to manage the flow of the coroutine). The rest of the code -two asynchronous tasks (sleep_task, threaded_task) and a demo app- are shown solely as an example.

Simple event dispatcher (Python)

2010-10-19T21:18:45-07:00

Python recipe 577432 by Daniele Esposti (dispatcher, event).

An example of a simple event dispatcher mini-framework

Flexible observer pattern implementation (Python)

2012-12-06T19:23:11-08:00

Python recipe 576962 by Glenn Eychaner (event, observer, pattern, publish, subscribe, threadsafe). Revision 16.

A simple, flexible, general-purpose observer pattern.

Observers can be callable objects or objects with a particular named method (handle_notify() by default). Events can be any object, and observers can select which events they are interested in receiving. Support for a number of different types of lightweight event objects is included.

Simple signal library, similar to PyQT signals (Python)

2017-04-01T21:11:50-07:00

Python recipe 580764 by Miguel Martínez López (event, pubsub, signal). Revision 11.

Simple signal library similar to PyQT signals. Signals helps to decouple code in GUI applications. This code could be used in Tkinter applications for example.

Inspired and based in these other modules:

https://github.com/shaunduncan/smokesignal

https://github.com/dgovil/PySignal

https://github.com/jek/blinker

Observer Design Pattern for python gevent coroutine package (Python)

2010-12-08T08:33:30-08:00

Python recipe 577491 by Andrey Nikishaev (event, gevent, observer, python).

This is simple implementation of the observer design pattern. Acting as a registration hub, it fires events when requested. Also i have gevent.Timeout like interface in situations when you need to run event-method in the same greenlet. Example:

e = Observer()
ev = e.wait('kill')
try:
    gevent.sleep(3)
except FiredEvent:
    print 'Fired!'
else:
    print 'Not Fired!'
finally:
    ev.cancel()

But rememeber, if you are using subscribe method, event-method will be executed in another greenlet.

Connect PyGTK object events to class methods automatically (Python)

2011-02-19T17:50:42-08:00

Python recipe 577577 by Pavel Krc (automate, connect, decorator, event, pygtk). Revision 2.