Popular recipes tagged "nonblocking" but not "line"

Tail multiple pidgin IRC logfiles (Python)

2015-06-06T12:13:00-07:00

Python recipe 579066 by Anton Vredegoor (colorize, irc, iterators, logfiles, merging, nonblocking, pidgin, tail, threads). Revision 2.

Tail multiple pidgin IRC logfiles.

Pidgin should be connected to IRC with the channels one wants to tail joined, and it should save logs as text.

The script needs two arguments:

the directory containing the directories with channel logs

a list of channel names, quoted and separated by spaces

Example command:

python pidgin-irctail.py -d ~/.purple/logs/irc/YOUR_IRC_HANDLE@irc.freenode.net -c "#chan1 #chan2 #chan3"

Some text elements are higlighted, and channel names are inserted into the log lines after the time info.

If more than one channel is entered, the output of the logs is merged.

asyncore scheduler (Python)

2011-07-25T23:42:21-07:00

Python recipe 577808 by Giampaolo Rodolà (asynchronous, asyncore, heapq, nonblocking, scheduler, twisted). Revision 5.

The thing I miss mostly in asyncore is a system for calling a function after a certain amount of time without blocking. This is crucial for simple tasks such as disconnecting a peer after a certain time of inactivity or more advanced use cases such as bandwidth throttling.

This recipe was initially inspired by Twisted's internet.base.DelayedCall class:

http://twistedmatrix.com/trac/browser/tags/last_vfs_and_web2/twisted/internet/base.py#L34

...then included into pyftpdlib:

http://code.google.com/p/pyftpdlib/issues/detail?id=72

...and finally proposed for inclusion into asyncore:

http://bugs.python.org/issue1641

Run asynchronous tasks using coroutines (Python)

2010-08-06T16:16:20-07:00

Python recipe 577129 by Arnau Sanchez (coroutine, event, generator, gobject, gtk, gui, network, nonblocking, pygtk). Revision 20.

This recipe shows a simple, transparent (and hopefully pythonic) way of running asynchronous tasks when writing a event-driven application (i.e. GUI). The aim is to allow a programmer to write time-consuming functions (usually IO-bound, but not only) with sequential-looking code, instead of scattering the logic over a bunch of callbacks. We will take advantage of the coroutines introduced in Python 2.5 (see http://www.python.org/dev/peps/pep-0342).

The goal: wouldn't it be great if we could write something like this?

def myjob(entry, arg1, arg2, arg3):
    result1 = function_that_takes_eons_to_complete(arg1, arg2)    
    result2 = another_function_that_downloads_a_big_really_big_file(result1, arg3)
    entry.set_text("The result is: %d" % result2)

def on_start_button___clicked(button, entry):
    myjob(entry, 1, 2, 3)

...    
gtk.main()

Indeed, but we can't! The GUI will hang until the job is done and the user will be rightfully angry. Coroutines to the rescue: the absolute minimal change we can make to this code is transforming myjob into a coroutine and yield every time we do blocking stuff:

def myjob(entry, arg1, arg2, arg3):
    result1 = yield some_task(arg1, arg2)    
    result2 = yield some_other_task(result1, arg3)
    entry.set_text("The result is: %d" % result2)

def on_start__clicked(button, entry):
    start_job(myjob(entry, 1, 2, 3))

some_task and some_other_task are here the asynchronous implementation of the sequential tasks used in the first fragment, and start_job the wrapper around the coroutine. Note that we still have to implement non-blocking versions of the tasks, but they are usually pretty generic (wait some time, download a file, ...) and can be re-used. If you happen to have a CPU-bound function or even a IO-bound code you cannot split (urllib2 anyone?), you can always use a generic threaded task (granted, the whole point of using co-routines should be avoiding threads, but there is no alternative here).

At the end, all the plumbing we need to make it work is just 1 function: start_job (wrapper around the job to manage the flow of the coroutine). The rest of the code -two asynchronous tasks (sleep_task, threaded_task) and a demo app- are shown solely as an example.

Multicontext (e.g. asynchronous) inline execution framework using coroutines (Python)

2012-12-06T19:32:20-08:00

Python recipe 576965 by Glenn Eychaner (asynchronous, coroutine, decorator, generator, inline, nonblocking, pattern, thread). Revision 14.

A framework for executing inline code, contained in a generator, across multiple execution contexts, by pairing it with an executor that handles the context switching at each yield. An example of a generator which executes some iterations synchronously and some asynchronously is provided. The framework is general enough to be applied to many different coroutine situations.