Top-rated recipes tagged "generator"

Run asynchronous tasks using coroutines (Python)

2010-08-06T16:16:20-07:00

Python recipe 577129 by Arnau Sanchez (coroutine, event, generator, gobject, gtk, gui, network, nonblocking, pygtk). Revision 20.

This recipe shows a simple, transparent (and hopefully pythonic) way of running asynchronous tasks when writing a event-driven application (i.e. GUI). The aim is to allow a programmer to write time-consuming functions (usually IO-bound, but not only) with sequential-looking code, instead of scattering the logic over a bunch of callbacks. We will take advantage of the coroutines introduced in Python 2.5 (see http://www.python.org/dev/peps/pep-0342).

The goal: wouldn't it be great if we could write something like this?

def myjob(entry, arg1, arg2, arg3):
    result1 = function_that_takes_eons_to_complete(arg1, arg2)    
    result2 = another_function_that_downloads_a_big_really_big_file(result1, arg3)
    entry.set_text("The result is: %d" % result2)

def on_start_button___clicked(button, entry):
    myjob(entry, 1, 2, 3)

...    
gtk.main()

Indeed, but we can't! The GUI will hang until the job is done and the user will be rightfully angry. Coroutines to the rescue: the absolute minimal change we can make to this code is transforming myjob into a coroutine and yield every time we do blocking stuff:

def myjob(entry, arg1, arg2, arg3):
    result1 = yield some_task(arg1, arg2)    
    result2 = yield some_other_task(result1, arg3)
    entry.set_text("The result is: %d" % result2)

def on_start__clicked(button, entry):
    start_job(myjob(entry, 1, 2, 3))

some_task and some_other_task are here the asynchronous implementation of the sequential tasks used in the first fragment, and start_job the wrapper around the coroutine. Note that we still have to implement non-blocking versions of the tasks, but they are usually pretty generic (wait some time, download a file, ...) and can be re-used. If you happen to have a CPU-bound function or even a IO-bound code you cannot split (urllib2 anyone?), you can always use a generic threaded task (granted, the whole point of using co-routines should be avoiding threads, but there is no alternative here).

At the end, all the plumbing we need to make it work is just 1 function: start_job (wrapper around the job to manage the flow of the coroutine). The rest of the code -two asynchronous tasks (sleep_task, threaded_task) and a demo app- are shown solely as an example.

Wrap any iterable context manager so it closes when consumed (Python)

2012-11-19T20:10:35-08:00

Python recipe 578342 by Andrew Barnert (contextmanager, context_manager, generator, iterable, iterator, with_statement).

There are a few types in Python—most notably, files—that are both iterators and context managers. For trivial cases, these features are easy to use together, but as soon as you need to use the iterator lazily or asynchronously, a with statement won't help. That's where this recipe comes in handy:

send_async(with_iter(open(path, 'r')))

This also allows you to "forward" closing for a wrapped iterator, so closing the outer iterator also closes the inner one:

sync_async(line.upper() for line in with_iter(open(path, 'r')))

Left-handed password generator (Python)

2011-10-31T12:51:14-07:00

Python recipe 577930 by Alex (generator, lefthand, lefthanded, password, passwords). Revision 2.

Generate passwords that can easily be typed with only the left hand, very simple script

Retry loop (Python)

2013-05-23T20:45:01-07:00

Python recipe 578527 by Ryan Nowakowski (exception, generator, loop, retry, timeout).

Encapsulates the logic of a retry loop using a generator function.

Asynchronous subprocess using asyncore (Python)

2013-01-21T19:51:00-08:00

Python recipe 576957 by Glenn Eychaner (async, asynchronous, asyncore, coroutine, decorator, generator, ipc, subprocess). Revision 21.

A coroutine-based wrapper for subprocess.Popen that uses asyncore to communicate with child processes asynchronously. This allows subprocesses to be called from within socket servers or clients without needing a complicated event loop to check both. Uses recipe 576965 to provide the asynchronous coroutine framework, recipe 576967 to provide asynchronous pipes, and recipe 577600 to provide multiple alarms.

Run async code inline, nonblocking (Python)

2009-11-11T12:55:01-08:00

Python recipe 576952 by Thomas Ahle (asynchronous, closure, decorator, generator, threads). Revision 7.

A decorator, that makes it easy to switch between the mainthread and background threads.

Python Short URL Generator (Python)

2014-11-09T11:19:07-08:00

Python recipe 578961 by Des Wagner (deterministic, generator, short, tiny, unique, url).

Python implementation for generating Tiny URL- and bit.ly-like URLs.

And yet another round-robin generator (Python)

2013-11-13T19:30:29-08:00

Python recipe 578768 by Jan Müller (generator, itertools, python, roundrobin).

I know that there is a nice recipe already that even made it into the Python documentation, but this one is more concise and at the same time simpler.

>>> list(roundrobin('ABC', 'D', 'EF'))
['A', 'D', 'E', 'B', 'F', 'C']

Retry loop (Python)

2013-05-24T05:19:50-07:00

Python recipe 578163 by Oren Tirosh (exception, generator, loop, retry, timeout). Revision 4.

Encapsulates the logic of a retry loop using a generator function.

Password Generator (Python)

2011-10-14T13:35:21-07:00

Python recipe 577905 by Alexander Thomas Cruz (generator, password, randomization). Revision 4.

Generates a really strong and secure password. It uses multiple characters and symbols. Remember to score please! It really means a lot to me. Revision 4 includes an input so you can choose how many characters you want your password to have. It is also faster due to a suggestion by Garel Alex. Thanks!

Configurable password generator/checker (Python)

2008-11-10T01:50:54-08:00

Python recipe 576561 by Dario Lopez-Kästen (checker, generator, pasword, policy). Revision 2.

Yet another password generator module. This module contains utility functions for both generating and checking passwords in accordance to a (policy).

Default Password Properties Policy:

Minimum Password lenght is 8 tokens
At least four unique tokens
At least four character tokens
At least one number token
At least one special character token

grep in Python (Python)

2014-03-05T19:47:50-08:00

Python recipe 578845 by Andy Dustman (generator, grep, search).

The grep() function is inspired by UNIX grep, but is not limited to string patterns and files or streams.

Yet Another Python Generator... (Python)

2011-12-19T08:14:55-08:00

Python recipe 577990 by Barry Walker (audio, demo, generator, linux, signal, sound, source, sweep).

Aha, but not what big guns were expecting...

LF Audio Sweep Generator.

Another kids level project to do for yourselves...

This is a Python DEMO to show the power of the sound card using Linux to generate an Audio Sweep Signal from 4KHz down to 100Hz and back again.

Written in such a way that anyone can understand how it works... This is for Linux and Python 2.x.x. Read the code for much more information, and...... A Python 3.x.x version is here:-

http://www.linuxformat.com/forums/viewtopic.php?t=14411

Enjoy finding simple solutions to often VERY difficult problems...

Bazza, G0LCU...

Random Passwords (Python)

2010-07-27T20:28:17-07:00

Python recipe 577339 by Danillo Souza (generator, password, python, random).

Generate strong random passwords of specified length.

Windowing an iterable with itertools (Python)

2010-04-15T18:45:41-07:00

Python recipe 577196 by Daniel Cohn (generator, iterable, iterator, loop, peeking, window).

Oftentimes a programmer needs to peek into an iterator without advancing it, a task for which many good solutions already exist. But what if the intrepid coder needs a fast and pythonic way to 'window' the data? This recipe demonstrates how to wrap any iterable with a class that adds two methods, prev and peek.

Yet another Python implementation of PEP 380 (yield from) (Python)

2010-04-25T23:08:07-07:00

Python recipe 577153 by Arnau Sanchez (coroutine, decorator, generator, refactor). Revision 3.

Any Python programmer knows how extremely powerful generators are. Now (since version 2.5) Python generators can not only yield values but also receive them, so they can be used to build coroutines.

One drawback of the current implementation of generators is that you can only yield/receive values to/from the immediate caller. That means, basically, that you cannot easily refactor your code and write nested generators. PEP-380 is the most serious effort to overcome this issue, but until it gets approved we can still play around with pure Python implementations of yield from.

This recipe follows terminology used by others in the past (recipe566726, recipe576728), but I've tried to simplify the code as much as possible.

Concurrent buffer for generators (Python)

2010-05-05T22:47:33-07:00

Python recipe 576999 by Javier Ruere (concurrency, generator, iterator, threading). Revision 2.

A buffer that will partially consume an iterator in the background.

Very useful for reading files and merging the data using the excellent http://code.activestate.com/recipes/491285/

Python Short URL Generator (Python)

2011-09-19T14:06:36-07:00

Python recipe 576918 by Michael Fogleman (deterministic, generator, short, tiny, unique, url). Revision 3.

Python implementation for generating Tiny URL- and bit.ly-like URLs.

sin, cos, tan for Decimal (Python)

2007-07-05T19:49:49-07:00

Python recipe 523018 by Alain Mellan (algorithms, generator). Revision 2.

Implementation of sine, cosine, tangent functions for Decimal arithmetic, using Taylor series expansion. It uses simple numerator and denominator generators.

The nice part is, the code is independent from the Decimal library. Feed it a float, it works just the same as if you feed it a Decimal (apart from the precision :-)

Pluggable Python generators (Python)

2016-03-18T19:11:48-07:00

Python recipe 580628 by Vasudev Ram (generator, generators, python).

This is a simple recipe to show that Python generators are pluggable, i.e., they can be passed as arguments into functions, and then used inside those functions.