Popular recipes tagged "concurrency"

Non-blocking readlines() (Python)

2014-06-30T20:30:57-07:00

Python recipe 578900 by Zack Weinberg (concurrency, input, line, nonblocking).

A generator function which takes a file object (assumed to be some sort of pipe or socket, open for reading), and yields lines from it without blocking. If there is no input available, it will yield an endless stream of empty strings until input becomes available again; caller is responsible for not going into a busy loop. (Newlines are normalized but not stripped, so if there is actually a blank line in the input, the value yielded will be '\n'.) The intended use case is a thread which must respond promptly to input from a pipe, and also something else which cannot be fed to select (e.g. a queue.Queue). Note that the file object is ignored except for its fileno.

Only tested on Unix. Only tested on 3.4; ought to work with any python that has bytearray, locale.getpreferredencoding, and fcntl.

A multithreaded, concurrent version of map() (Python)

2010-08-16T23:04:48-07:00

Python recipe 577360 by Wai Yip Tung (concurrency, multithreading, threads).

map() applies a function to a list of data sequentially. This is a variation to map that execute each function call concurrently in a thread.

Git Shell Script to enhance inline automation script... (Bash)

2010-11-01T00:43:03-07:00

Bash recipe 577447 by Patrick Riendeau (code, concurrency, git, oop, oriented, programs, property, property_creation, repository, shell, shelve).

This tiny effort, depend from Fnct.D ActiveState no.577446 Core, also available from github http://github.com/priendeau/Fnct.d, can develop basic methodology of implementing oriented program within uses of function re-declaration with function-parser to create both property-function to discover uses of getter-function and setter-function

Multiprocess-safe logging file-handler + interprocess RLock (Python)

2010-09-22T17:30:10-07:00

Python recipe 577395 by Jan Kaliszewski (concurrency, file, file_lock, flock, handler, logging, process, rlock, threading, threadsafe). Revision 11.

A Python 2.x/3.x-compatibile multiprocess-safe logging file-handler (logging.FileHandler replacement, designed for logging to a single file from multiple independent processes) together with a simple interprocess recursive lock -- universal abstract classes + Unix/Linux implementation.

Update: It's is a deeply revised version. Especially, now it --

is Python 2.4, 2.5, 2.6, 3.1 -compatibile (previously Py>=2.6 was needed); probably works also with 2.7, 3.0 and 3.2 (but not tested if it does);
is multiprocess-safe as well as thread-safe (proviously thread safety within a process was missed);
is based on public interfaces only (previously FileHandler._open() was called and overriden);
implement full RLock instance interface, as documented for threading.RLock (previously non-blocking mode and context-manager interface were missing).

The module contains:

Unix/Linux-only example implementation (with flock-based locking): FLockRLock and FLockFileHandler classes.
universal abstract classes -- which may be useful at developing implementation for non-Unix platforms: MultiprocessRLock, MultiprocessFileHandler, LockedFileHandler,

Also a quick-and-dirty test was added.

It is still an alpha version -- I'll be very grateful for any feedback.

Further updates:

2010-09-20: Some corrections, especially: non-blocking mode bug in MultiprocessRLock.acquire() fixed; _test() function improved; plus fixes in the description below.
2010-09-22: _test() improved and moved to description section. Mistaken copyright-notice removed.

Fast, re-entrant, optimistic lock implemented in Cython (Python)

2010-07-27T20:25:20-07:00

Python recipe 577336 by Stefan Behnel (concurrency, lock, locking). Revision 3.

This is a C-level implementation of a fast, re-entrant, optimistic lock for CPython. It is written in Cython. Under normal conditions, it is about 10x faster than threading.RLock because it avoids all locking unless two or more threads try to acquire it at the same time. Under congestion, it is still about 10% faster than RLock due to being implemented in Cython.

Generator Attributes (Python)

2010-02-18T17:23:59-08:00

Python recipe 577057 by Longpoke (concurrency, state).

A decorator that creates a generator factory that creates generators that can have attributes set on them (like a normal python class).

Concurrent buffer for generators (Python)

2010-05-05T22:47:33-07:00

Python recipe 576999 by Javier Ruere (concurrency, generator, iterator, threading). Revision 2.

A buffer that will partially consume an iterator in the background.

Very useful for reading files and merging the data using the excellent http://code.activestate.com/recipes/491285/

Simple lock-queue via Memcached (Python)

2008-11-19T01:32:07-08:00

Python recipe 576567 by Tal Einat (concurrency, contextmanager, lock, locking, memcache, memcached, queue).

A simple lock-queue (FIFO) context manager implemented with Memcached.

In essence this is a normal lock, where the requests to acquire the lock are granted in the order in which they were originally made. Note that requests to acquire the lock are always blocking.