Popular recipes tagged "threading" but not "python"

Run OS command with timeout on a list of files using several threads (Python)

2015-05-19T19:31:59-07:00

Python recipe 579056 by Antoni Gual (command, subprocess, threading). Revision 3.

This hack runs a command sequentially on a list of files using two simultaneous threads. If one of the commands takes more than a set time, it's killed and the program goes for the next file. EDITED to add some exception handling.

Primitive Peer to Peer Chat (Python)

2013-07-07T02:09:57-07:00

Python recipe 578591 by teddy_k (chat, peer, select, socket, threading).

This took me longer than it should have. I am putting it here in the hopes that this post will spare others from having to Google things for extended periods of time.

In short, you either listen for a connection (Chat_Server), or connect to a remote IP address (Chat_Client). From there, you can send text strings back and forth.

This is a bit rough-hewn, obviously; I apologize in advance.

Distributed lock manager for Python (Python)

2012-07-04T21:03:32-07:00

Python recipe 578194 by pavel (distributed, lock, locking, locks, manager, socket, thread, threading, wsgi). Revision 2.

Distributed lock manager provides mutex(es) over network. It is used to synchronize processes running on different machines, e.g. WSGI processes in case of web applications. Lock object is compatible with threading.Lock and can be used as a context manager ("with statement"). It can be easily modified to use UNIX sockets instead of TCP/IP. Communication protocol is text based.

First start server process:

$ chmod +x dlm.py
$ ./dlm.py

Usage:

from dlm import LockClient

client = LockClient('localhost', 27272, 'client_name')

lock = client.mkLock('lock_name')

lock.acquire()
# critical section here...
lock.release()

# using context manager
with lock:
    # critical section here...

threadbox.py (Python)

2012-10-09T22:40:09-07:00

Python recipe 578152 by Stephen Chappell (metaclass, threading). Revision 2.

Provide a way to run instance methods on a single thread.

This module allows hierarchical classes to be cloned so that their instances run on one thread. Method calls are automatically routed through a special execution engine. This is helpful when building thread-safe GUI code.

Win32 named mutex class for system-wide mutex (Python)

2011-07-15T21:12:40-07:00

Python recipe 577794 by Ben Hoyt (ctypes, lock, mutex, process, threading, win32, windows).

NamedMutex is a class for using Windows (Win32) named mutexes for system-wide locks. For example, we use these to lock system-wide log files that multiple processes can write to.

actorish decorator for making async code look more like sync one and a less blocking (Python)

2011-10-30T19:59:44-07:00

Python recipe 577931 by Przemyslaw Podczasi (actor, threading).

I like how gevent is making async code to look like sync but non blocking without all the ugly callbacks. I tried doing that with threads and object proxy (I found great one at: http://pypi.python.org/pypi/ProxyTypes written by Phillip J. Eby, and this is where the actual magic happens).

For every function that is decorated it returns a proxy and the io call (or anything else) won't block until the value is actually needed. (should be some pools and args pickling there, to make it more like message passing but I didn't want to fuzzy the example) To use it as actor model, I guess it would require to queue requests to decorated object's methods and create a single thread to process them an in LazyProxy callback set q.get() instead of t.join()

Future decorator in a pretty pythonic way (Python)

2011-10-20T16:51:48-07:00

Python recipe 577915 by Filippo Squillace (threading).

Without thinking in thread creation the idea is to call several times a function assigning a thread for each call with related parameters and returning the list of results in a pretty pythonic way. For example, if we have already defined the function 'func': res = func(par) we want to call that several times and get the values in the future. So: func(par1) func(par2) func(par3).get() func(par4).get_all()

assigning a thread for each call. The 'get' method returns the first possible results, and 'get_all' returns all the values in a list. The decorator works fine with kwargs too.

This recipe is based on: http://code.activestate.com/recipes/355651-implementing-futures-with-decorators/ that use only one call for the function. The problem in that recipe is that each call blocks the execution.

Vote if you like it!

Multi-threaded Mandelbrot Fractal (Python)

2011-05-01T17:33:20-07:00

Python recipe 577680 by FB36 (fractal, graphics, image, math, mathematics, thread, threading, threads). Revision 3.

Multi-threaded Mandelbrot Fractal.

Command Threads (Python)

2010-12-25T23:23:49-08:00

Python recipe 577505 by Shao-chuan Wang (threading). Revision 4.

A thread that accepts callable objects as "commands" or "jobs" to execute.

Multiprocess-safe logging file-handler + interprocess RLock (Python)

2010-09-22T17:30:10-07:00

Python recipe 577395 by Jan Kaliszewski (concurrency, file, file_lock, flock, handler, logging, process, rlock, threading, threadsafe). Revision 11.

A Python 2.x/3.x-compatibile multiprocess-safe logging file-handler (logging.FileHandler replacement, designed for logging to a single file from multiple independent processes) together with a simple interprocess recursive lock -- universal abstract classes + Unix/Linux implementation.

Update: It's is a deeply revised version. Especially, now it --

is Python 2.4, 2.5, 2.6, 3.1 -compatibile (previously Py>=2.6 was needed); probably works also with 2.7, 3.0 and 3.2 (but not tested if it does);
is multiprocess-safe as well as thread-safe (proviously thread safety within a process was missed);
is based on public interfaces only (previously FileHandler._open() was called and overriden);
implement full RLock instance interface, as documented for threading.RLock (previously non-blocking mode and context-manager interface were missing).

The module contains:

Unix/Linux-only example implementation (with flock-based locking): FLockRLock and FLockFileHandler classes.
universal abstract classes -- which may be useful at developing implementation for non-Unix platforms: MultiprocessRLock, MultiprocessFileHandler, LockedFileHandler,

Also a quick-and-dirty test was added.

It is still an alpha version -- I'll be very grateful for any feedback.

Further updates:

2010-09-20: Some corrections, especially: non-blocking mode bug in MultiprocessRLock.acquire() fixed; _test() function improved; plus fixes in the description below.
2010-09-22: _test() improved and moved to description section. Mistaken copyright-notice removed.

Decoupled Proxy of a State Bearing Object in a Multiprocessing Environment (Python)

2010-04-08T04:36:05-07:00

Python recipe 577180 by Mateyuzo (access, multiprocessing, process, processing, subprocess, synchronization, syncmanager, threading, __getattribute__). Revision 2.

Describes a simple example of the potential for shared Singleton or Borg object methods to be proxy'd by an object that SyncManger can present to requesting subprocesses.

Concurrent buffer for generators (Python)

2010-05-05T22:47:33-07:00

Python recipe 576999 by Javier Ruere (concurrency, generator, iterator, threading). Revision 2.

A buffer that will partially consume an iterator in the background.

Very useful for reading files and merging the data using the excellent http://code.activestate.com/recipes/491285/

Thread pool (grows on demand) (Python)

2009-11-03T02:28:12-08:00

Python recipe 576910 by Pepe Aracil (threading, threads, thread_pool). Revision 3.

Thread pool that grows on demand.

Simple threading decorator (Python)

2009-03-10T01:38:51-07:00

Python recipe 576684 by david.gaarenstroom (async, decorator, run_asynchronously, thread, threading). Revision 4.

When you're new at threading, using threads can be a bit daunting at first. If all you want is just to "run this function in parallel (= asynchronously) to the main program code", then this recipe can be of use. Simply use "@run_async" as a decorator for the function you want to run asynchronously. A call to that function will return immediately but the function itself will run in parallel.

pyGTK threading (Python)

2009-04-01T07:07:56-07:00

Python recipe 576705 by oohay_email2004 (pygtk, threading). Revision 2.

This is my first successful attempt at threading in pyGTK.

Suggestions please.

Multithreaded FIFO Gate (Python)

2008-09-08T08:29:14-07:00

Python recipe 576418 by Anand (semaphores, synchronization, system, threading). Revision 5.

While programming with multiple threads, sometimes one needs a construct which allows to suspend the execution of a set of running threads. This is normally required by an outside thread which wants to suspend the running threads for performing a specific action. The threads need to resume after the action in the same order in which they got suspended.

A thread gate (or gateway) allows you to do this. It acts like a gate through which only one thread can pass at a time. By default the gate is open, allowing all threads to "enter" the gate. When a thread calls "close", the gate is closed, blocking any threads which make a further call to "enter", till the gate is re-opened by the owner, whence the threads resume the order in which they got blocked.

The real-life parallel for this is a human operated level cross, which allows only one vehicle to pass at a time.