Popular recipes tagged "meta:requires=select"

Primitive Peer to Peer Chat (Python)

2013-07-07T02:09:57-07:00

Python recipe 578591 by teddy_k (chat, peer, select, socket, threading).

This took me longer than it should have. I am putting it here in the hopes that this post will spare others from having to Google things for extended periods of time.

In short, you either listen for a connection (Chat_Server), or connect to a remote IP address (Chat_Client). From there, you can send text strings back and forth.

This is a bit rough-hewn, obviously; I apologize in advance.

Distributed lock manager for Python (Python)

2012-07-04T21:03:32-07:00

Python recipe 578194 by pavel (distributed, lock, locking, locks, manager, socket, thread, threading, wsgi). Revision 2.

Distributed lock manager provides mutex(es) over network. It is used to synchronize processes running on different machines, e.g. WSGI processes in case of web applications. Lock object is compatible with threading.Lock and can be used as a context manager ("with statement"). It can be easily modified to use UNIX sockets instead of TCP/IP. Communication protocol is text based.

First start server process:

$ chmod +x dlm.py
$ ./dlm.py

Usage:

from dlm import LockClient

client = LockClient('localhost', 27272, 'client_name')

lock = client.mkLock('lock_name')

lock.acquire()
# critical section here...
lock.release()

# using context manager
with lock:
    # critical section here...

Multi threaded SMTP proxy (Python)

2010-06-12T22:27:00-07:00

Python recipe 577260 by Lobsang (mail, proxy, smtp). Revision 2.

This smtp proxy can be used to process any part of the message (header and body). It is also possible to process all the body part just before it is send to the MTA.

The aim of this proxy is to allow a modification of the message on the fly. It had been tested with the postfix before queue content filter

POSIX context manager to temporarily silence, or filter lines from stdout (Python)

2010-03-03T06:17:23-08:00

Python recipe 577083 by pwaller (contextmanager, freopen, silence, stdout, with). Revision 2.

Fed up with libraries you don't have control over emitting text into your precious stdout?

If they use stdout through python, then you can just change sys.stdout to be something else. If they are printing directly to stdout through a C module, or some other means, then you are stuck.

.. at least until you discover the with silence(): block!

Caveats: Non-portable, tested only on 2.6 under Linux, uses threading.

Example output:

$ python silence_file.py 
Before with block..
Sensible stuff!
After the silence block

Subprocess with async I/O pipes class (Python)

2009-05-17T02:02:04-07:00

Python recipe 576759 by Mike Kazantsev (asynchronous, ipc, non_blocking_i_o, pipe, subprocess). Revision 2.

Just stumbled upon the need to move data chunks between subprocesses in a non-linear way with some logic in-between, so tee(1) and fifo(7)'s weren't too good option. Inspired by 440554, but rewritten from scratch to remove unnecessary delays due to sleep(3) calls and suboptimal try/sleep-based polling.

Chat server & client using select.select (Python)

2007-09-28T08:57:53-07:00

Python recipe 531824 by Anand (network). Revision 3.

This recipe demos how to write a simple command line chat server & client using multiplexing using select.select. The server uses select call to multiplex multiple clients and the client uses it to multiplex command line & socket I/O.

how to implement kbhit() on Linux (Python)

2008-05-03T05:13:00-07:00

Python recipe 572182 by Nelson Rush . Revision 2.

kbhit on linux, the sample just prints dots until you press any key. I updated the sample to show how to implement unbuffered getch and getche safely.

Multithreaded Multiplexer/Demultiplexer to allow multiple transactions to be performed in parallel on a single channel (Python)

2008-10-04T00:12:00-07:00

Python recipe 576520 by david decotigny (demultiplexer, multiplexer, thread, transaction). Revision 8.

Imagine you have one wire and you want to send a message from a machine A to another machine B, and wait for an answer from B (a "request/response transaction" in the following). Now imagine you have many threads on A which can make this kind of transaction "simultaneously", then you have to be cautious, otherwise you might get the wrong responses for your transactions. One solution is to serialize the transactions: the threads on A will have to wait for their turn to make their request/response transaction.

Another solution is to map a thread requesting a transaction on A, to a thread handling it on B. This allows for several transactions to be in flight at the same time on the same wire. This is what this module does: on A lives a multiplexer and threads call its transaction() method to initiate the request/response. On B, process_transaction() of a demultiplexer will get called by worker threads. All this in a multi-threaded way on a single wire. The module takes care of managing the interleaving requests/responses.

CallPipe: Call the methods of an object in a remote process (Python)

2008-09-17T17:33:36-07:00

Python recipe 576509 by david decotigny (call, multiprocessing, pipe, processing, remote). Revision 6.

I am process A and I own object Obj. Now I decide to fork(): process B is born.

How do I do to make process B call the methods of A's object Obj (and not the methods of its own copy of Obj...) ?

This is what this module does: you create the "CallPipe" for Obj prior to fork()ing, and then process B can call any method of A's object Obj through it.

unix subprocess wrapper (Python)

2008-07-29T07:11:17-07:00

Python recipe 576387 by Pádraig Brady (group, popen, subprocess, unix). Revision 2.

I have used this for ages to control child processes (and all their children). Some of the existing subprocess module was based on this, but I find this simpler for my uses at least.

Example:

import subProcess
process = subProcess.subProcess("your shell command")
process.read() #timeout is optional
handle(process.outdata, process.errdata)
del(process)

Access mplayer (slave mode) from python (Python)

2008-01-14T16:09:13-08:00

Python recipe 542195 by Fabien Devaux .

Light but somehow easy to use interface to access mplayer process. It works with introspection, I hope the class is self-explanatory. I tested it under Linux only.

Asyncore/asynchat in GTK (Python)

2008-03-17T13:51:05-07:00

Python recipe 551774 by Niklas Janlert (ui).

This recipe makes it possible to run asyncore/asynchat dispatchers in the GTK main thread.

A stoppable XMLRPCServer (Python)

2007-05-17T02:04:48-07:00

Python recipe 520583 by Eli Golovinsky (network).

SimpleXMLRPCServer is too simple. It's serve_forever function doesn't allow it to be incorporated into anything that needs to be gracefully stopped at some point. This code extends SimpleXMLRPCServer with the server_until_stopped and stop_serving functions.

A generic, multi protocol asynchronous server (Python)

2007-04-05T05:35:23-07:00

Python recipe 511453 by Pierre Quentel (web).

This recipe implements the concept of asynchronous socket programming in a more straighforward way than in the modules asyncore and asynchat of the standard distribution, which I find difficult to understand

Module For Running Simple Proxies (Python)

2007-02-01T21:51:53-08:00

Python recipe 502204 by Stephen Chappell (network).

Proxies can be useful at times but may not be simple to create and run. This recipe provides a single class that can build proxy objects capable of being both started and stopped.

Module to allow Asynchronous subprocess use on Windows and Posix platforms (Python)

2006-12-01T17:30:02-08:00

Python recipe 440554 by Josiah Carlson (sysadmin). Revision 10.

The 'subprocess' module in Python 2.4 has made creating and accessing subprocess streams in Python relatively convenient for all supported platforms, but what if you want to interact with the started subprocess? That is, what if you want to send a command, read the response, and send a new command based on that response?

Now there is a solution. The included subprocess.Popen subclass adds three new commonly used methods: recv(maxsize=None), recv_err(maxsize=None), and send(input), along with a utility method: send_recv(input='', maxsize=None).

recv() and recv_err() both read at most maxsize bytes from the started subprocess. send() sends strings to the started subprocess. send_recv() will send the provided input, and read up to maxsize bytes from both stdout and stderr.

If any of the pipes are closed, the attributes for those pipes will be set to None, and the methods will return None.

v. 1.3 fixed a few bugs relating to *nix support v. 1.4,5 fixed initialization on all platforms, a few bugs relating to Windows support, added two utility functions, and added an example of how to use this module. v. 1.6 fixed linux _recv() and test initialization thanks to Yuri Takhteyev at Stanford. v. 1.7 removed _setup() and __init__() and fixed subprocess unittests thanks to Antonio Valentino. Added 4th argument 'tr' to recv_some(), which is, approximately, the number of times it will attempt to recieve data. Added 5th argument 'stderr' to recv_some(), where when true, will recieve from stderr. Cleaned up some pipe closing. v. 1.8 Fixed missing self. parameter in non-windows _recv method thanks to comment. v. 1.9 Fixed fcntl calls for closed handles.

Waitable, cross-process threading.Event class. (Python)

2006-10-10T11:18:34-07:00

Python recipe 498191 by David Wilson (threads).

This class implements an interface similar to threading.Event using os.pipe() as the mechanism for signalling. This allows the class to be used to wake up select.select() and select.poll() loops without having to peridiodically resume execution to check the status of an event.

IPv6 Multicast (Python)

2005-10-28T11:43:52-07:00

Python recipe 442490 by Shane Holloway (network).

IPv6 can be tricky in many ways, and multicast makes it even more fun. This recipe is intended to give you a working implementation to start from.

My first application server (Python)

2009-02-23T11:53:57-08:00

Python recipe 392879 by Pierre Quentel (web). Revision 8.

For those who want to start dynamic web programming, but don't know what to choose among the many Python web frameworks, this program might be a good starting point

ScriptServer is a minimalist application server, handling both GET and POST requests, including multipart/form-data for file uploads, HTTP redirections, and with an in-memory session management. It can run Python scripts and template files using the standard string substitution format

The scripts are run in the same process as the server, avoiding the CGI overhead. The environment variables are provided in the namespace where the script runs

To start the server, run

python ScriptServer.py

In your web browser, enter http://localhost, this will show you a listing of the directory. Add the scripts in the same directory as ScriptServer

Simple HTTP server based on asyncore/asynchat (Python)

2005-10-16T08:26:59-07:00

Python recipe 259148 by Pierre Quentel (web). Revision 8.

A simple HTTP Server, intended to be as simple as the standard module SimpleHTTPServer, built upon the asyncore/asynchat modules (uses non-blocking sockets). Provides a Server (copied from medusa http_server) and a RequestHandler class. RequestHandler handles both GET and POST methods and inherits SimpleHTTPServer.SimpleHTTPRequestHandler

It can be easily extended by overriding the handle_data() method in the RequestHandler class