Welcome, guest | Sign In | My Account | Store | Cart

Generator Based Concurrency, with Real Threads when Needed. (Python recipe) by S W
ActiveState Code (http://code.activestate.com/recipes/440670/)

This recipe uses NanoThreads. http://lgt.berlios.de/

It shows how simulated concurrency, (using generators as tasks), can be transparently combined with OS Level Python threads, as and when needed.

Python, 28 lines
Download
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import nanothreads
import time
import sys

def task_1():
    while True:
        print "Performing work in task_1"
        yield nanothreads.CONTINUE()
        #yielding nanothreads.UNBLOCK will perform 
        #the next iteration in a seperate thread.
        yield nanothreads.UNBLOCK()
        print "Simulating some Blocking IO..."
        time.sleep(10)
        print "Finished Blocking IO."
        yield nanothreads.CONTINUE()

def task_2():
    while True:
        print "Performing work in task_2"
        yield nanothreads.CONTINUE()
        time.sleep(1)


a = nanothreads.install(task_1())
b = nanothreads.install(task_2())
#defer an exit call for 11 seconds, so this test eventually stops.
nanothreads.defer_for(11, sys.exit)
nanothreads.loop()

The above code produces the following output:

Performing work in task_1 Performing work in task_2 Simulating some Blocking IO... Performing work in task_2 Performing work in task_2 Performing work in task_2 Performing work in task_2 Performing work in task_2 Performing work in task_2 Performing work in task_2 Performing work in task_2 Performing work in task_2 Finished Blocking IO. Performing work in task_2 Performing work in task_1 Performing work in task_2

which demonstrates how a single step of a generator task can be run inside a real thread, allowing other generator tasks to continue iterating.

Tags: threads

1 comment

Rod Montgomery 14 years, 3 months ago  # | flag

http://lgt.berlios.de/ seems to be dead.

The current surviving successor to nanothreads seems to be fibra at http://code.google.com/p/fibra/source/browse/trunk/README