Welcome, guest | Sign In | My Account | Store | Cart
6

Although Python's thread syntax is nicer than in many languages, it can still be a pain if all one wants to do is run a time-consuming function in a separate thread, while allowing the main thread to continue uninterrupted. A Future provides a legible and intuitive way to achieve such an end.

Python, 41 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
from threading import *
import copy

class Future:

    def __init__(self,func,*param):
        # Constructor
        self.__done=0
        self.__result=None
        self.__status='working'

        self.__C=Condition()   # Notify on this Condition when result is ready

        # Run the actual function in a separate thread
        self.__T=Thread(target=self.Wrapper,args=(func,param))
        self.__T.setName("FutureThread")
        self.__T.start()

    def __repr__(self):
        return '<Future at '+hex(id(self))+':'+self.__status+'>'

    def __call__(self):
        self.__C.acquire()
        while self.__done==0:
            self.__C.wait()
        self.__C.release()
        # We deepcopy __result to prevent accidental tampering with it.
        a=copy.deepcopy(self.__result)
        return a

    def Wrapper(self, func, param):
        # Run the actual function, and let us housekeep around it
        self.__C.acquire()
        try:
            self.__result=func(*param)
        except:
            self.__result="Exception raised within Future"
        self.__done=1
        self.__status=`self.__result`
        self.__C.notify()
        self.__C.release()

To run a function in a separate thread, simply put it in a Future:

>>> A=Future(longRunningFunction, arg1, arg2 ...)

It will continue on its merry way until you need the result of your function. You can read the result by calling the Future like a function, for example:

>>> print A()

If the Future has completed executing, the call returns immediately. If it is still running, then the call blocks until the function completes. The result of the function is stored in the Future, so subsequent calls to it return immediately.

A few caveats: Since one wouldn't expect to be able to change the result of a function, Futures are not meant to be mutable. This is enforced by requiring the Future to be "called", rather than directly reading __result. If desired, stronger enforcement of this rule can be achieved by playing with __getattr__ and __setattr__.

The Future only runs the function once, no matter how many times you read it. You will have to re-create the Future if you want to re-run your function; for example, if the function is sensitive to the time of day.

For more information on Futures, and other useful parallel programming constructs, read Gregory V. Wilson's _Practical Parallel Programming_.

12 comments

Matthew King 14 years, 7 months ago  # | flag

what about exceptions? cool idea. minor nit... if the function being run in a Future generates an exception the Condition is never released...

maybe a try / finally around the self.__result=func(*param)?

David Perry (author) 14 years, 3 months ago  # | flag

Re: what about exceptions? That's a very good point, thanks for pointing it out. A try/finally where you suggested is probably the best way to deal with it. I'll give it a shot.

David Perry (author) 14 years, 3 months ago  # | flag

Fixed. And done. I made it a try / except, rather than try / finally, so the Future can report an error message rather than dying silently.

H. Krekel 14 years, 3 months ago  # | flag

very nice. This is really nice. it's a pity you can't easily stop the function from where you started the future. Sometimes one would certainly like timeouts or other termination conditions.

Jerry McRae 14 years, 3 months ago  # | flag

Am I done? If a thread takes a REAL long time, one might want to query the thread to see if it is done. Something like this (in the class) should work:

def isDone(self): return self.__done

If one stored the starting time, one could query the lenght of time the thread has been running.

Question. Is there any way that the actual running thread can pass information back to the instance of Future? So I could know it is working on iteration 23 of 100, or some such?

Bjorn Pettersen 13 years, 11 months ago  # | flag

Exception handling. In my version I added self.__excpt = None in __init__, then in __call__ I added (just before copying the result):

# an exception was thrown in the thread, re-raise it here.
if self.__excpt: raise self.__excpt

and finally in Wrapper, I added an extra layer of exception handling:

try:
    self.__result == func(*param)
except Exception, e: # we got an exception object, save it.
    self.__excpt = e
except:
    self.__result = "Unknown exception raised within Future"

Assuming the user inherits his exception from Exception (which he is supposed to), this will make futures semi transparent in exception cases, i.e. the exception will be thrown when the user fetches the value of the future.

John Costello 12 years ago  # | flag

Back-communication is simple: just pass the 'self' object into the function in Wrapper, and it will be able to assign to an appropriate property on the Future instance.

post 11 years, 6 months ago  # | flag

Maximum number of feeds. Is there a maximum number of feeds this can handle?

Graham Horler 11 years, 4 months ago  # | flag

Better exceptions. In my version, Wrapper() does this:

try:
    self.__result = func(*args, **kwargs)
except:
    self.__result = "Exception raised within Future"
    self.__excpt = sys.exc_info()

and __call__() does this:

if self.__excpt:
    raise self.__excpt[0], self.__excpt[1], self.__excpt[2]

This way, all exceptions are caught, regardless of inheriting from Exception, and you even get a nice traceback (with a little gc expense).

Brian Quinlan 7 years, 1 month ago  # | flag

I'm working on a more complete implementation at: http://code.google.com/p/pythonfutures

Trent Mick 6 years ago  # | flag

Note: Brian Quinlan's work seems destined for the Python 3.2 core (as concurrent.futures)! http://www.python.org/dev/peps/pep-3148/

Imre 1 year ago  # | flag

I guess there is a minor typo in line 39:

self.__status=self.__result

My machine reported an error because of those accent graves, I had to replace them with the standard apostrophes (').

Add a comment

Sign in to comment

Created by David Perry on Tue, 30 Oct 2001 (PSF)
Python recipes (4482)
David Perry's recipes (1)

Required Modules

Other Information and Tasks