Welcome, guest | Sign In | My Account | Store | Cart

I use this for my database lookup function to minimize sql execution. It can also be useful in other contexts. I think it work even without "make_immutable", but it's probably safer this way. The class "DictTuple" is ugly. However, AFAIK, there are no ImmutableDict.

Python, 45 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
from datetime import date, time, timedelta

def make_immutable(value):
    if isinstance(value, (int, long, date, time, timedelta, basestring, buffer)): return value
    if isinstance(value, (list, tuple)): return tuple([make_immutable(x) for x in value])
    if isinstance(value, dict):
        class DictTuple(tuple): pass
        return DictTuple([(make_immutable(k), make_immutable(v)) for k, v in value.iteritems()])
    raise NotImplemented()

class CacheCaller(object):
    def __init__(self, acallable):
        self.acallable = acallable
        self.cache = {}
    def __call__(self, *args, **kwargs):
        key = (make_immutable(args), make_immutable(kwargs))
        if key in self.cache:
            result = self.cache[key]
        else:
            result = self.acallable(*args, **kwargs)
            self.cache[key] = result
        return result

def example():
    
    # slow operation
    def get_value(param1, param2, param3, param4):
        from time import sleep
        sleep(1)
        return '%s %s %s %s' % (param1, param2, param3, param4)

    # not a good idea to do it repeatly
    print get_value(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})
    print get_value(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})
    print get_value(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})
    print get_value(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})
    print get_value(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})

    # make it faster with cache
    caller = CacheCaller(get_value)
    print caller(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})
    print caller(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})
    print caller(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})
    print caller(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})
    print caller(1, 'string', [1,2,3], {'one': 1, 'two': 2, 'three': 3})

1 comment

Andreas Kloss 17 years, 3 months ago  # | flag

makeImmutable, class & instancemethods, etc. Disclaimer: This comment is for the recipe "Cache function/method results" (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/298337). If it gets placed wrongly by the comment system, well it's not my fault.

  1. No, it won't work without the makeImmutable. The keys of a dict must be immutable. If you wanted to skip the makeImmutable function, you'd need to use a different structure than a dict, for example a list of key, value tuples. But then you'd have to search through the list, a time consuming operation, which would certainly defeat the purpose of the caching. I used inspect.getmembers to get something immutable from other objects. It seems to work for me, but if anyone has a better trick, I'd appreciate that.

  2. The cached class only works with plain functions. If you try to wrap a method of a class into a cached object, you will find that your method turns into a staticmethod, that is, you have to feed the instance to it as a first argument.

    class X: def method(sef, *args): ... method = cached(method)

    i = X() i.method(i, arg1, ...) # ok i.method(arg1, ...) # error

This yields an error, because method is called without self argument. To avoid this, I created a method:

import new
def cachedmethod(method):
    return new.instancemethod(cached(method), None, method.im_class)

I can call this like:

class X:
    def method(self, *args):
        ...
X.method = cachedmethod(X.method)

This function has to be called outside the class, because if it is placed inside the class, the class is not built and thus can not be referenced by new.instancemethod. Is there a better way to do this?