Welcome, guest | Sign In | My Account | Store | Cart

Uses object proxying to expose a more intuitive interface to shelves. Requires the Proxy recipe (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496741).

Python, 114 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
import shelve


class InvalidationError(Exception):
    pass

class ShelfProxy(Proxy):
    __slots__ = ["_key", "_shelf", "_invalidated"]
    
    def __init__(self, obj, shelf, key):
        Proxy.__init__(self, obj)
        object.__setattr__(self, "_shelf", shelf)
        object.__setattr__(self, "_key", key)
        object.__setattr__(self, "_invalidated", False)
    
    def __del__(self):
        try:
            sync_proxy(self)
        except InvalidationError:
            pass

class ShelfWrapper(object):
    def __init__(self, shelf):
        self.__shelf = shelf
        self.__cache = {}
    
    def __del__(self):
        self.close()
        
    def __getattr__(self, name):
        return getattr(self.__shelf, name)
    
    def __contains__(self, key):
        return key in self.__shelf
    
    def __len__(self, key):
        return len(self.__shelf)
    
    def __delitem__(self, key):
        if key in self.__cache:
            object.__setattr__(self.__cache[key], "_invalidated", True)
            del self.__cache[key]
        del self.__shelf[key]
    
    def __getitem__(self, key):
        try:
            obj = self.__cache[key]
        except KeyError:
            self.__cache[key] = obj = ShelfProxy(self.__shelf[key], self.__shelf, key)
        return obj
    
    def __setitem__(self, key, value):
        if key in self.__cache:
            object.__setattr__(self.__cache[key], "_invalidated", True)
            self.__cache[key] = ShelfProxy(value, self.__shelf, key)
        self.__shelf[key] = value
    
    def sync(self):
        for obj in self.__cache.itervalues():
            try:
                sync_proxy(obj)
            except InvalidationError:
                pass
    
    def close(self):
        self.sync()
        self.__cache.clear()
        self.__shelf.close()


def sync_proxy(proxy):
    if object.__getattribute__(proxy, "_invalidated"):
        raise InvalidationError("the proxy has been invalidated (the key was reassigned)")
    shelf = object.__getattribute__(proxy, "_shelf")
    key = object.__getattribute__(proxy, "_key")
    obj = object.__getattribute__(proxy, "_obj")
    shelf[key] = obj
    shelf.sync()

def open(*args):
    return ShelfWrapper( shelve.open(*args) )


------ example ------
>>> db = open("blah.db")
>>> db["mylist"]=[1,2,3]
>>> db["mylist"].append(4)
>>> db["mylist"]
[1, 2, 3, 4]
>>> p = db["mylist"]
>>> type(p)
<class '__main__.ShelfProxy(list)'>
>>> p.append(5)
>>> p2 = db["mylist"]
>>> p2
[1, 2, 3, 4, 5]
>>> p2 is p
True

----- invalidation -----
When we reassign a key that have been proxies earlier, the proxy 
instance becomes invalidated, so it will not override the new value.

>>> db["mylist"] = 19
>>> sync_proxy(p)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "Proxy.py", line 152, in sync_proxy
    raise InvalidationError("the proxy has been invalidated (the key was reassigned)")
__main__.InvalidationError: the proxy has been invalidated (the key was reassigned)
>>>
>>> db["mylist"] += 1
>>> db["mylist"]
20

The shelve module is a nice interface to a database of python objects. The shelf looks like dictionary, where keys are strings, and the values are any picklable python objects. Getting an key retrieves its value from the database, and setting the key's value stores it in the database. However, shelf[key] returns a normal object... so changes you perform on it do not affect the database directly. You have to do obj = shelf[key] let's assume obj is a list obj.append("lala") now you must do shelf[key] = obj in order to change (replace) the in-database version object.

With this recipe, shelf[key] returns a proxy to the objects, so changes to the proxy are reflected both on the in-memory and in-database version of the object. When the object is deleted, it writes itself back into the database. See the example above for more details.

Created by tomer filiba on Fri, 26 May 2006 (PSF)
Python recipes (4591)
tomer filiba's recipes (12)

Required Modules

  • (none specified)

Other Information and Tasks