Stale Pickles ... New Classes. Pickles are by far the easiest way to achieve persistent objects in Python. A problem arises however, when a class is modified between the time when an object is pickled and when it is un-pickled. When a stale pickle is extracted, the old data is coupled with the new class code. This is not a problem as long as no new data attributes are required. For instance, a new method that relies only on old data may be added and still work with the old pickle.
Of course, if attributes are added, the old pickle files will no longer function correctly with the new class definition.
This recipe provides a framework for writing upgradable classes that support backward compatibility with stale pickles.
Download
Copy to clipboard1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | import pickle
class Upgradable:
class_version = '3.0'
def __init__(self):
self.version = self.__class__.class_version
self.new_attr = 42
def pickle(self):
return pickle.dumps(self)
def new_method(self):
'''
Return the answer to life the universe and everything.
Would normally break pickles prior to the introduction of new_attr.
'''
return self.new_attr
@staticmethod
def unpickle(data):
out = pickle.loads(data)
if not hasattr(out, 'version'):
out.version = '0.0'
if out.version != out.class_version:
out.upgrade()
return out
def upgrade(self):
version = float(self.version)
print 'upgrade from version %s' % self.version
if version < 1.:
self.version = '1.0'
print 'upgrade to version 1.0'
if version < 2.:
self.version = '2.0'
print 'upgrade to version 2.0'
if version < 3.:
self.version = '3.0'
self.new_attr = 42
print 'upgrade to version 3.0'
def __test__():
ug0 = Upgradable()
# downgrade to lower version
del ug0.version
del ug0.new_attr
try:
# breaks old pickles!
ug0.new_method()
raise Exception('Should have raised AttributeError!')
except AttributeError:
pass
# check to see if automatic upgrade works
ug3 = ug0.unpickle(ug0.pickle())
assert ug3.version == '3.0'
assert ug3.new_attr == 42
assert ug3.new_method() == 42
__test__()
|
I found this approach while upgrading my home-brew accounting software that uses pickle files to store account information between runs. When I upgraded the code, I broke the old pickles. Ad-hoc procedures to upgrade the pickles seem shoddy and error prone. This simple solution provides a framework that will support as many upgrades as you can dish out.
When the object is initialized, the current class version number is stored as a attribute. This attribute will be stored when the object is pickled and verified against the class_version used to de-pickle the object.
