Clear "sys.modules" of stale code without having to restart your server. It's a hell of a lot harder to do right then it sounds.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
"""Clear ``sys.modules`` of specific types of modules if one is stale.""" __docformat__ = "restructuredtext" _lastModuleUpdate = time.time() def clearModules(): """Clear ``sys.modules`` of specific types of modules if one is stale. See ``properties.CLEAR_MODULES``. I took this method out of the ``InternalLibrary`` class so that you can call it *really* early, even before you create a ``Context`` to pass to ``InternalLibrary``. """ global _lastModuleUpdate if not properties.CLEAR_MODULES: return deleteTheseTypes = properties.CLEAR_MODULES if not isinstance(deleteTheseTypes, list): # Update Seamstress Exchange's properties file if you change this. deleteTheseTypes = ["aquarium.layout","aquarium.navigation", "aquarium.screen", "aquarium.widget"] deleteThese = [ moduleName for moduleType in deleteTheseTypes for moduleName in sys.modules.keys() if (moduleName == moduleType or moduleName.startswith(moduleType + ".")) ] for i in deleteThese: try: file = sys.modules[i].__file__ except AttributeError: continue if file.endswith(".pyc") and os.path.exists(file[:-1]): file = file[:-1] ST_MTIME = 8 if (_lastModuleUpdate < os.stat(file)[ST_MTIME]): staleModules = True break else: staleModules = False if staleModules: for i in deleteThese: # You can't modify a dictionary del sys.modules[i] # during an iteration. _lastModuleUpdate = time.time()
Note that this comes from the Aquarium Web application framework, but any references to Aquarium can easily be removed.
The problem that this method solves is simple: if I change a file, I don't want to have to restart the server. It's a simple problem, but it's tough to implement right. To prevent repeating mistakes, here's what has failed in the past:
Remove all modules from "sys.modules" on every page load.
- Some modules have state.
Delete only those modules that don't have state.
- There's no convenient way to know which ones have state.
Use module attributes.
- It's not convenient.
Delete only those modules that have grown stale.
- If a parent class gets reloaded, child classes in other modules will need to get reloaded, but we don't know which modules those classes are in.
Look through all the modules for module references to the modules that'll get deleted and delete those too.
- Its very common to only import the class, not the whole module. Hence, we still don't know which modules have child classes that need to get reloaded.
Just clear out "sys.modules" of all modules of certain types on every page load.
- Even a moderate amount of kiddie clicking will result in exceptions. I think the browsers hide the problem, but you'll see the exceptions in the logs.
Clear out "sys.modules" of all modules of certain types on every page load, but only if at least one of the modules is stale.
- This is good because it handles the kiddie clicking case, and it also handles the super class case.