ActiveState Code

Recipe 305277: Import modules from a remote server.

Storing modules in a central location, which are used by remote, seperate clients has obvious benefits. This recipe describes a method by which modules can be fetched from a remote server and compiled into bytecode modules which are used by the client.

The recipe has 3 sections (server.py, client.py, test.py) which need to be copied into 3 files.

Python 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
#server.py
from SimpleXMLRPCServer import SimpleXMLRPCServer

def export(module):
    exec('import %s as m' % module)
    filename = m.__file__[:-1] #make pyc = py
    return open(filename).read()

server = SimpleXMLRPCServer(('localhost',1979))
server.register_function(export)
server.serve_forever()

#client.py
from xmlrpclib import ServerProxy
import sha

def remoteImport(module):
    """
    Import a module from a remote server.
    Returns a module object.
    """
    server = ServerProxy('http://localhost:1979')
    filename = sha.new(module).hexdigest() #create a temporary filename
    try:
        code = server.export(module)
    except: #if anything goes wrong, try and read from a (possibly) cached file.
        try:
            code = open('%s.py' % filename).read()
        except IOError: #if we don't have a cached file, raise ImportError
            raise ImportError, 'Module %s is not available.' % module
    #dump the code to file, import it and return the module
    open(filename+'.py','w').write(code)
    exec('import %s as m' % filename)
    return m

if __name__ == "__main__":
    m = remoteImport('test')
    print m.add(1,3)

#test.py
def add(a,b):
    return a + b

Discussion

This solution is ideal for storing configuration modules, which can be downloaded by client applications, when the client application is run. If a module cannot be retrieved, a cached copy from a previous session is used.

Modules can be replaced at any time, as long as they maintain the previous module interface.

This recipe uses XMLRPC for transport, however, it could easily use HTTP. This recipe would also benefit (read, needs) from application or transport level authentication and security.

Comments

  1. 1. At 2:28 a.m. on 16 sep 2004, David Boddie said:

    Remote importing. This could be extended to allow remote modules to be imported through an import hook. The xhtmlhook module enables this by allowing URLs to be specified in the sys.path lists:

    http://www.boddie.org.uk/python/xhtmlhook/

    There are obviously lots of other things being done in that module, but the remote importing could be stripped out and reused here.

  2. 2. At 1:34 a.m. on 12 mar 2006, jure vrscaj said:

    urlimport. This one uses path hooks to import remote modules: http://urlimport.codeshift.net/.

Sign in to comment