ActiveState Code

Recipe 413609: Twisted reStructuredText Server


a webserver which serves files from a specified directory and transforms files containing reStructuredText to HTML on the fly.

Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
PORT = 8080                                                                     
PATH = '/tmp/'                                                                  
EXTENSION = '.rst'                                                              
                                                                                
from twisted.web.resource import Resource                                       
from twisted.web import server                                                  
from twisted.web import static                                                  
from twisted.internet import reactor
                                            
from docutils.core import publish_string                                                                  
                                                      
class ReStructured( Resource ):                                                                                                                                 
    def __init__( self, filename, *a ):                                         
        self.rst = open( filename ).read( )                                                                                                               
                                                                                
    def render( self, request ):                                                
        return publish_string( self.rst, writer_name = 'html' )                 
                                                                                

resource = static.File( PATH )                                                  
resource.processors = { EXTENSION : ReStructured }                              
resource.indexNames = [ 'index' + EXTENSION ]                                   
                                                                                
reactor.listenTCP(                                                              
        PORT,                                                                   
        server.Site( resource )                                                 
        )                                                                       
reactor.run( )            

Discussion

i'm using reStructuredText files for some simple websites and was looking for a way to serve them without having to transform them to HTML manually. since i already use twisted as a webserver for this sites i just needed a simple preprocessor.

Comments

  1. 1. At 10:49 p.m. on 9 may 2005, Michele Simionato said:

    is there any caching here? I really like this recipe. However, I wonder what happens if 1000 users call the same page: is the rendering being executed 1000 times or is there some caching mechanism built in the Resource class? If not, I guess I would need to change the render method in something like this:

    def render(self, request):
        try:
            return self.html
        except AttributeError:
            self.html = publish_string(self.rst, writer_name = 'html')
            return self.html
    

    Also, what happens for long pages such that the rendering takes a long time? Should I execute the rendering in a separate thread? I really don't know what Resource does under the hood.

                  Michele Simionato
    
  2. 2. At 11:43 p.m. on 9 may 2005, Michele Simionato said:

    thinking a bit more ... After some experiments, I see that twisted does not cache the resources by default. Actually, each time I click on the link the a new resource object is created and the .rst file is re-read. Assuming the .rst file does not change often, this is a waste. A solution would be to memoize resource creation with a Memoize metaclass:

    class Memoize(type):
        @memoize # a memoize decorator, there are many in the cookbook
        def __call__(cls, *args):
            return super(Memoize, cls).__call__(*args)
    
    class ReStructured(Resource, object):
        __metaclass__ = Memoize
        def __init__(self, filename, *args):
            self.rst = open(filename).read()
            self.html = publish_string(self.rst, writer_name = 'html')
        def render(self, request):
            return self.html
    

    Now one should add an "update" method to reinitialize the resource object when the .rst source file changes on the filesystem, but that is easy.

  3. 3. At 5:52 p.m. on 12 may 2005, Andrew Bennetts said:

    Twisted's Resource doesn't thread. So if a .rst file is large enough, it'll block all other activity in that Twisted server.

    The simplest way to fix that would be:

    from twisted.internet.threads import deferToThread
    from twisted.python import log
    from twisted.web.server import NOT_DONE_YET
    ...
    
        def render(self, request):
            d = deferToThread(publish_string, self.rst, writer_name='html')
            d.addCallback(request.write)
            d.addCallback(lambda _: request.finish())
            d.addErrback(log.err)
            return NOT_DONE_YET
    
  4. 4. At 10:01 a.m. on 16 may 2005, Markus Gritsch said:

    Snakelets version. This is a version of this recipe for the Snakelets web application server.

    • Download and extract Snakelets from http://snakelets.sourceforge.net/

    • Create a new directory in the 'webapps' folder named 'rest'

    • Create a file named '__init__.py' containing the code below in the 'rest' folder

    • Run Snakelets 'serv.py' script

    • Point your browser at http://localhost:9080/rest/

      from snakeserver.snakelet import Snakelet from docutils.core import publish_string

      class ReStructured(Snakelet): def serve(self, request, response): filepath = self.getWebApp().getFileSystemPath() + '/..' + \ request.getRequestURL().replace('%20', ' ').rsplit('.', 1)[0] + '.txt' f = open(filepath, 'rt'); cont = f.read(); f.close() response.getOutput().write(publish_string(cont, writer_name = 'html'))

      name='Snakelets reStructuredText Webapp' docroot='.' snakelets= { '.txt': ReStructured, # reStructuredText files have this extension '.html': ReStructured # to be able to navigate in the generated page }

      def dirListAllower(path): return True

Sign in to comment