Welcome, guest | Sign In | My Account | Store | Cart

Twisted reStructuredText Server (Python recipe) by Siegmund Führinger
ActiveState Code (http://code.activestate.com/recipes/413609/)

a webserver which serves files from a specified directory and transforms files containing reStructuredText to HTML on the fly.

Python, 28 lines
Download
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
PORT = 8080                                                                     
PATH = '/tmp/'                                                                  
EXTENSION = '.rst'                                                              
                                                                                
from twisted.web.resource import Resource                                       
from twisted.web import server                                                  
from twisted.web import static                                                  
from twisted.internet import reactor
                                            
from docutils.core import publish_string                                                                  
                                                      
class ReStructured( Resource ):                                                                                                                                 
    def __init__( self, filename, *a ):                                         
        self.rst = open( filename ).read( )                                                                                                               
                                                                                
    def render( self, request ):                                                
        return publish_string( self.rst, writer_name = 'html' )                 
                                                                                

resource = static.File( PATH )                                                  
resource.processors = { EXTENSION : ReStructured }                              
resource.indexNames = [ 'index' + EXTENSION ]                                   
                                                                                
reactor.listenTCP(                                                              
        PORT,                                                                   
        server.Site( resource )                                                 
        )                                                                       
reactor.run( )

i'm using reStructuredText files for some simple websites and was looking for a way to serve them without having to transform them to HTML manually. since i already use twisted as a webserver for this sites i just needed a simple preprocessor.

Tags: web

4 comments

Michele Simionato 18 years, 11 months ago  # | flag

is there any caching here? I really like this recipe. However, I wonder what happens if 1000 users call the same page: is the rendering being executed 1000 times or is there some caching mechanism built in the Resource class? If not, I guess I would need to change the render method in something like this:

def render(self, request):
    try:
        return self.html
    except AttributeError:
        self.html = publish_string(self.rst, writer_name = 'html')
        return self.html

Also, what happens for long pages such that the rendering takes a long time? Should I execute the rendering in a separate thread? I really don't know what Resource does under the hood.

              Michele Simionato
Michele Simionato 18 years, 11 months ago  # | flag

thinking a bit more ... After some experiments, I see that twisted does not cache the resources by default. Actually, each time I click on the link the a new resource object is created and the .rst file is re-read. Assuming the .rst file does not change often, this is a waste. A solution would be to memoize resource creation with a Memoize metaclass:

class Memoize(type):
    @memoize # a memoize decorator, there are many in the cookbook
    def __call__(cls, *args):
        return super(Memoize, cls).__call__(*args)

class ReStructured(Resource, object):
    __metaclass__ = Memoize
    def __init__(self, filename, *args):
        self.rst = open(filename).read()
        self.html = publish_string(self.rst, writer_name = 'html')
    def render(self, request):
        return self.html

Now one should add an "update" method to reinitialize the resource object when the .rst source file changes on the filesystem, but that is easy.

Andrew Bennetts 18 years, 11 months ago  # | flag

Twisted's Resource doesn't thread. So if a .rst file is large enough, it'll block all other activity in that Twisted server.

The simplest way to fix that would be:

from twisted.internet.threads import deferToThread
from twisted.python import log
from twisted.web.server import NOT_DONE_YET
...

    def render(self, request):
        d = deferToThread(publish_string, self.rst, writer_name='html')
        d.addCallback(request.write)
        d.addCallback(lambda _: request.finish())
        d.addErrback(log.err)
        return NOT_DONE_YET
Markus Gritsch 18 years, 11 months ago  # | flag

Snakelets version. This is a version of this recipe for the Snakelets web application server.

  • Download and extract Snakelets from http://snakelets.sourceforge.net/

  • Create a new directory in the 'webapps' folder named 'rest'

  • Create a file named '__init__.py' containing the code below in the 'rest' folder

  • Run Snakelets 'serv.py' script

  • Point your browser at http://localhost:9080/rest/

    from snakeserver.snakelet import Snakelet from docutils.core import publish_string

    class ReStructured(Snakelet): def serve(self, request, response): filepath = self.getWebApp().getFileSystemPath() + '/..' + \ request.getRequestURL().replace('%20', ' ').rsplit('.', 1)[0] + '.txt' f = open(filepath, 'rt'); cont = f.read(); f.close() response.getOutput().write(publish_string(cont, writer_name = 'html'))

    name='Snakelets reStructuredText Webapp' docroot='.' snakelets= { '.txt': ReStructured, # reStructuredText files have this extension '.html': ReStructured # to be able to navigate in the generated page }

    def dirListAllower(path): return True