Welcome, guest | Sign In | My Account | Store | Cart
1

a webserver which serves files from a specified directory and transforms files containing reStructuredText to HTML on the fly.

Python, 28 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
PORT = 8080                                                                     
PATH = '/tmp/'                                                                  
EXTENSION = '.rst'                                                              
                                                                                
from twisted.web.resource import Resource                                       
from twisted.web import server                                                  
from twisted.web import static                                                  
from twisted.internet import reactor
                                            
from docutils.core import publish_string                                                                  
                                                      
class ReStructured( Resource ):                                                                                                                                 
    def __init__( self, filename, *a ):                                         
        self.rst = open( filename ).read( )                                                                                                               
                                                                                
    def render( self, request ):                                                
        return publish_string( self.rst, writer_name = 'html' )                 
                                                                                

resource = static.File( PATH )                                                  
resource.processors = { EXTENSION : ReStructured }                              
resource.indexNames = [ 'index' + EXTENSION ]                                   
                                                                                
reactor.listenTCP(                                                              
        PORT,                                                                   
        server.Site( resource )                                                 
        )                                                                       
reactor.run( )            

i'm using reStructuredText files for some simple websites and was looking for a way to serve them without having to transform them to HTML manually. since i already use twisted as a webserver for this sites i just needed a simple preprocessor.

4 comments

Michele Simionato 9 years, 6 months ago  # | flag

is there any caching here? I really like this recipe. However, I wonder what happens if 1000 users call the same page: is the rendering being executed 1000 times or is there some caching mechanism built in the Resource class? If not, I guess I would need to change the render method in something like this:

def render(self, request):
    try:
        return self.html
    except AttributeError:
        self.html = publish_string(self.rst, writer_name = 'html')
        return self.html

Also, what happens for long pages such that the rendering takes a long time? Should I execute the rendering in a separate thread? I really don't know what Resource does under the hood.

              Michele Simionato
Michele Simionato 9 years, 6 months ago  # | flag

thinking a bit more ... After some experiments, I see that twisted does not cache the resources by default. Actually, each time I click on the link the a new resource object is created and the .rst file is re-read. Assuming the .rst file does not change often, this is a waste. A solution would be to memoize resource creation with a Memoize metaclass:

class Memoize(type):
    @memoize # a memoize decorator, there are many in the cookbook
    def __call__(cls, *args):
        return super(Memoize, cls).__call__(*args)

class ReStructured(Resource, object):
    __metaclass__ = Memoize
    def __init__(self, filename, *args):
        self.rst = open(filename).read()
        self.html = publish_string(self.rst, writer_name = 'html')
    def render(self, request):
        return self.html

Now one should add an "update" method to reinitialize the resource object when the .rst source file changes on the filesystem, but that is easy.

Andrew Bennetts 9 years, 6 months ago  # | flag

Twisted's Resource doesn't thread. So if a .rst file is large enough, it'll block all other activity in that Twisted server.

The simplest way to fix that would be:

from twisted.internet.threads import deferToThread
from twisted.python import log
from twisted.web.server import NOT_DONE_YET
...

    def render(self, request):
        d = deferToThread(publish_string, self.rst, writer_name='html')
        d.addCallback(request.write)
        d.addCallback(lambda _: request.finish())
        d.addErrback(log.err)
        return NOT_DONE_YET
Markus Gritsch 9 years, 6 months ago  # | flag

Snakelets version. This is a version of this recipe for the Snakelets web application server.

  • Download and extract Snakelets from http://snakelets.sourceforge.net/

  • Create a new directory in the 'webapps' folder named 'rest'

  • Create a file named '__init__.py' containing the code below in the 'rest' folder

  • Run Snakelets 'serv.py' script

  • Point your browser at http://localhost:9080/rest/

    from snakeserver.snakelet import Snakelet from docutils.core import publish_string

    class ReStructured(Snakelet): def serve(self, request, response): filepath = self.getWebApp().getFileSystemPath() + '/..' + \ request.getRequestURL().replace('%20', ' ').rsplit('.', 1)[0] + '.txt' f = open(filepath, 'rt'); cont = f.read(); f.close() response.getOutput().write(publish_string(cont, writer_name = 'html'))

    name='Snakelets reStructuredText Webapp' docroot='.' snakelets= { '.txt': ReStructured, # reStructuredText files have this extension '.html': ReStructured # to be able to navigate in the generated page }

    def dirListAllower(path): return True

Add a comment

Sign in to comment