Top-rated recipes tagged "web"

Http client to POST using multipart/form-data (Python)

2002-08-23T07:56:39-07:00

Python recipe 146306 by Wade Leftwich (web).

A scripted web client that will post data to a site as if from a form using ENCTYPE="multipart/form-data". This is typically used to upload files, but also gets around a server's (e.g. ASP's) limitation on the amount of data that can be accepted via a standard POST (application/x-www-form-urlencoded).

Simple HTTP server supporting SSL secure communications (Python)

2008-08-02T16:04:56-07:00

Python recipe 442473 by Sebastien Martini (https, openssl, ssl, web). Revision 8.

This recipe describes how to set up a simple HTTP server supporting SSL secure communications. It extends the SimpleHTTPServer standard module to support the SSL protocol. With this recipe, only the server is authenticated while the client remains unauthenticated (i.e. the server will not request a client certificate). Thus, the client (typically the browser) will be able to verify the server identity and secure its communications with the server.

This recipe requires you already know the basis of SSL and how to set up OpenSSL. This recipe is mostly derived from the examples provided with the pyOpenSSL package.

In order to apply this recipe, follow these few steps:

Install the OpenSSL package in order to generate key and certificate. Note: you probably already have this package installed if you are under Linux, or *BSD.
Install the pyOpenSSL package, it is an OpenSSL library binding. You'll need to import this module for accessing OpenSSL's components.
Generate a self-signed certificate compounded of a certificate and a private key for your server with the following command (it outputs them both in a single file named server.pem): openssl req -new -x509 -keyout server.pem -out server.pem -days 365 -nodes
Assuming you saved this recipe in SimpleSecureHTTPServer.py, start the server (with the appropriate rights): python SimpleSecureHTTPServer.py
Finally, browse to https://localhost, or https://localhost:port if your server listens a different port than 443.

A simple XML-RPC server (Python)

2001-10-13T11:34:19-07:00

Python recipe 81549 by Brian Quinlan (web).

This recipe demonstrates the creation of a simple XML-RPC server using the SimpleXMLRPCServer class. It requires either Python 2.2 or later or the XML-RPC package from PythonWare (http://www.pythonware.com/products/xmlrpc/index.htm) to run.

Simple Web Crawler (Python)

2011-01-31T21:57:58-08:00

Python recipe 576551 by James Mills (crawler, network, parsing, web). Revision 2.

NOTE: This recipe has been updated with suggested improvements since the last revision.

This is a simple web crawler I wrote to test websites and links. It will traverse all links found to any given depth.

See --help for usage.

I'm posting this recipe as this kind of problem has been asked on the Python Mailing List a number of times... I thought I'd share my simple little implementation based on the standard library and BeautifulSoup.

--JamesMills

Calculating the distance between zip codes (Python)

2006-04-25T20:40:00-07:00

Python recipe 393241 by Kevin Ryan (web). Revision 2.

I came across the mention of a formula labeled "The Great Circle Distance Formula" that purported to calculate the distance between any two points on the earth given their longitude and latitude points (the reference was in a Linux Magazine article). So, I looked up some information and cooked up a Python version of the calculation. There are references in the code where you can obtain approximate zip code data for free (e.g., if you wanted to enhance your website by adding a "Search within x mi's" feature), as well as references to the GCDF if you have further interest. Enjoy!

04/25/2006 update: I've decided to update this recipe with an object oriented bent where the information is cached once the object is instantiated. I've also added command line access to automatically download the zipcode file from the census website (just use 'python zips.py -d' and it will download a copy to your harddrive under 'zips.txt'). Lastly, I've added some unit testing so that if any future changes are made this will automatically run and tell me if anything pops out as wrong.

My first application server (Python)

2009-02-23T11:53:57-08:00

Python recipe 392879 by Pierre Quentel (web). Revision 8.

For those who want to start dynamic web programming, but don't know what to choose among the many Python web frameworks, this program might be a good starting point

ScriptServer is a minimalist application server, handling both GET and POST requests, including multipart/form-data for file uploads, HTTP redirections, and with an in-memory session management. It can run Python scripts and template files using the standard string substitution format

The scripts are run in the same process as the server, avoiding the CGI overhead. The environment variables are provided in the namespace where the script runs

To start the server, run

python ScriptServer.py

In your web browser, enter http://localhost, this will show you a listing of the directory. Add the scripts in the same directory as ScriptServer

Simple HTTP server based on asyncore/asynchat (Python)

2005-10-16T08:26:59-07:00

Python recipe 259148 by Pierre Quentel (web). Revision 8.

A simple HTTP Server, intended to be as simple as the standard module SimpleHTTPServer, built upon the asyncore/asynchat modules (uses non-blocking sockets). Provides a Server (copied from medusa http_server) and a RequestHandler class. RequestHandler handles both GET and POST methods and inherits SimpleHTTPServer.SimpleHTTPRequestHandler

It can be easily extended by overriding the handle_data() method in the RequestHandler class

HTML/CSS to PDF converter (Python)

2008-04-20T06:06:52-07:00

Python recipe 572160 by Dirk Holtwick (web).

Most people know how to write a page with HTML and CSS. Why not using these skills to dynamically generate PDF documents using it? The open source project "pisa" http://www.htmltopdf.org enables you to to this quite simple in a pythonic way.

Simple XML RPC server over HTTPS (Python)

2006-06-07T07:17:34-07:00

Python recipe 496786 by Laszlo Nagy (web).

Simple program that demonstrates how to write an XMLRCP server that uses https for transporting XML data.

Simple web request benchmark (Python)

2006-10-31T00:27:13-08:00

Python recipe 442412 by Josiah Carlson (web). Revision 4.

Recently, there has been a discussion between myself and another individual as to the performance of particular web servers under certain situations. Being that web server testing frameworks have different ways of measuring how long a 'request' takes, I thought I would take the guesswork out of it and measure everything explicitly.

There are generally 5 portions to a web request: 1. create the connection 2. start sending the request 3. finish sending the request 4. start recieving the response 5. finish reading the response

This recipe measures the amount of time to perform each portion of a request to a single file many times, and prints the results in a somewhat reasonable fashion.

Access password-protected web applications for scraping. (Python)

2005-04-05T20:29:34-07:00

Python recipe 391929 by George Geller (web). Revision 2.

Use John J. Lee's ClientCookie and ClientForm classes to easily access password-protected web applications. A group on yahoo.com is used as an example.

Rendering arbitrary objects with Nevow (Python)

2004-07-22T20:09:38-07:00

Python recipe 286260 by Valentino Volonghi (web). Revision 4.

This example is a simplified version of the irenderer example inside Nevow distrib by Matt Goodall (who deserves most of the glory for this example). It makes use of interfaces and adapters to render objects on a web page.

ActiveState recipe statistics (Python)

2011-06-02T14:52:50-07:00

Python recipe 577732 by Kaan Ozturk (html, regular_expressions, statistics, urllib2, web). Revision 2.

Downloads "All Recipe Authors" pages in ActiveState, uses regular expressions to parse author name and number of their recipes on each page. Finally, it displays the recipe submission distribution (the count of how many authors have submitted how many recipes each).

Image Downloader (Python)

2014-02-24T03:49:51-08:00

Python recipe 577385 by FB36 (html, http, url, urllib2, web). Revision 4.

Finds and downloads all images from any given URL.

Important note:

If your download location path has spaces then put quotes around it!

Method-based URL dispatcher for the Tornado web server (Python)

2009-11-20T11:51:47-08:00

Python recipe 576958 by Dan McDougall (dispatcher, shortcuts, subclass, tornado, url, web). Revision 5.

The MethodDispatcher is a subclass of tornado.web.RequestHandler that will use the methods contained in subclasses of MethodDispatcher to handle requests. In other words, instead of having to make a new RequestHandler class for every URL in your application you can subclass MethodDispatcher and use the methods contained therein as your URLs.

The MethodDispatcher also adds the convenience of automatically passing arguments to your class methods. So there is no need to use Tornado's get_argument() method.

Example

To demonstrate the advantages of using MethodDispatcher I'll present a standard Tornado app with multiple URLs and re-write it using MethodDispatcher...

The standard Tornado way

class Foo(tornado.web.RequestHandler):
    def get(self):
        self.write('foo')

class Bar(tornado.web.RequestHandler):
    def get(self):
        self.write('bar')

class SimonSays(tornado.web.RequestHandler):
    def get(self):
        say = self.get_argument("say")
        self.write('Simon says, %s' % `say`)

application = tornado.web.Application([
    (r"/foo", Foo),
    (r"/bar", Bar),
    (r"/simonsays", SimonSays),
])

The MethodDispatcher way

class FooBar(MethodDispatcher):
    def foo(self):
        self.write("foo")

    def bar(self):
        self.write("bar")

    def simonsays(self, say):
        self.write("Simon Says, %s" % `say`)

application = tornado.web.Application([
    (r"/.*", FooBar)
])

Notes

As you can see from the above example, using the MethodDispatcher can significantly reduce the complexity of Tornado applications. Here's some other things to keep in mind when using the MethodDispatcher:

MethodDispatcher will ignore any methods that begin with an underscore (_). This prevents builtins and private methods from being exposed to the web.
The '/' path is special: It always maps to self.index().
MethodDispatcher does not require that your methods distinquish between GET and POST requests. Whether a GET or POST is performed the matching method will be called with any passed arguments or POSTed data. Because of the way this works you should not use get() and post() in your MethodDispatcher subclasses unless you want to override this functionality.
When an argument is passed with a single value (/simonsays?say=hello) the value passed to the argument will be de-listed. In other words, it will be passed to your method like so: {'say': 'hello'}. This overrides the default Tornado behavior which would return the value as a list: {'say': ['hello']}. If more than one value is passed MethodDispatcher will use the default behavior.

Twisted XML RPC server with basic HTTP authentication (Python)

2007-08-09T09:48:48-07:00

Python recipe 526625 by Anand (web). Revision 3.

This recipe illustrates an XML RPC server using twisted with support for HTTP basic client authentication.

xmlrpc server/client which does cookie handling and supports basic authentication (Python)

2007-01-23T10:51:39-08:00

Python recipe 501148 by Vaibhav Bhatia (web).

xmlrpc server client which do the following: * client sends a request with basic authentication * server on successful authentication sends response back and also sends cookies with authentication id * these cookies are saved by client and then used for authentication on any subsequent requests

Builtin i18n _() function in a multi-threaded environment. (Python)

2006-01-02T10:57:10-08:00

Python recipe 465756 by Martin Blais (web).

Injecting _() in the __builtin__ module in order to inject global functions _() and N_() is common in applications which need i18n. This is a variation on the theme that does this in a multi-threaded environment, using threading.local from Python-2.4.

Yet another reinvention of a Python HTML generation mechanism (Python)

2005-11-22T07:25:44-08:00

Python recipe 440563 by Josiah Carlson (web). Revision 3.

The other day I was complaining about writing html, forms, etc., for Python cgi and/or web programming. I had pointed out a selection of three examples, the first of which ended up being very much like Nevow.stan . Thinking a bit about it, I realized that stan had issues in that you couldn't really re-use pre-defined tags with attributes via map, and keyword arguments were just too darn convenient to swap the calling and getitem syntax.

Instead, I hacked together a mechanism that supports: T.tagname("content", T.tagname(...), ..., attr1='value', ...) T.tagname(attr1='value', ...)("content", T.tagname(...), ...) x = T.tagname(attr1='value', ...) y = T.tagname(*map(x, ['content', ...])) ... and many other options.

Essentially, you can mix and match calls as much as you want, with three memory and sanity saving semantics: 1. Creating a new tag object via T.tagname, or any call of such, will create a shallow copy of the object you are accessing. 2. smallred = T.font(size='-1', color='red');bigred = smallred(size='+1') Works exactly the way you expect it to. If it doesn't work the way you expect it to, then your expectations are confused. 3. If you are adding content that sites within the tag, the content is replaced, not updated, like attributes.

This simple version handles auto-indentation of content as necessary (or desireable), auto-escaping of text elements, and includes an (I believe) nearly complete listing of entities which don't require closing tags.

I don't know where this is going, whether it can or will expand into something more, or what, but I believe what I have managed to hack together is better than other similar packages available elsewhere (including this recipe over here http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/366000 , which I discovered after writing my own). Funny how these things work out. Astute observers will note that I borrow nevow.stan's meme of using T.tagname for generating tag objects.

HTMLTags - generate HTML in Python (Python)

2009-10-24T10:30:38-07:00

Python recipe 366000 by Pierre Quentel (web). Revision 11.

The HTMLTags module defines a class for each valid HTML tag, written in uppercase letters. To create a piece of HTML, the general syntax is :

t = TAG(innerHTML, key1=val1,key2=val2,...)

so that "print t" results in :

<TAG key1="val1" key2="val2" ...>innerHTML</TAG>

For instance :

print A('bar', href="foo") ==> <A href="foo">bar</A>