Grab a document from the web (Python recipe) by Gisle Aas

ActiveState Code (http://code.activestate.com/recipes/52199/)

Grab a document from the web. This is an amazing example of the power of python. These one-liners are great for beginners like me who want to tap into this power right up front!

How about a "Python Power" category for these simple but "not so obvious to the newbie" power tips.

Adding support for proxy. Nice, but any real-life usage require a proxy.

Proxy in python. Set an environment variable HTTP_PROXY to your proxyserver:port So it'll look something like this:

set HTTP_PROXY=http://proxy.domain.com:8080

You need to have the http:// in front... or else it won't work!

Cheers,

Kraulin

Support for proxy authentication? Is there any way to work with a proxy server that requires authentication?

Proxy auth urllib. There is an example here:

http://pydoc.org/2.3/urllib2.html



import urllib2

# set up authentication info
authinfo = urllib2.HTTPBasicAuthHandler()
authinfo.add_password('realm', 'host', 'username', 'password')

proxy_support = urllib2.ProxyHandler({"http" : "http://ahad-haam:3128"})

# build a new opener that adds authentication and caching FTP handlers
opener = urllib2.build_opener(proxy_support, authinfo, urllib2.CacheFTPHandler)

# install it
urllib2.install_opener(opener)

f = urllib2.urlopen('http://www.python.org/')

thats the thing about python, simple yet so powerful.