This script allows how to transparently install a HTTP proxy (proxy HTTP 1.1, using CONNECT command) on all outgoing sockets.
I did that to bring TCP over HTTP to FTPlib, transparently. It should enable HTTP tunneling for all methods / modules that use the low-level socket API.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | #!/usr/bin/python
import socket
# Class that wraps a real socket and changes it to a HTTP tunnel whenever a connection is asked via the "connect" method
class ProxySock :
def __init__(self, socket, proxy_host, proxy_port) :
# First, use the socket, without any change
self.socket = socket
# Create socket (use real one)
self.proxy_host = proxy_host
self.proxy_port = proxy_port
# Copy attributes
self.family = socket.family
self.type = socket.type
self.proto = socket.proto
def connect(self, address) :
# Store the real remote adress
(self.host, self.port) = address
# Try to connect to the proxy
for (family, socktype, proto, canonname, sockaddr) in socket.getaddrinfo(
self.proxy_host,
self.proxy_port,
0, 0, socket.SOL_TCP) :
try:
# Replace the socket by a connection to the proxy
self.socket = socket.socket_formal(family, socktype, proto)
self.socket.connect(sockaddr)
except socket.error, msg:
if self.socket:
self.socket.close()
self.socket = None
continue
break
if not self.socket :
raise socket.error, ms
# Ask him to create a tunnel connection to the target host/port
self.socket.send(
("CONNECT %s:%d HTTP/1.1\r\n" +
"Host: %s:%d\r\n\r\n") % (self.host, self.port, self.host, self.port));
# Get the response
resp = self.socket.recv(4096)
# Parse the response
parts = resp.split()
# Not 200 ?
if parts[1] != "200" :
raise Exception("Error response from Proxy server : %s" % resp)
# Wrap all methods of inner socket, without any change
def accept(self) :
return self.socket.accept()
def bind(self, *args) :
return self.socket.bind(*args)
def close(self) :
return self.socket.close()
def fileno(self) :
return self.socket.fileno()
def getsockname(self) :
return self.socket.getsockname()
def getsockopt(self, *args) :
return self.socket.getsockopt(*args)
def listen(self, *args) :
return self.socket.listen(*args)
def makefile(self, *args) :
return self.socket.makefile(*args)
def recv(self, *args) :
return self.socket.recv(*args)
def recvfrom(self, *args) :
return self.socket.recvfrom(*args)
def recvfrom_into(self, *args) :
return self.socket.recvfrom_into(*args)
def recv_into(self, *args) :
return self.socket.recv_into(buffer, *args)
def send(self, *args) :
return self.socket.send(*args)
def sendall(self, *args) :
return self.socket.sendall(*args)
def sendto(self, *args) :
return self.socket.sendto(*args)
def setblocking(self, *args) :
return self.socket.setblocking(*args)
def settimeout(self, *args) :
return self.socket.settimeout(*args)
def gettimeout(self) :
return self.socket.gettimeout()
def setsockopt(self, *args):
return self.socket.setsockopt(*args)
def shutdown(self, *args):
return self.socket.shutdown(*args)
# Return the (host, port) of the actual target, not the proxy gateway
def getpeername(self) :
return (self.host, self.port)
# Install a proxy, by changing the method socket.socket()
def setup_http_proxy(proxy_host, proxy_port) :
# New socket constructor that returns a ProxySock, wrapping a real socket
def socket_proxy(af, socktype, proto) :
# Create a socket, old school :
sock = socket.socket_formal(af, socktype, proto)
# Wrap it within a proxy socket
return ProxySock(
sock,
proxy_host,
proxy_port)
# Replace the "socket" method by our custom one
socket.socket_formal = socket.socket
socket.socket = socket_proxy
|
Usage
Just call the setup_http_proxy method
setup_http_proxy("my.proxy", 8080)
Example
Here is a sample script that does FTP over HTTP
# Imports
from ftplib import FTP
from StringIO import StringIO
# Constants
PROXY_HOST="my.proxy"
PROXY_PORT=8080
FTP_HOST="my.ftp.site"
FTP_USER="anonymous"
FTP_PASS="anonymous"
FTP_FILE="/path.to/file.txt"
FTP_DIR="/path/to/folder/"
# Install the proxy on the "socket" module
setup_http_proxy(
PROXY_HOST,
PROXY_PORT)
# Now, the libftp module can do FTP over HTTP !!, transaprently
ftp = FTP(
FTP_HOST,
FTP_USER,
FTP_PASS)
# List content of a folder
print "Content of folder '%s' : %s" % (FTP_DIR, ftp.nlst(FTP_DIR))
# Get one file into buffer
buffer= StringIO()
ftp.retrlines("retr %s" % FTP_FILE, lambda line : buffer.write(line + "\n"))
# Print it
print "Content of file '%s' :\n%s" % (
FTP_FILE,
buffer.getvalue()
Limitations, bugs
The next step should be to enable t black list of "no_proxy" hosts (like 'localhost' or '*.mydomain.com'). It should be fairly easy to do.
I'm behind a proxy server right now. Usually I connect with Filezilla over HTTP\1.1 using CONNECT method, but for some tasks I need more functionality. When trying to connect using your module I get this exception:
Is there a way to fix the problem? I'm using Python2.7
Ok I didn't copy the attribute from the wrapped socket. It worked fine for me.
Let's add some code in the ProxySock constructor : self.family = socket.family self.type = socket.type self.proto = socket.proto
I have updated the code. Please try again and tell if it works.
Now it works fine, thanks. Now I have all the instruments I need to work over an ftp
Thanks to you for your feedback. I'm glad to see it is useful to someone. That's the reason why I share code snipnets here.
You should use a new-style class (
class ProxySock(object)
) with__getattr__
instead of copying attributes and wrapping methods:Hello Raphael, My proxy needs an authentification with a username and a password. (different from the username and the password of the ftp I try to reach) How can I implement it with your solution ? Thanks
Hello Raphael, I'm using Python 3.5 and I get TypeError was unhandled by user code Message: makefile() got an unexpected keyword argument 'encoding'
during creation of FTP.
Can you please help me fix it?