Welcome, guest | Sign In | My Account | Store | Cart

This script reads the contents of a web file and copies them into a local file, named the same as the web file.

Python, 26 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#!/usr/bin/env python

"""File downloading from the web.
"""

def download(url):
	"""Copy the contents of a file from a given URL
	to a local file.
	"""
	import urllib
	webFile = urllib.urlopen(url)
	localFile = open(url.split('/')[-1], 'w')
	localFile.write(webFile.read())
	webFile.close()
	localFile.close()

if __name__ == '__main__':
	import sys
	if len(sys.argv) == 2:
		try:
			download(sys.argv[1])
		except IOError:
			print 'Filename not found.'
	else:
		import os
		print 'usage: %s http://server.com/path/to/filename' % os.path.basename(sys.argv[0])

A fast alternative to the Unix "wget" bash command. Windows platforms do not support any program of similar nature, so this one is a good solution.

7 comments

Ian Bicking 17 years, 8 months ago  # | flag

urllib. urllib.urlretrieve will do this. One problem with the code as given is that the entire file will be read into memory, and then written out to the file; briefly, but for a moment your program could get very memory hungry. urlretrieve will write the file out in chunks. shutil.copyfileobj will also copy between file objects in a chunked manner.

Aristotelis Mikropoulos (author) 17 years, 8 months ago  # | flag

Tip appreciated. Thank you a lot Ian. I will bare this in mind.

thanos vassilakis 16 years, 10 months ago  # | flag

What's its use ? Would it be better to take a optional local file name ? You current code might clobber an important local file.

It really depends on what the use context is.

Chad Cooper 16 years, 10 months ago  # | flag

wget is available for Windows. http://www.gnu.org/software/wget/

I use this extensively for downloading images, using something similar to this:

os.system('wget %s -a log.log' % fullurl)

Mark Tomlinson 16 years, 1 month ago  # | flag

Under windows images fail. You need to set the b flag in the write statement

localFile = open(url.split('/')[-1], 'wb')

Window isn't good at telling the difference we have to tell it.

Anthony Fischbach 13 years, 3 months ago  # | flag

Is there a handy way to list all files under an http url, then download them one-by-one using urlib?

Vijayakanth P 12 years, 11 months ago  # | flag

Probably you could have used the os module

like import os

localFile=os.path.split()[-1]

this is compatible with both windows and linux

bye!!!