ActiveState Code

Recipe 327141: Simple FTP directory synch


This script filled the need to have a scheduled directory synch occur via FTP. I also realized I could use it to clean out a directory without much effort. There are probably more robust examples out there, but this one should be easily modifiable for FTP newbies. It uses Sets to speed up finding missing files from the local directory.

Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
def moveFTPFiles(serverName,userName,passWord,remotePath,localPath,deleteRemoteFiles=False,onlyDiff=False):
	"""Connect to an FTP server and bring down files to a local directory"""
	import os
	from sets import Set
	from ftplib import FTP
	try:
		ftp = FTP(serverName)
	except:
		print "Couldn't find server"
	ftp.login(userName,passWord)
	ftp.cwd(remotePath)
	
	try:
		print "Connecting..."
		if onlyDiff:
			lFileSet = Set(os.listdir(localPath))
			rFileSet = Set(ftp.nlst())
			transferList = list(rFileSet - lFileSet)
			print "Missing: " + str(len(transferList))
		else:
			transferList = ftp.nlst()
		delMsg = ""	
		filesMoved = 0
		for fl in transferList:
			# create a full local filepath
			localFile = localPath + fl
			grabFile = True
			if grabFile:				
				#open a the local file
				fileObj = open(localFile, 'wb')
				# Download the file a chunk at a time using RETR
				ftp.retrbinary('RETR ' + fl, fileObj.write)
				# Close the file
				fileObj.close()
				filesMoved += 1
				
			# Delete the remote file if requested
			if deleteRemoteFiles:
				ftp.delete(fl)
				delMsg = " and Deleted"
			
		print "Files Moved" + delMsg + ": " + str(filesMoved) + " on " + timeStamp()
	except:
		print "Connection Error - " + timeStamp()
	ftp.close() # Close FTP connection
	ftp = None

def timeStamp():
    """returns a formatted current time/date"""
    import time
    return str(time.strftime("%a %d %b %Y %I:%M:%S %p"))

if __name__ == '__main__':
	#--- constant connection values
	ftpServerName = "ftpservername.com"
	ftpU = "ftpusername"
	ftpP = "ftppassword"
	remoteDirectoryPath = "remote/ftp/subdirectory"
	localDirectoryPath = """local\sub\directory"""
	
	print "\n-- Retreiving Files----\n"
	
	deleteAfterCopy = False 	#set to true if you want to clean out the remote directory
	onlyNewFiles = True			#set to true to grab & overwrite all files locally
	moveFTPFiles(ftpServerName,ftpU,ftpP,remoteDirectoryPath,localDirectoryPath,deleteAfterCopy,onlyNewFiles)

Discussion

This could stand to have a more robust synch method - just grabbing the files that don't exist locally doesn't help you with newer/changed files of the same name.

I'm also using a try/except block that covers a little too much code. More granular error handling would be nice, but might make it's readability drop.

Comments

  1. 1. At 2:10 p.m. on 20 jun 2005, Guy Argo said:

    huh? Where's the definition of ZzWw?

  2. 2. At 1:36 p.m. on 16 nov 2005, EyePulp (the author) said:

    Rats. Sorry about that - it appears my pop-up blocking/ad munching software killed that line when I submitted (and tried to re-edit) the code.

    The line has been changed to "open"

  3. 3. At 1:58 p.m. on 1 sep 2006, se5a se5a said:

    stumped. I can't get this working

    this is what I get:

    -- Retreiving Files----

    Couldn't find server Traceback (most recent call last):

    File "C:\Python24\projects\zipstuff\ftp.py", line 66, in ?

    moveFTPFiles(ftpServerName,ftpU,ftpP,remoteDirectoryPath,localDirectoryPath,
    

    deleteAfterCopy,onlyNewFiles)

    File "C:\Python24\projects\zipstuff\ftp.py", line 10, in moveFTPFiles

    ftp.login(userName,passWord)
    

    UnboundLocalError: local variable 'ftp' referenced before assignment

    The ftpservername is correct, as is the ftpU and ftpP.

    What could I be doing wrong?

  4. 4. At 1:01 p.m. on 6 sep 2006, se5a se5a said:

    got it. ah I see, you don't include the ftp:// in the address, just the address without the ftp:// duh

Sign in to comment