Welcome, guest | Sign In | My Account | Store | Cart

write_path and load_path methods for file generation scripts (Python recipe) by Trent Mick
ActiveState Code (http://code.activestate.com/recipes/577172/)

I have a lot of scripts that end up writing files (often build system stuff). Everytime I either end up writing the obvious quick content = open(path).read() or I re-implement a function that handles things like: making a backup, some typical logging, encoding support, trying to make it no-op if no changes, etc.

In this recipe I'll try to add a number of these features so I don't have to keep re-writing this. :) So far this is just a start.

Current features:

  • rudimentary encoding support
  • logging on a given log argument
  • create_backup argument to create a backup file
  • writes to a temporary file and uses atomic os.rename to avoid destroying the existing file if writing fails
Python, 68 lines
Download
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
def _write_path(path, text, encoding, create_backup=False, log=None):
    """Write content to a path.
    
    @param path {str}
    @param text {unicode}
    @param encoding {str} The file encoding to use.
    @param create_backup {bool} Default False. Whether to create a backup
        file. The path of the backup will be `<path>.bak`. If that path
        exists it will be overwritten.
    @param log {logging.Logger} A logger to use for logging. No logging is
        done if it this is not given.
    """
    import os
    from os.path import exists, split, join
    import codecs
    
    # Write out new content to '.foo.tmp'.
    dir, base = split(path)
    tmp_path = join(dir, '.' + base + '.tmp')
    f = codecs.open(tmp_path, 'wb', encoding=encoding)
    try:
        f.write(text)
    finally:
        f.close()
    
    # Backup to 'foo.bak'.
    if create_backup:
        bak_path = path + ".bak"
        if exists(bak_path):
            os.rename(path, bak_path)
    elif exists(path):
        os.remove(path)
    
    # Move '.foo.tmp' to 'foo'.
    os.rename(tmp_path, path)
    if log:
        log.info("wrote `%s'", path)

def _load_path(path, encoding="utf-8", log=None):
    """Return the content of the given path.
    
    @param path {str}
    @param encoding {str} Default 'utf-8'.
    @param log {logging.Logger} A logger to use for logging. No logging is
        done if it this is not given.
    @returns {2-tuple} (<text>, <encoding>) where `text` is the
        unicode text content of the file and `encoding` is the encoding of
        the file. `text` is None if there was an error. Errors are logged
        via `log.error`.
    """
    import codecs
    try:
        f = codecs.open(path, 'rb', encoding)
    except EnvironmentError, ex:
        if log:
            log.error("could not open `%s': %s", path, ex)
        return None, None
    else:
        try:
            try:
                text = f.read()
            except UnicodeDecodeError, ex:
                if log:
                    log.error("could not read `%s': %s", path, ex)
                return None, None
        finally:
            f.close()
    return text, encoding
Tags: files

2 comments

Denis Barmenkov 14 years ago  # | flag

These functions becomes unstable in production: each of os.* functons shall be enclosed with try..except clause.

I use a set of functions like this:

def rename_careful(f1, f2):
    try:
        os.rename(f1, f2)
        return 1
    except:
        pass
    return 0

then test for file <f2> existing.

Trent Mick 13 years, 10 months ago  # | flag

Updated to work with Python 2.4 (can't use try/except/finally in 2.4).