Popular recipes by ccpizza

Recursive find replace in files using regex (Python)

2016-04-28T13:24:15-07:00

Python recipe 580653 by ccpizza (files, regex, replace, search). Revision 5.

Recursively find and replace text in files under a specific folder with preview of changed data in dry-run mode

Example Usage

See what is going to change (dry run):

find_replace.py --dir project/myfolder --search-regex "\d{4}-\d{2}-\d{2}" --replace-regex "2012-12-12" --dryrun

Do actual replacement:

find_replace.py --dir project/myfolder --search-regex "\d{4}-\d{2}-\d{2}" --replace-regex "2012-12-12"

Do actual replacement and create backup files:

find_replace.py --dir project/myfolder --search-regex "\d{4}-\d{2}-\d{2}" --replace-regex "2012-12-12" --create-backup

Same action as previous command with short-hand syntax:

find_replace.py -d project/myfolder -s "\d{4}-\d{2}-\d{2}" -r "2012-12-12" -b

Output of find_replace.py -h:

DESCRIPTION:
    Find and replace recursively from the given folder using regular expressions

optional arguments:
  -h, --help            show this help message and exit
  --dir DIR, -d DIR     folder to search in; by default current folder
  --search-regex SEARCH_REGEX, -s SEARCH_REGEX
                        search regex
  --replace-regex REPLACE_REGEX, -r REPLACE_REGEX
                        replacement regex
  --glob GLOB, -g GLOB  glob pattern, i.e. *.html
  --dryrun, -dr         don't replace anything just show what is going to be
                        done
  --create-backup, -b   Create backup files

USAGE:
   find_replace.py -d [my_folder] -s <search_regex> -r <replace_regex> -g [glob_pattern]

Enable Clear Type font smoothing on Windows (ctypes version) (Python)

2013-05-07T09:17:56-07:00

Python recipe 578500 by ccpizza (fontsmoothing, windows). Revision 3.

Running the script without parameters will enable Clear Type font smoothing. Pass 0, false, off, or disable to turn off Clear Type.

This version requires the ctypes module.

For a version that uses pywin32 see http://code.activestate.com/recipes/578499-enable-clear-type-font-smoothing-on-windows-pywin3/

Enable Clear Type font smoothing on Windows (pywin32 version) (Python)

2013-03-22T11:14:44-07:00

Python recipe 578499 by ccpizza (fontsmoothing, pywin32, windows_api). Revision 2.

Running the script without parameters will enable Clear Type font smoothing. Pass 0, false, off, or disable to turn off Clear Type.

This version requires the pywin32 module from http://sourceforge.net/projects/pywin32/files/pywin32/Build%20218/

For a ctypes-based version see: http://code.activestate.com/recipes/578500-enable-clear-type-font-smoothing-on-windows-ctypes/

Recursive find and replace in all files in directory with regex (Python)

2013-03-21T12:33:44-07:00

Python recipe 578312 by ccpizza (directory, recursion, regular_expressions, replace).

Search and replace files recursively in a given directory. Accepts regular expressions as search and replacement patterns.

Example usage:

python find_replace_regex.py ".", "<span[^>]*>", "<div>", "*.html" backup

Note: On win32 the < and > symbols are interpreted by the shell as input/output redirection even when they are surrounded with single or double quotes, and therefore need to be escaped with the caret character ^.

Android Gmail: export messages from SQLite database blobs (Python)

2013-03-22T11:45:11-07:00

Python recipe 578106 by ccpizza (android, sqlite). Revision 5.

This script will extract email message bodies from the SQLite database stored in an android phone.

The gmail database is typically located on your phone under the following location:

`\data\data\com.google.android.gm\databases\mailstore.YOURUSERNAME@gmail.com.db`

To use the script, copy the file above from your phone to your machine and rename it to gmail.db.

NOTE: You need a rooted phone in order to get access to the folder above.

Dropbox file uploader via web interface using Python with urllib2 and mechanize (Python)

2016-04-28T13:21:43-07:00

Python recipe 578030 by ccpizza (dropbox, mechanize). Revision 7.

UPDATE: This is script is not maintained and does not anymore with the current version of Dropbox. For a proper command line interface to dropbox I recommend dropbox_uploader: https://github.com/andreafabrizi/Dropbox-Uploader

Originally inspired by the example at http://joncraton.org/blog/62/uploading-dropbox-python.

The script uses mechanize to logon to the web page and upload the file(s) to the Dropbox root folder or to the folder supplied on the command line as dir:/my_dropbox_path (if present, this must be the first parameter).

Multiple files and/or glob patterns names are accepted as script arguments.

Example usage

dropbox.py file1.txt                       # upload to root folder
dropbox.py dir:/Backups/2012 file1.txt     # upload to custom folder
dropbox.py dir:/Backups/2012 *.txt         # upload by file mask
dropbox.py dir:/Backups/2020 *             # upload all files in current dir

Limitations: only files in current folder are processed, subfolders are ignored.

NOTE

The script requires the mechanize module - use pip install mechanize or easy_install mechanize to add it to your site-packages.

NOTE2

I have found a cleaner way to manage dropbox files from the console - see the dropbox-uploader script at https://github.com/andreafabrizi/Dropbox-Uploader - it is a Bash script that works using the official Dropbox API rather than emulating a web browser.

Base64 encode-decode Five Liner (Python)

2013-04-26T09:37:21-07:00

Python recipe 577626 by ccpizza (base64). Revision 2.

Encodes or decodes base64-encoded files. If the input or output file(s) are ommited standard input/output will be used.

The encoding direction is detected from the file name: for decoding save the script e.g. as encode64.py. For decoding create a symlink with decode in the name.

Minimalistic PostgreSQL console shell (Python)

2011-11-30T13:43:28-08:00

Python recipe 577468 by ccpizza (postgres, postgresql, sql). Revision 17.

Primitive shell for running PostgreSQL quieries from console. Make sure you set the right connection settings in the script.

All queries are executed in autocommit mode.

Example command line usage:

pg.py select * from mytable where oidh=1

The script recognizes a number of magic keywords:

pg.py show databases
pg.py show schemas
pg.py describe <table_name>

Embed lyrics into MP3 files using mutagen (USLT tag), optionally set other ID3 tags (Python)

2011-05-17T15:56:56-07:00

Python recipe 577138 by ccpizza (id3, lyrics, mp3, uslt). Revision 5.

Quick and dirty script to embed unsynchronized lyrics or any other text into MP3 files. The text files with the lyrics are expected to be in the same folder: i.e. for MySong.mp3 the lyrics text should be in the file MySong.txt.

The encoding of the text file will be probed in the following order: 'utf8','iso-8859-1','iso-8859-15','cp1252','cp1251','latin1'. If you need support for more encodings, a list is available at http://docs.python.org/release/2.5.2/lib/standard-encodings.html

To see the lyrics on an iPod (tested on 6G Classic) you need to press the middle button four times while a song is playing.

The script can also be used to set other ID3 tags. By default SET_OTHER_ID3_TAGS is False so existing ID3 tags will NOT be overwritten.

Usage: Running the file without arguments will process all MP3 files in the current directory.

Alternatively the path to the folder with MP3's can be passed as the first argument.

Copy files over SSH using paramiko (Python)

2015-12-06T14:19:15-08:00

Python recipe 576810 by ccpizza (paramiko, scp, ssh). Revision 20.

This script copies files in unattended mode over SSH using a glob pattern. It uses the paramiko module behind the scenes. It operates as an actual SSH client, and does not rely on any command line utilities, such as scp.

It first tries to connect using a key from a private key file or from an SSH agent. If RSA authentication fails, it will try to authenticate with a password if passwords are allowed on the SSH server. Assumes the rsa_private_key was generated with an empty passphrase.

On most linux/unix-like systems paramiko can be installed with sudo easy_install paramiko or pip install paramiko or using the built-in system package manager.

Remove ID3 tags from MP3 files (Python)

2010-03-29T10:55:27-07:00

Python recipe 577139 by ccpizza (id3, mp3, mutagen). Revision 3.

Remove ID3 tags from all files in the current directory

Rename non-ASCII filenames to readable ASCII, i.e. replace accented characters, etc (Python)

2011-11-03T17:50:55-07:00

Python recipe 577226 by ccpizza (conversion, renaming, unicode). Revision 3.

The script converts any accented characters in filenames to their ASCII equivalents. e.g.:

Example:

â > a
ä > a
à > a
á > a
é > e
í > i
ó > o
ú > u
ñ > n
ü > u
...

Before-and-after example:

01_Antonín_Dvořák_Allegro.mp3   >>>  01_Antonin_Dvorak_Allegro.mp3

Usage:

Running the script without arguments will rename all files in the current folder. 
!!!WARNING!!! ***No*** backups are created.

Download chromium browser nightly builds for any OS (with proxy support) (Python)

2014-07-05T18:47:47-07:00

Python recipe 577162 by ccpizza (chrome, chromium, download, unzip). Revision 19.

Downloads the latest Chromium browser build from http://commondatastorage.googleapis.com/chromium-browser-continuous/ using urllib2 or wget (with Python versions below 2.5) and unzips the downloaded zip file to a predefined folder.

To use a custom proxy define the HTTP_PROXY system variable.

The script will figure out the OS but you can also pass the platform as the first parameter (one of win32, linux, linux64, mac).

Prerequisites (!! only for Python versions below 2.5):

wget - usually already installed on most linux systems. Windows users can get it here.
unzip - used for unpacking the archive; usually already installed on most linux systems. Windows users can get it here.

Both wget and unzip should be available in PATH (for Python 2.5+ native Python modules are used).

For most Linux distros this script does not make much sense since the built-in package managers do a better job of managing chromium builds and dependencies, but it still can be useful if you are using a stable Chromium build but would like to be able to test the nightly builds too.

For OSX an additional installation step will be performed using the install.sh that is included in the OSX build. The OSX installer will copy the package to ~/Applications/Chromium, and set some permissions that are required for Chromium to run. Running the unpacked zip without doing the installation will most likely will not work because of missing executable permissions on some files.

Convert PDF to plain text (Python)

2010-11-25T15:30:52-08:00

Python recipe 577095 by ccpizza (converter, pdf).

This is a very raw PDF converter which has absolutely no idea of the page layout or text positioning.

To install the required module try easy_install pypdf in a console.

Convert Trac wiki markup to HTML (Python)

2010-03-09T08:02:36-08:00

Python recipe 577094 by ccpizza (converter, trac).

Convert trac wiki markup to HTML Usage: %s <inputfile.wiki> [outputfile.html] If outputfile.html is missing output is redirected to stdout in utf8 encoding.

Crop PDF File with pyPdf (Python)

2011-11-03T17:42:10-07:00

Python recipe 576837 by ccpizza (pdf, pypdf). Revision 3.

This recipe was originally posted by sjvr767 on http://www.mobileread.com/forums/showthread.php?t=25565 and I decided to also make it available here.

It uses pypdf (http://pybrary.net/pyPdf/)

The script is supposed to be run like this:

pdf_crop.py" -m "120 50 120 180" -i mypdf.pdf

where the margins are left top right bottom

To install pyPdf try easy_install pypdf.

Rename MP3 files from ID3 tags (does not require external ID3 modules) (Python)

2011-11-05T15:27:32-07:00

Python recipe 576811 by ccpizza (id3, mp3, rename). Revision 5.

Rename MP3 files in the current folder according to ID3 tags. This is based on Ned Batchelder's id3reader class. I only added the code in the __main__ method.

When run without arguments, the script renames the files in the current folder. The directory can be specified explicitly as the first argument.
The files do not need to necessarily have the MP3 extension.
To move the files to directories based on album name use the -d switch.
Does not work with other file types, such as OGG, M4A, MP4, etc.
This is a quick and dirty script that works for me most of the times. If you need more power, try mutagen, picard, mp3tag or something similar.

Generate HTML syntax-highlighted listings for any file using pygments (Python)

2011-02-01T14:21:59-08:00

Python recipe 576812 by ccpizza (pretty_printer, pygments, syntax_highlighter). Revision 6.

Generates HTML highlighted code listings for source code files in any language known to pygments. For a list of supported formats see http://pygments.org/languages

Make sure you have pygments is installed. Try easy_install pygments.

Example usage:

output to stdout:

python highlight.py my_source_file.java

output to file:

python highlight.py my_source_file.java > my_source_file.html