Popular recipes by Albert-Jan Roskam

git pre-commit hook to reject large files using Python (Python)

2015-03-10T09:36:28-07:00

Python recipe 578883 by Albert-Jan Roskam (git, hook, precommit, python). Revision 9.

This script should be saved in the templatedir, so it ends up in .git/hooks whenever you do a new git init. By default, commits that contain files larger than 5 Mb are blocked. This is useful for preventing accidental large commits that are not caught by .gitignore. You can easily bypass the hook by specifing "--no-verify" with git commit. (in a previous version of this script, this did not work correctly)

sav2mdb (Python)

2014-02-17T20:53:56-08:00

Python recipe 578826 by Albert-Jan Roskam (access, spss). Revision 2.

sav2mdb.py: convert SPSS system files (codepage) to Microsoft Access files

Python reader + writer for spss sav files (Linux, Mac & Windows) (Python)

2013-02-20T22:07:27-08:00

Python recipe 577811 by Albert-Jan Roskam (reader, sav, spss, writer). Revision 12.

Python Program to READ & WRITE Spss system files (.sav) (Linux,Mac or Windows)

Check https://pypi.python.org/pypi/savReaderWriter/ for the latest version (including the libraries!)
Requires libspssdio.so.1 (LINUX) or libspssdio.dylib (MAC) or spssio32.dll (WINDOWS) plus associated libaries, which can be freely downloaded from: https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/We70df3195ec8_4f95_9773_42e448fa9029/page/Downloads%20for%20IBM%C2%AE%20SPSS%C2%AE%20Statistics?lang=en
It is recommended to download the v21 I/O files (required for writing zlib (.zsav) compressed files)
December 2012 (complete rewrite):
Added support for slicing, indexing, array slicing + other special methods
Added support for writing spss date fields
Added support for almost all meta data (missing values, sets, roles, etc.)
Added support for 64 bit Windows (tested with Win7) and other OSs (z/Linux, Solaris, HP Linux, IBM AIX (untested though)
Added support for reading and writing zlib compressed (.zsav) files
Removed pesky segfault error when freeing memory
Removed errors related to encoding
Changed some Reader defaults (verbose=False, returnHeader=False)
Renamed SavDataDictionaryReader into SavHeaderReader

LINUX:

Installation (tested on Linux Ubuntu 10):

additional packages/files needed are: intel-icc8-libs_8.0-1_i386.deb,libicu32_3.2-3_i386.deb, libstdc++5_3.3.6-20_i386.deb, libirc.so.
Run the following commands in your terminal: sudo apt-get install intel-icc8-libs; sudo apt-get install libicu32; sudo apt-get install libstdc++5.
Then convert libirc.a (static) to libirc.so (dynamic), save in same location as libspssdio.so.1: ar vx intel-icc8-libs_8.0-1_i386.deb; tar -xzvf data.tar.gz ./usr/lib/libirc.a; ar -x libirc.a.

Calling the program:

In the TERMINAL type: export LD_LIBRARY_PATH=/path/of/additional/sofiles; python /some/path/wrapperToProgram.py. You may also add ld_library_path to .bashrc
The wrapper starts with "from SavReaderWriter import *", followed by e.g. stuff from the if __name__ == '__main__' section

MAC OS:

you must put all the dylib files that come with the IBM SPSS_Statistics_InputOutput_Modules_* package in the macos directory somewhere that OS X can find them
one simple way to accomplish this is to copy them to /usr/lib

WINDOWS:

You can also find this dll in the installation directory of SPSS (although SPSS is _not_ needed!)
The .dll should be saved in the same location as this program.

USAGE: See docstrings + __main__ section

Python reader for spss sav files (Python)

2011-07-25T18:59:49-07:00

Python recipe 577650 by Albert-Jan Roskam (files, pasw, sav, spss, statistics). Revision 3.

Python Program to read Spss system files (.sav)

* version 2 (FASTER!) *
Requires spssio32.dll, which can be freely downloaded from: https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/We70df3195ec8_4f95_9773_42e448fa9029/page/Downloads%20for%20IBM%C2%AE%20SPSS%C2%AE%20Statistics?lang=en
You can also find this dll in the installation directory of SPSS (although SPSS is _not_ needed!)
The .dll should be saved in the same location as this program.
Parameters: savFileName: the file name of the spss data file; returnHeader: Boolean that indicates whether the first record should be a list of variable names (default: True); recodeSysmisTo: indicates to which value missing values should be recoded (default: ""); selectVars: indicates which variables in the file should be selected.The variables should be specified as a list or a tuple of valid variable names. If None is specified, all variables in the file are used (default: None); verbose: Boolean that indicates whether information about the spss data file (e.g., number of cases, variable names, file size) should be printed on the screen (default: True).
rawMode: Boolean that indicates whether values should get SPSS-style formatting,and whether date variables (if present) should be converted to ISO-dates. If True, the program does not format any values, which increases processing speed. (default: = False) interfaceEncoding Indicates the mode in which text communicated to or from the I/O Module will be. Valid values are 'UTF-8' or 'CODEPAGE' (default = 'CODEPAGE')
Typical use: savFileName = "d:/someFile.sav" with SavReader(savFileName) as sav: header = sav.next() for line in sav: process(line)
Note: --New version: If you downloaded this previously, use the current version as it is MUCH faster!! --this code currently only works on Windows (32 bits). I might make it work on Linux Ubuntu 10 at some point. --date fields in spss are represented as the number of seconds since the Gregorian calendar. The program converts these, wherever possible, to ISO-dates (yyyy-mm-dd).

Any feedback is welcome! I'm still learning!