pymdeco 0.1.1 (experimental)

PyMDECO - Python Meta Data Extractor and Collection Organizer library

Example usage

Example with a video file:

>>> from pymdeco import services
>>> srv = services.FileMetadataService()
>>> meta = srv.get_metadata('/tests/big_buck_bunny_720p_surround.avi')
>>> print(meta.to_json(indent=2)) # to pretty print the metadata
{
  "file_name": "big_buck_bunny_720p_surround.avi",
  "file_type": "video",
  "file_size": 332243668,
  "mime_type": "video/x-msvideo",
  "file_hash": {
    "value": "b957d6e6212638441b52d3b620af157cc8d40c2a0342669294854a06edcd528c",
    "algorithm": "sha256"
  },
  "file_timestamps": {
    "modified": "2008-06-11 13:29:26",
    "created": "2008-06-11 13:29:26"
  },
  "video_metadata": {
    "streams": [
      {
        "sample_aspect_ratio": "1:1",
        "codec_type": "video",
        "codec_name": "mpeg4",
        "duration": "596.457",
        "nb_frames": "14315",
        "codec_time_base": "1/24",
        "index": 0,
        "width": 1280,
        "divx_packed": "0",
        "pix_fmt": "yuv420p",
# ... output truncated

Example with an audio file:

>>> meta = srv.get_metadata('/tests/jonobacon-freesoftwaresong2.mp3')
>>> print(meta.to_json(indent=2)) # to pretty print the metadata
{
  "file_name": "jonobacon-freesoftwaresong2.mp3",
  "file_type": "audio",
  "file_size": 3169160,
  "mime_type": "audio/mpeg",
  "file_hash": {
    "value": "d7ebc161d5d8fb802659fea949204af2958906b91913ca7577cfaeece90ffb78",
    "algorithm": "sha256"
  },
  "file_timestamps": {
    "modified": "2012-01-09 20:02:23",
    "created": "2012-01-09 20:02:23"
  },
  "audio_metadata": {
    "streams": [
      {
        "index": 0,
        "sample_fmt": "s16",
        "codec_tag": "0x0000",
        "bits_per_sample": 0,
        "r_frame_rate": "0/0",
        "start_time": "0.000",
        "time_base": "1/14112000",
        "codec_tag_string": "[0][0][0][0]",
        "codec_type": "audio",
        "channels": 2,
        "codec_long_name": "MP3 (MPEG audio layer 3)",
        "codec_name": "mp3",
... # output truncated

Example with an image file:

>>> meta = srv.get_metadata('/tests/some_image.jpg')
>>> print(meta.to_json(indent=2)) # to pretty print the metadata
{
  "file_name": "some_image.jpg",
  "file_type": "image",
  "file_size": 159894,
  "mime_type": "image/jpeg",
  "file_hash": {
    "value": "844a8750f2c9e1a24175c8f158abb6d204ec2b79fc49aba512cded3cdb3a0111",
    "algorithm": "sha256"
  },
  "file_timestamps": {
    "modified": "2012-01-09 20:43:12",
    "created": "2012-01-09 20:43:12"
  },
  "image_metadata": {
    "Exif": {
      "Photo": {
        "LightSource": 0,
        "PixelXDimension": 900,
        "SubSecTime": "16",
    # ... output truncated ... Image details follow
      "Image": {
        "YResolution": 72,
        "ResolutionUnit": 2,
        "Orientation": 1,
        "Copyright": "Some Rights Reserved                                  ",
        "Artist": "Yovko Lambrev                       ",
        "Make": "NIKON CORPORATION",
        "DateTime": "2008-09-13 11:26:41",
    # ... output truncated ... XMP details follow
    "Xmp": {
      "iptc": {
        "CreatorContactInfo/Iptc4xmpCore:CiEmailWork": "yovko@simplestudio.org",
        "CreatorContactInfo/Iptc4xmpCore:CiAdrCity": "Sofia",
        "CreatorContactInfo/Iptc4xmpCore:CiUrlWork": "http://simplestudio.org",
        "CreatorContactInfo/Iptc4xmpCore:CiAdrCtry": "Bulgaria"
    # ... output truncated ... IPTC details follow
    "Iptc": {
      "Application2": {
        "CountryName": "['Bulgaria']",
        "Byline": "['Yovko Lambrev']",
        "BylineTitle": "['Photographer']",
    # ... output truncated

Example with a binary file:

>>> meta = srv.get_metadata('/tests/minimal_ubuntu_11.04_natty.iso')
>>> print(meta.to_json(indent=2)) # to pretty print the metadata
{
  "file_name": "minimal_ubuntu_11.04_natty.iso",
  "file_type": "application",
  "file_size": 19922944,
  "mime_type": "application/x-iso9660-image",
  "file_hash": {
    "value": "8607e2c06090db13b06a216efbeb65d7aeff4ca8666904e6874cf4a5960f2366",
    "algorithm": "sha256"
  },
  "file_timestamps": {
    "modified": "2011-05-02 17:24:18",
    "created": "2011-05-02 17:24:18"
  }
}

Example with the command line tool:

python metadump.py -p /some/path

Dependencies

• Python 2.7+ or Python 3.2+
• pyexiv2 library (which is the Python bidnings to exiv2). It is used to extract EXIF, XMP and IPTC tags from photos and images (JPEG, PNG, GIF, etc).
• ffprobe binary (from ffmpeg suite version 0.9.0 or later) located in the system's PATH. Older versions of the ffprobe (prior 0.9.x) cannot output in JSON format.
• Sphinx (>= 1.0.0) - for building the documentation

History and future directions

The library was created out of necessity to manage large collection of photos (tens of thousands) and video clips (many thousands) collected over a large period of time from different sources and created by various models of photo cameras, camcorders and webcams.

I quickly found out that to identify duplicates and effectively manage the backups I needed good and reliable metadata handling and storage. After experimenting with half a dozen Python packages and tools I settled on building a common API which makes use of well-established third-party libraries and external tools.

So far only the metadata extraction part has been mostly completed, but development of some sort of persistent storage for that metadata is on the roadmap (involving the usage of MongoDB or Sqlite as potential backends). This persistent storage will be used to build collections with metadata and then constructing queries like "is this file already in the collection", "is there a similar file in the collection", "get all files with this metadata tag" or "list all the files created by the same device on the same date as this file".

See TODO.txt for more details.