django-fabfile 2013.08.03.1

Fabric tasks for Django and Amazon Web Services

Required arrangements

export PYTHONPATH=<path-to-folder-with-django_fabfile-package>

Pre-run configuration

Django settings will be checked first if environment variable DJANGO_SETTINGS_MODULE configured properly. If option was not configured within Django settings, then its value will be taken from ./fabfile.cfg file - copy-paste there rows that should be overriden from defaults :download:`django_fabfile/fabfile.cfg.def <../django_fabfile/fabfile.cfg.def>`.

Django settings should be configured in following form (with preserving sections case):

FABFILE = {
    'DEFAULT': {
        'LOGGING_FOLDER': '.',
    },
    'Credentials': {
        'AWS_ACCESS_KEY_ID': 'AKIAABCDEFGHIJKLMNOP',
        'AWS_SECRET_ACCESS_KEY': 'ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLM',
    },
}

Example of required AWS IAM Policy

For using Fabric tasks you need AWS account with following permissions:

• create-snapshot:

  {
    "Statement": [
      {
        "Sid": "Stmt1304003463574",
        "Action": [
          "ec2:CreateSnapshot",
          "ec2:CreateTags",
          "ec2:DescribeInstances",
          "ec2:DescribeRegions",
          "ec2:DescribeSnapshots",
          "ec2:DescribeTags",
          "ec2:DescribeVolumes"
        ],
        "Effect": "Allow",
        "Resource": "*"
      }
    ]
  }
  

• delete-snapshot:

  {
    "Statement": [
      {
        "Sid": "Stmt1306410750989",
        "Action": [
          "ec2:DescribeRegions",
          "ec2:DescribeSnapshots",
          "ec2:DeleteSnapshot"
        ],
        "Effect": "Allow",
        "Resource": "*"
      }
    ]
  }
  

• reboot-instance:

  {
    "Statement": [
      {
        "Sid": "Stmt1312204628195",
        "Action": [
          "ec2:RebootInstances"
        ],
        "Effect": "Allow",
        "Resource": "*"
      },
      {
        "Sid": "Stmt1312276311113",
        "Action": [
          "ec2:DetachVolume"
        ],
        "Effect": "Allow",
        "Resource": "*"
      }
    ]
  }
  

• rsync-snapshot:

  {
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "EC2:AttachVolume",
          "EC2:CreateKeyPair",
          "EC2:CreateVolume",
          "EC2:DeleteKeyPair",
          "EC2:DeleteVolume",
          "EC2:DescribeAvailabilityZones",
          "EC2:DescribeImages",
          "EC2:DescribeKeyPairs",
          "EC2:RunInstances",
          "EC2:TerminateInstances"
        ],
        "Resource": "*"
      }
    ]
  }
  

Backup script

Following backup commands may be configured with crontab to run daily:

#!/bin/sh

VE=/home/backup-agent/.virtualenvs/backup

. $VE/bin/activate && \
fab --keepalive=60 -f $VE/lib/python2.6/site-packages/django_fabfile/backup.py \
        backup_instances_by_tag \
        trim_snapshots \
        rsync_region:src_region_name=us-east-1,dst_region_name=eu-west-1 \
        rsync_region:src_region_name=us-west-1,dst_region_name=eu-west-1 \
        rsync_region:src_region_name=eu-west-1,dst_region_name=us-east-1

With 30 production instances it tooks around 6 hours to replicate snapshots updates. First replication tooks much more time obviously.

Running tests

The package has test cases to make sure, that all methods, being tested, keep on the entire program logic. The test cases are using packages fudge and unittest. The test files ('test_*.py') are located in folder '/django_fabfile/tests/'. To run tests from command line, implement:

python -m unittest discover django_fabfile

Recent changes

See :doc:`changelog` for recent updates.

Licensing

The product is licensed by oDesk http://www.odesk.com/ under :download:`GNU GENERAL PUBLIC LICENSE Version 3<license.txt>` except portions with attached inline license information like :func:`django_fabfile.backup._trim_snapshots`.