Why?

Django is a great framework for developing websites but as with most projects there isn’t a particular focus on the system administration side of running a real site. There are great instructions describing the source-level changes you’ll want to make and what you’ll need to configure your webserver to do but … what about afterwards?

The process of deploying any site has a few basic steps: update the code, apply any database changes and reload the running site. How people choose to do this varies wildly but in the Python world it tends to involve a lot of manual work setting up the Python environment and running commands by hand or using a tool such as Fabric or Buildout to run those commands for you.

This approach works but it has a few drawbacks:

• If you’re installing anything which uses a native library (images, databases, clients for things like memcache, etc.) you’ll need install a ton of extra dependencies on your production servers: gcc, development headers, etc.
• It’s extremely slow compared to normal Linux software installation
• You’re exposed to failures in outside services such as PyPI, Sourceforge, Github, Bitbucket, etc. This can be a problem if you can’t download the package and disastrous if the upstream source has updated to a newer version which you haven’t tested
• Adding a second server requires you to duplicate everything, which is time consuming, and you then have to apply your changes in lockstep across every server to avoid requests being processed depending on which server handles the request
• It’s hard to tell in advance what you’ll need to install an application unless you tediously compare the installed dependencies on a known-working server
• You can have silent failures which only show up in production - the classic example being the Python Imaging Library, which can install “successfully” without optional components like JPEG support if it fails to detect them, which will only show up the first time someone attempts to use a JPEG file with your site.

None of these are new problems - in fact, the BSD and Linux communities have been working on package managers for the last decade or two, which is why you can install a brand new Linux system with hundreds of applications in less time than it takes to bring a large Python website up on a new server.

How

UPDATE: This has turned into a Github project and the latest version of these instructions are on Github pages.

Structuring the application

A well-behaved application is going to do a few things:

• Install in a well-defined location not used by other applications - i.e. /opt/my_app rather than /var/www/html.
• Provide a clean way to customize your app with server-specific settings (e.g. file locations, database info, etc.) which doesn’t involve editing the packaged files - otherwise they’ll be overwritten on the next release. This also makes it easy to cleanly install the same package on development, test, and production servers.

For Django apps, I’m using the following conventions:

• Everything installs in /opt/my_app - this includes a virtualenv which is pre-loaded with our dependencies, avoiding the possibility of conflicts between projects. Want to have a Django 1.1 app installed on the same server as an older Django 1.0 app? This makes that easy.
• Apache configuration is split into a separate common configuration (e.g. WSGI config, media expiration, etc) designed to be included by a server-specific file which specifies things like hostnames, SSL config, etc.:

  <VirtualHost example.org:80>
      ServerName example.org
  
      ErrorLog logs/my_app.errors.log
      CustomLog logs/my_app.access.log combined
      LogLevel info
  
      SSLEngine on
      SSLProtocol all -SSLv2
      SSLCipherSuite ALL:!ADH:!EXPORT56:+HIGH:+SSLv3:+TLSv1
      SSLCertificateFile "/etc/httpd/ssl.crt/my_app.crt"
      SSLCertificateKeyFile "/etc/httpd/ssl.key/my_app.key"
  
      Include /opt/my_app/deploy/apache_common.conf
  </VirtualHost>
      

• Django customization is managed by a local_settings.py file which is imported by settings.py.

  try:
      import local_settings
  except ImportError:
      logging.warning("No local settings - running in development mode")
      

  This is where you put things like database username and password, production email contact addresses, etc.

Building an RPM

1. Setup your RPM build environment
2. Create your specfile (see below) in SPECS/my_site.spec
3. Create source archives for everything you need to install: this be as simple as downloading a tarfile from the library provider or creating your own from your version control system:
   • Git archive:
     git archive --format=tar --prefix=my_app-1.0/ my_app-1.0 | gzip -9 > ~/rpmbuild/SOURCES/my_app-1.0.tar.gz
   • Subversion:
     svn export . /tmp/my_package
tar -C /tmp -cjf ~/rpmbuild/SOURCES/my_app.tar.bz2
4. Now you’re ready to compile the actual RPM:
   rpmbuild -ba --clean SPECS/my_site.spec
5. Install the RPM on your test server

If you want to see what files your RPM will install, use RPM’s query options: rpm -q --fileprovide -p RPMS/noarch/my_site.rpm

For future releases the process is simple: update the specfile if you’ve changed your dependencies (add, remove, change versions, etc.) and recompile.

Here’s an example project containing an RPM specfile and the general recommended site structure. There are a few key things you will want to customize:

• Set dependencies for any libraries which you need, particularly if there are version requirements - that way RPM won’t allow you to install the site if the installed Postgres client is older than you want.
• The %post and %install sections can contain arbitrary shell scripts, allowing you to do things like run `manage.py syncdb`, push updated schema to Solr, etc. If this gets too complicated I recommend writing a Python program or Django management command which does the actual work.