Why?

Django is a great framework for developing websites but as with most projects there isn’t a particular focus on the system administration side of running a real site. There are great instructions describing the source-level changes you’ll want to make and what you’ll need to configure your webserver to do but … what about afterwards?

The process of deploying any site has a few basic steps: update the code, apply any database changes and reload the running site. How people choose to do this varies wildly but in the Python world it tends to involve a lot of manual work setting up the Python environment and running commands by hand or using a tool such as Fabric or Buildout to run those commands for you.

This approach works but it has a few drawbacks:

• If you’re installing anything which uses a native library (images, databases, clients for things like memcache, etc.) you’ll need install a ton of extra dependencies on your production servers: gcc, development headers, etc.
• It’s extremely slow compared to normal Linux software installation
• You’re exposed to failures in outside services such as PyPI, Sourceforge, Github, Bitbucket, etc. This can be a problem if you can’t download the package and disastrous if the upstream source has updated to a newer version which you haven’t tested
• Adding a second server requires you to duplicate everything, which is time consuming, and you then have to apply your changes in lockstep across every server to avoid requests being processed depending on which server handles the request
• It’s hard to tell in advance what you’ll need to install an application unless you tediously compare the installed dependencies on a known-working server
• You can have silent failures which only show up in production - the classic example being the Python Imaging Library, which can install “successfully” without optional components like JPEG support if it fails to detect them, which will only show up the first time someone attempts to use a JPEG file with your site.

None of these are new problems - in fact, the BSD and Linux communities have been working on package managers for the last decade or two, which is why you can install a brand new Linux system with hundreds of applications in less time than it takes to bring a large Python website up on a new server.

How

UPDATE: This has turned into a Github project and the latest version of these instructions are on Github pages.

Structuring the application

A well-behaved application is going to do a few things:

• Install in a well-defined location not used by other applications - i.e. /opt/my_app rather than /var/www/html.
• Provide a clean way to customize your app with server-specific settings (e.g. file locations, database info, etc.) which doesn’t involve editing the packaged files - otherwise they’ll be overwritten on the next release. This also makes it easy to cleanly install the same package on development, test, and production servers.

For Django apps, I’m using the following conventions:

• Everything installs in /opt/my_app - this includes a virtualenv which is pre-loaded with our dependencies, avoiding the possibility of conflicts between projects. Want to have a Django 1.1 app installed on the same server as an older Django 1.0 app? This makes that easy.
• Apache configuration is split into a separate common configuration (e.g. WSGI config, media expiration, etc) designed to be included by a server-specific file which specifies things like hostnames, SSL config, etc.:

  <VirtualHost example.org:80>
      ServerName example.org
  
      ErrorLog logs/my_app.errors.log
      CustomLog logs/my_app.access.log combined
      LogLevel info
  
      SSLEngine on
      SSLProtocol all -SSLv2
      SSLCipherSuite ALL:!ADH:!EXPORT56:+HIGH:+SSLv3:+TLSv1
      SSLCertificateFile "/etc/httpd/ssl.crt/my_app.crt"
      SSLCertificateKeyFile "/etc/httpd/ssl.key/my_app.key"
  
      Include /opt/my_app/deploy/apache_common.conf
  </VirtualHost>
      

• Django customization is managed by a local_settings.py file which is imported by settings.py.

  try:
      import local_settings
  except ImportError:
      logging.warning("No local settings - running in development mode")
      

  This is where you put things like database username and password, production email contact addresses, etc.

Building an RPM

1. Setup your RPM build environment
2. Create your specfile (see below) in SPECS/my_site.spec
3. Create source archives for everything you need to install: this be as simple as downloading a tarfile from the library provider or creating your own from your version control system:
   • Git archive:
     git archive --format=tar --prefix=my_app-1.0/ my_app-1.0 | gzip -9 > ~/rpmbuild/SOURCES/my_app-1.0.tar.gz
   • Subversion:
     svn export . /tmp/my_package
tar -C /tmp -cjf ~/rpmbuild/SOURCES/my_app.tar.bz2
4. Now you’re ready to compile the actual RPM:
   rpmbuild -ba --clean SPECS/my_site.spec
5. Install the RPM on your test server

If you want to see what files your RPM will install, use RPM’s query options: rpm -q --fileprovide -p RPMS/noarch/my_site.rpm

For future releases the process is simple: update the specfile if you’ve changed your dependencies (add, remove, change versions, etc.) and recompile.

Here’s an example project containing an RPM specfile and the general recommended site structure. There are a few key things you will want to customize:

• Set dependencies for any libraries which you need, particularly if there are version requirements - that way RPM won’t allow you to install the site if the installed Postgres client is older than you want.
• The %post and %install sections can contain arbitrary shell scripts, allowing you to do things like run `manage.py syncdb`, push updated schema to Solr, etc. If this gets too complicated I recommend writing a Python program or Django management command which does the actual work.

Either Google is failing me or the docs for configuring Solr and Jetty could be improved. If you need to lock down Solr on a deadline, the process looks like this - and the process is applicable to any other Jetty-based embedded webserver:

1. Change into your Solr directory
2. Generate a password hash for the user(s) you intend to use:

   chris@server:~/project/solr [git master] $ java -cp lib/jetty-6.1.3.jar:lib/jetty-util-6.1.3.jar org.mortbay.jetty.security.Password ACCOUNT SECRET PASSWORD
   OBF:… OBF HASH …
   MD5:… MD5 HASH …
   CRYPT:… CRYPT HASH …
   

3. Create etc/realm.properties with something like this using the OBF value from above:

   admin: OBF HASH, solr-admin
   

4. Add the following into etc/jetty.xml inside the UserRealms set (search for <Set name=”UserRealms”>, which probably has a commented-out example similar to the config below):

   <Array type="org.mortbay.jetty.security.UserRealm">
       <Item>
           <New class="org.mortbay.jetty.security.HashUserRealm">
               <Set name="name">Solr Administration</Set>
               <Set name="config"><SystemProperty name="jetty.home" default="."/>/etc/realm.properties</Set>
           </New>
       </Item>
   </Array>
   

5. In etc/webdefault.xml add this inside the <web-app> block:

     <security-constraint>
         <web-resource-collection>
             <web-resource-name>Solr Administration</web-resource-name>
             <url-pattern>/admin/*</url-pattern>
         </web-resource-collection>
         <auth-constraint>
             <role-name>solr-admin</role-name>
         </auth-constraint>
     </security-constraint>
   
     <login-config>
         <auth-method>BASIC</auth-method>
         <realm-name>Solr Administration</realm-name>
     </login-config>
   

Add this point you should be able to start Solr and verify that the admin interface requires a password. Depending on your site configuration you might want to have a separate password used to make any sort of queries, access /update, etc. This can be done by modifying the url-pattern above - note that the role-name can be used multiple times so you could easily create a policy like “solr-user may access /, solr-updater may access both / and /update and solr-admin may access everything”.

A very fast presentation about Tornado for ZPUGDC:

I recently needed to publish a Git repository to an Apache server for simple sharing with a collaborator. There are a few guides out there, but there’s a key missing step required with the current Git releases for the simple publish-over-SSH-clone-over-HTTP model.

On the server, run the following commands in the directory which you want to contain the repository after changing PROJECT to the actual name of the project:

mkdir PROJECT.git 
cd PROJECT.git
git --bare init 
git update-server-info -f
chmod +x hooks/post-update 

The -f options for update-server-info appears to now be necessary the first time you update the repository (subsequent pushes seem fine without it). None of the guides I found with a quick search used it so I assume it changed at some point fairly recently.

You can now publish to this repository over SSH:

git push server:path/to/PROJECT.git master

At this point your collaborators can clone the repository:

git clone http://server/path/to/PROJECT.git

Note: if you do this often, add it as a remote so you can simply push to a convenient name (replacing PROJECT_HTTP_NAME with whatever’s memorable to you):

git remote add PROJECT_HTTP_NAME host:path/to/PROJECT.git