Drupal Migraine

Drupal test-to-production migration tool

Drupal has many good qualities, but one thing its web site platform lacks is a good way to develop freely on a test site while still allowing content updates to continue on a production site.

This small script allows you to maintain separate test and production versions of your Drupal site. After you’ve made configuration changes on the test site, this script can migrate them to the production site for you. Your production site stays up while you make changes to the test site, and content created on the production site in that period is preserved.

As a bonus, it will generate lean database backups stripped of cache and log data, all sorted in a standard order so that 'diff' shows the exact changes made between revisions. Revision control systems such as Subversion can store the backups efficiently.

Credits

Thanks to Noosphere Networks for releasing this script.

Limitations

This is alpha software. Only use it if you are comfortable with MySQL and Drupal’s table structure, and with shell access.

Drupal doesn’t have a strong concept of separation between site configuration and site content. This script attempts to provide one. However, there are some cases where it needs some help.

The main one is adding, removing, or editing CCK field definitions. CCK can alter the content schema, rendering the content on the production site incompatible with the configuration on the test site. The script specifically tests for this and displays an error, showing the diff between the test site’s content schema and the production site’s content schema. The workaround is to make the same CCK changes again on the production server before you migrate, which will clear the error and allow all the other changes to go through.

If you have custom content types or modules on your site that I don’t, you will be called on the first time through to manually categorize their tables as content or config.

This script is called from the command line and therefore requires shell access. It uses command-line arguments internally to pass your database password to MySQL’s command-line tools, which may lower security on shared hosting setups.

Note that you shouldn’t put the script in an unprotected area within your web root.

Sample Session

Back up test and prod databases, and commit backup to Subversion.

$ python migraine.py --config=migraine.ini
Enter test database server password: 
Enter prod database server password: 
Dumping databases (dump-test)...
Dumping to file 'test_dump/config.sql'...
Dumping to file 'test_dump/content_create.sql'...
Dumping to file 'test_dump/content_data.sql'...
Dumping to file 'test_dump/temp_create.sql'...
Dumping to file 'test_dump/cache_create.sql'...
Dumping to file 'test_dump/sequences.sql'...
Dumping databases (dump-prod)...
Dumping to file 'prod_dump/config.sql'...
Dumping to file 'prod_dump/content_create.sql'...
Dumping to file 'prod_dump/content_data.sql'...
Dumping to file 'prod_dump/temp_create.sql'...
Dumping to file 'prod_dump/cache_create.sql'...
Dumping to file 'prod_dump/sequences.sql'...

$ svn commit -m "Backing up databases prior to migration."
Committed revision 478.

Migrate backed-up test database to prod

$ python migraine.py --migrate-to-prod --config=migraine.ini
Enter prod database server password: 
Dumping to file 'test_dump/content_create_tempstate.sql'...
Loading disk file 'test_dump/config.sql' into database 'drupal_prod'.
Loading disk file 'test_dump/cache_create.sql' into database 'drupal_prod'.
Updating sequence 'view_view_vid' to 43 (was 41).
Updating sequence 'bc_blocks' to 66 (was 64).
Updating sequence 'menu_mid' to 211 (was 209).

Production Migration Procedure

This is the long version of the production procedure, including saving before- and after-migration database snapshots. (Depending on the types of changes, you may be able to skip some steps as appropriate.)

  1. Log into your test site with SSH.
  2. Make sure any source code changes or module upgrades are committed to Subversion.
  3. Take a snapshot of test DB, with python migraine.py --dump-test
  4. Commit the test DB snapshot to Subversion.
  5. If prod is on a different server than test, log in to prod and copy the test DB dumps there. A regular Subversion update command should do it.
  6. Under Admin / Site Maintenance on the prod site, take down your Drupal site for maintenance.

  7. Take a snapshot of prod DB, for backup purposes, and commit it to Subversion (you may be able to combine this with the test DB dump step if they’re on the same server). python migraine.py --dump-prod
  8. If source code has changed, update production PHP code with svn update.
  9. Run update.php on the production server, if modules may have database changes or upgrades to content tables. (That would be for new modules, or if module upgrades had made it necessary to run update.php the test server. You can enable any new modules in Admin / Site Building / Module Settings for best results.)
  10. Migrate test to prod: python migraine.py --migrate-to-prod. (If migration cannot complete due to CCK schema changes, repeat on prod the same CCK config changes that were made on test, and run the command again.)
  11. Flush CSS aggregation cache: rm ~/httpdocs/files/css/*
  12. Optionally, dump the prod database again and commit to Subversion, for post-migration backup.
  13. Bring the prod site online again under Site Maintenance.
Language Python.
Compatibility Tested with Drupal 5.1 and 5.2. Requires Python 2.4 or higher, or Python 2.3 plus the subprocess module. Python is pre-installed on most hosting plans. Requires shell access.
Download Download Migraine 1.0a1 (17K).
Or see the project’s Subversion repository.
License Freely available under the GNU General Public License (GPL).