Home

The Problem

You have dev/test/production style configuration for change management. Code lives in source control and migrates up smoothly, but database structural changes and data that is added in the development or staging process needs to make its way into production as well to support whatever new feature or changes you wish to deploy. The crux of the problem is that there is production data that has to be preserved, development data and configurations that have to be merged in in an seamless, error free manner.

• http://drupal.org/node/175721#comment-272784
• http://drupal.org/node/1223632

Current methodologies

1. Duplicate your installation and configuration efforts in development, and the re-do them on production.
   1. Install module A.
   2. enable module A.
   3. Configure Module A.
   4. If you are happy with the results, do steps 1-3 on test.
   5. If everybody else is happy with the results do steps 1-3 on production. Gets hard to track exactly what you need to do, especially if theres changes that should be implemented at the staging/test level. There are some tools that help with this process, like the devel macro tool, but if you have a lot of changes to make for the next requested featureset.. well good luck not working nights n weekends with this process.
2. Have intimate awareness of what each contributed module is doing behind the scenes, and leverage that with innumerable methods. Thinks like modules built in “export” feature (like views, or CCK content copy, taxonomy XML and the like), or plow into the guts of what ought to be black box plugins, and ensure that all the changes are made on disk, and therefore in source control. This adds major slowdown to the process in using contributed modules.
   • http://www.databasepublish.com/blog/using-drupal-module-install-scripts-simplify-site-deployments
   • http://sachachua.com/wp/2009/04/23/drupal-staging-and-deployment-tips-its-all-code/
   • http://groups.drupal.org/node/5792#comment-16920
3. Merge the data base changes after the fact: do all the development you need to for a release in dev on a copy of production database. Migrate the changes up using database scripts, and mySql diff sort of things. The main issue with this methodology is sequential key collisions. If there is data created in dev and prod on the same table that has a sequential key ( nodes?) and that data gets moved to production, the keys will collide. Methods around this involve altering keys for “odd is dev, even is production” sort of hacks, base ten hacks (keys ending in 1 are dev, 2 are staging 3 are production, and 4-0 are individual developer stuff.)
   • http://codebaboon.com/method-syncronising-database-changes
   • http://drupal.org/project/dbscripts

Proposed Solution

Capture the SQL queries as the happen. Much like ‘binary query logs’ in mysql, grab everything that actually changes the database, and then replay those changes to staging and production. The replaying merges the best of solution 2 and 3 without either of their disadvantages.

Other Discussions

• http://drupal.org/node/501724
• http://drupal.org/node/206647

Related projects

• http://drupal.org/project/backup_migrate
• http://drupal.org/project/deploy
• http://drupal.org/project/patterns

Competition/Comrades

• http://drupal.org/project/dbcm

Other possible benefits

ability to package complex module groups with default configurations. LIke “add a calendar/views/date” module to your site thats already got all the local sports teams events presubscribed, or a “news aggregator module” config script that packages “liberal politics blogs” into a group.