The Economist

What they needed

Four Kitchens has worked with The Economist since 2008 to replace fragile, hard-to-maintain legacy systems with modern content-management and development workflows. The primary challenge has been balancing migration work with synchronous development of new, revenue-generating features, while also providing a constantly-improving, uninterrupted user experience. Throughout the process, the teams have architected workflows, infrastructure, testing strategies and coding standards that will expand and evolve with The Economist and Drupal.

How we helped

The Four Kitchens’ team has been integrated into the internal development team since our collaboration began. We’ve been involved in
migrating user data and comments, architecting and deploying infrastructure, implementing load-balancing and testing strategies, theme development from comps to launch, and custom-feature building like the homepage creation tools and related topics integration. Meanwhile, we are working across teams on solving the many migration challenges required to pull the plug on the legacy site.

Project outcome

Our work with The Economist continues and there have been important milestones along the way: the deployed infrastructure provides undisrupted access to all site features across both platforms; editors use custom tools for creating content; workflow systems for code deployment and testing have enabled quicker, more reliable innovation; major site improvements like related topic integration have improved user experience, revenue prospects, and internal confidence in Drupal as a solution.

With a small, expert team of Four Kitchens, The Economist, and external developers, we initially migrated user profiles and comments from Pluck to a Drupal-powered web service. We continuously migrated comments, users, and profiles from the legacy system and Pluck and worked with The Economist to have the ColdFusion pages call the web service to post and list comments, as well as allowing users to flag comments for review. Four Kitchens built out a small, well-tuned cluster of servers to run the production infrastructure. Transition to the Drupal-backed comment and profile system went smoothly, allowed The Economist to avoid the ongoing cost of using Pluck, and reduced the remaining work to serve these features – and eventually the complete site – from Drupal. Back-end administration tools ran directly on Drupal.

Slice by slice, Four Kitchens worked with The Economist to move functionality to Drupal and develop new features. The next step was delivering pages to end-users directly from Drupal without proxying through the legacy CMS. Four Kitchens developed a strategy to use single sign-on tools and to configure the primary load balancer to deliver traffic based on URL patterns. This approach allowed seamless switching of services from the legacy CMS to Drupal while mitigating risk by allowing rapid fail-back.

The first “slices” for end-users to run directly from Drupal were the commenting and recommendation features. Four Kitchens developed two Drupal themes: one that looked identical to the legacy one and one reflecting the comps for the redesign. The former allowed switching comment pages to Drupal without disruption; the latter continued preparations for delivering the full redesign. Following extensive testing and according to plan, we switched the load balancer configuration to deliver comment and profile pages from Drupal.

In a disruption to our migration plan, The Economist’s colocation provider informed them that we had a short, rigid time window to move all services to a new data center. Four Kitchens worked with The Economist to plan, deploy, and integrate infrastructure in the new facility and decommission now-obsolete systems in the old one. The migration plan involved a private network link between the two facilities and load-balancer configuration to allow migrating systems without disrupting availability. Four Kitchens provided on-site support during their final transition.

Development continued on additional services to run from Drupal, including articles and entirely new features, like topic- and geography-focused channels. Each launch repeated the strategy of configuring load-balancers to move relevant slices of traffic. Not all of these launches were initially successful, and our fail-back strategy proved successful in moving a service back to the legacy cluster after an attempted migration.

The largest problems typically involved performance or scalability. In response, Four Kitchens worked with The Economist to build a representative load-testing strategy to reproduce known problems, discover unknown problems, and validate improvements to the code and infrastructure. The framework developed for this load testing was presented at DrupalCons in Paris and San Francisco and is now informing the design of a comprehensive, general-purpose load-testing framework for Drupal-based projects.

Development continues on this project, but The Economist is now running most existing pages, all interactive functionality, and all new features on the Drupal-based system.