February

Real results from the Materialized View API

I’ve faced a lot of skepticism (rightfully so) over my Materialized View module, which I’m pushing for inclusion in Drupal 7 as a solution to the overhead of table-per-field storage in Field API, as well as many other scalability issues.

I could have responded with contrived benchmarks, but I wanted real results. With Drupal 7 far from release and large projects on Drupal 7 even farther from release, I decided to rewrite MV for Drupal 6 and install it on Drupal.org to replace some of the worst queries.

Schema changes should be lazy

We’re in the middle of upgrading Drupal.org, and many of the longest-running upgrades involve schema changes. Unfortunately, MySQL with InnoDB has a very unfriendly method of doing schema changes: it rebuilds the table and blocks all writes until the new table is ready. A more sensible approach would be schema versioning that allows different parts of a table to have different schema versions. This would minimize blocking, allowing schema changes to happen without downtime.

David Strauss elected as a Permanent Member of the Drupal Association

Todd Ross Nienkerk

Congratulations to David Strauss, Four Kitchens co-founder and Drupal scalability guru, who was elected yesterday as a Permanent Member of the Drupal Association.

David’s goals as a member focus largely on improvements to infrastructure, community-building, and reaching out to other open-source projects. Details can be found in his his application:

What are the primary goals you would like to work on?

I would like to advance the infrastructure for development and sprints by working with the community to drive development and deployment of next-generation (read: not CVS) tools, both for issue tracking and version control. I would like to participate in discussions surrounding the membership software for Drupal Association membership, including CiviCRM (the current tool) and alternatives. I would like to work with major free culture and free software organizations to establish partnerships.

Developer preview of Materialized Views

I’ve posted a developer preview of Materialized Views for Drupal 6. I’d like ambitious Drupal developers to try it out so I can get feedback on the developer experience.

No-brainer shared branch storage on your workstation

For me, my projects are code-based and, hence, mostly a collection of version-controlled branches. So, my “Projects” directory (located at ~/Projects) contains a smattering of branches organized into multi-level directory structures.

In my last post, I noted that shared branch storage is the key to fast branching, good offline access, and efficient disk usage. I even covered a quick way to get it rolling for Drupal branches. But it’s one thing to know how and another to know “best practices,” so I’ll share how I do things.

A Bazaar branch of Drupal HEAD with all history

I originally created a Bazaar branch of Drupal HEAD with hourly snapshots of upstream updates (commits to CVS HEAD) to streamline work on patches to Drupal HEAD.

This snapshot method had a few advantages over using Launchpad’s Drupal branch:

  • Because of the shallow history, branching from the Four Kitchens server was relatively fast compared to branching from Launchpad.
  • The Four Kitchens server has more reserve bandwidth and capacity, making initial branching and updates faster.
  • Launchpad goes down more often than the Four Kitchens server.

The snapshot-based branch also had a big disadvantage: it didn’t keep a true commit-by-commit history that mapped to CVS HEAD commits, making it hard to understand ongoing changes. After some discussions with chx, I’ve decided to combine the best of both worlds: a mirror of Launchpad’s Drupal HEAD branch on the Four Kitchens server with instructions (provided below) to avoid the trouble of pulling thousands of revisions every time you branch.

Case study preview: Four Kitchens' new business cards

Todd Ross Nienkerk

Assignment: New business cards for our staff.

Goals: A clean, professional, and friendly design with room to write on. (People like to make notes on business cards — namely, when and where they met that person.)

Enforcing branch commit atomicity (or, why the git staging area is bad)

With CVS, one of the only repository-wide atomic operations is tagging a local checkout. And not all that long ago, Subversion introduced mainstream users of free, open-source version control systems to full-scale atomicity. Or, at least the ability to be atomic.