2009

December 8, 2009

Anticipage: scalable pagination, especially for ACLs

Pagination is one of the hardest problems for web applications supporting access-control lists (ACLs). Drupal and Pressflow support ACLs through the node access system.

Problems with traditional pagination

Because pagination uses row offsets into the results, browsing listings where newly published items get added to the beginning of the results creates “page drift.” Page drift is where a user already browsing through paginated results sees, for example, items E, D, and C on page one, waits awhile, clicks to the next page, and sees items C, B, and A. Going back to page one again shows F (newly published), E, and D. Item C “drifted” to page two while the user was reading page one. If new items are published frequently enough, pagination can become unusable due to this drifting effect.
Even if content and ordering are fully indexed, jumping n rows into the results remains inefficient; it scales linearly with depth into pagination.
Paginating sets where the content and ordering are not fully indexed is even worse, often to the point of being unusable.
The design is optimized around visiting arbitrary page offsets, which does not reflect user needs. Users only need to make relative jumps in pagination of up to 10 pages (or so) in either direction or to start from the end of the results. (If users are navigating results by hopping to arbitrary pages to drill down to what they need, there are other flaws in the system.)

November 30, 2009

Intelligent memcached and APC interaction across a cluster

Anyone experienced with high-performance, scalable PHP development is familiar with APC and memcached. But used alone, they each have serious limitations:

APC

Advantages
- Low latency
- No need to serialize/unserialize items
- Scales perfectly with more web servers
Disadvantages
- No enforced consistency across multiple web servers
- Cache is not shared; each web server must generate each item

memcached

Advantages
- Consistent across multiple web servers
- Cache is shared across all web servers; items only need to be generated once
Disadvantages
- High latency
- Requires serializing/unserializing items
- Easily shards data across multiple web servers, but is still a big, shared cache

Combining the two

Traditionally, application developers simply think about consistency needs. If consistency is unnecessary (or the scope of the application is one web server), APC is great. Otherwise, memcached is the choice. There is, however, a third, hybrid option: use memcached as a coordination system for invalidation with APC as the main item cache. This functions as a loose L1/L2 cache structure. To borrow terminology from multimaster replication systems, memcached stores “tombstone” records.

October 18, 2009

Update from the Drupal 7 Contributed Modules Sprint

The Vancouver Planetarium. Photo by Qole Pejorian.

chx and I gathered last week in Vancouver’s West End for a two-person performance sprint during the final code slush days, allowing us to finish several key improvements to Drupal’s database layer. Right afterward, many more people joined us for another sprint to port key modules to Drupal 7. People worked in-person, voluntarily over IRC, and involuntarily over IRC (lost passport).

I can say — without reservation — that our work was successful. We kicked off the weekend with Drupal 6 versions of Coder and Views. (Though there had been a touch of prior work on the Views port to Drupal 7’s new database layer.)

September 30, 2009

Why Drupal.org lacks good themes (and why CVS has nothing to do with it)

What CVS does to (some) designers

There’s been a lot of talk lately about how Drupal designers shouldn’t have to learn CVS. Nothing new to see here, really — just the same tired, self-fulfilling arguments about how much CVS sucks, how developers also hate using it, and how designers shouldn’t be expected to learn something so… technical.

September 9, 2009

Drupal does Dallas: DrupalCamp Dallas gets it right

In early August, David, Todd, and I took a road trip to Dallas, TX to attend the first ever DrupalCamp Dallas. It was a two-day event organized by several Dallas-based Drupal shops: LevelTen Interactive, Koine Media, and Tarakan Design. We had a lot of fun meeting fellow Texas Drupalers and saw some excellent presentations as well.

August 22, 2009

Benchmarks, hot off the Pressflow

Independent benchmarks by Josh Koenig from Chapter Three show a 3000x throughput increase and a 40x load reduction moving from plain Drupal to Pressflow + Varnish. His testing was performed on a small Amazon EC2 instance, demonstrating how Pressflow can deliver internet-scale performance on modest, inexpensive hardware. Pressflow is able to deliver this class of performance because it’s optimized to support Varnish and other enterprise-grade web infrastructure tools in ways that standard Drupal cannot.

With Pressflow’s API compatibility with Drupal, Josh’s move from Drupal to Pressflow on his project didn’t require any coding or extensive testing. He just replaced Drupal core with Pressflow. (It’s no harder than a minor Drupal update.)

For single-server setups in the Amazon EC2 cloud, Josh’s Project Mercury AMI provides a click-and-run, configured setup with the Pressflow + Varnish stack. For more complex setups, Four Kitchens provides infrastructure consulting services on the Pressflow system.

July 16, 2009

Need to scale Drupal on EC2? Check out Chapter Three's Mercury project

Josh Koenig from Chapter Three has made pre-release EC2 AMIs (pre-packaged virtual machine images) for Mercury, a project to combine Four Kitchens’ Drupal-derived, high-performance Pressflow with Varnish, Cache Router, and memcached. Initial results show it easily saturating the EC2’s pipe. Mercury instances directly update their Pressflow releases from the Four Kitchens Bazaar server.

Mercury is an exciting project for anyone who needs to run a high-traffic, Drupal-based site without having to configure a bunch of caching systems.

July 15, 2009

Giving schema back its good name

For modern applications, the word “schema” has become synonymous with the tables, columns, constraints, indexes, and foreign keys in a relational database management system. A typical relational schema affects physical concerns (like record layout on disk) and logical concerns (like the cascading deletion of records in related tables).

Schemas have gotten a bad name because current RDBMS tools give them these rotten attributes:

July 13, 2009

Pressflow 6 now offers direct downloads

For quite some time, Four Kitchens has provided Pressflow releases to its large-scale clients and anyone interested enough to request a copy. We provided limited access to copies so that we could understand what organizations expected of Pressflow, how they wanted to use it, and so that we could keep all users updated with the latest security, bug-fix, and feature releases.

June 24, 2009

Co-presenting "Accelerated grid theming using NineSixty" at DrupalCon Paris

Jake Strawn of Drupal Dynamics and I are teaming up to propose a Accelerated grid theming using NineSixty session for DrupalCon Paris. (To be fair, he beat me to it and was gracious enough to add me as a co-presenter.)

Our session will cover the following basics of the 960 grid system:

What is 960.gs?
Using the NineSixty theme as your starting point / parent theme
Understanding the grid-x, push-x, pull-x classes
Why a grid-based system can help speed up theme development
How to break the 960-pixel limit

In the last month, I have presented sessions on 960.gs and the NineSixty theme at DrupalCamps in Copenhagen, Helsinki, and Stockholm. Most recently, I co-presented a session at Drupal Design Camp Boston with Nathan Smith, creator of the 960 grid system. You can download the slide decks on our “Presentations” page.