JavaScript

Howdy perfers!

This week’s Webperf Wednesday is short and sweet, just like your page loads when you install this new module that enhances any Drupal theme. Magic is a set of frontend performance and development workflow tools for themers. Previously many themes had their own advanced settings — many of which did the same things as other themes, but they all did it a little differently — no more with Magic.

Built by Web Chef Ian Carrico and Sam Richard (of Aurora) with contributions from Sebastian Siemssen (of Omega), Magic was built by the desire to work together to make all themes better, instead of siloing improvements within specific themes.

What’s inside?

Performance features:

• Enhancements to CSS Aggregation
• Exclude CSS files from being included
• Option to move JavaScript to the footer
• Backport of Drupal 8 JavaScript handling
• Exclude JS files from being included

Development goodies

• Rebuild Theme Registry on Page Reload
• Display a Viewport Width indicator
• Display an indicator with classes applied to the HTML. Useful when used in conjunction with Modernizr
• Export theme settings

That last one is super important, as it makes Drupal themes a little more DRY. With Magic, you can take your settings from one theme to another — or to another site completely — because they’re fully exportable. Have two different projects, and want similar asset output despite one being Omega and one being Zen? No problem, just export!

Note: the full import process has yet to land, but it’s coming very soon.

If you have an awesome trick that you always rely on during theming, open an issue and propose it to Magic. They’d love to hear from you.

Give it a shot today! Go to drupal.org/project/magic

I’d like to demo a simple how-to. There are many, many techniques to make pages load faster, but this post attempts to demonstrate large gains from very small code changes.

People often build beautiful sites with multiple easy-to-use JavaScript libraries. Then, when it comes to addressing frontend performance, suddenly those libraries are an enormous download that the users are forced to bear.

Just one image

Before you go worrying about how to minify every last library or shave tests out of Modernizr, try and see if you can remove just one photo from your design. It will make a bigger difference.

Coined by Adam Sontag, the “one less JPG” idea — nay, MOVEMENT — is summed up perfectly here:

The solution to worrying about JS lib/framework size: include one less .jpg on your site. #throneofjs

— Mike Taylor (@miketaylr) July 22, 2012

Real example

Last year we re-launched Pressflow.org. We have some mobile traffic, but it’s likely people just browsing for info, since no one has a good reason to download Pressflow onto a phone or tablet. Let’s keep their attention and make the experience fast.

We have this huge, beautiful mountain on the homepage. It’s great. But it’s also 160K. I tried making it smaller, or splitting the photo off of the background pattern, but it decreased the quality of the photo too much when I lowered the file size. We made a wonderfully small SVG logo, but that’s not an option for a photograph with this kind of detail.

How much impact does it have?

A mountain is a big thing — just like the amount of traffic Pressflow can handle — and the image we chose was meant to convey that vastness. Since it doesn’t really pack the same punch on smaller screens, why include it at all? I decided to use Modernizr and conditionally load the stylesheet that references the mountain. That way it never gets loaded by tiny screens that don’t need it.

Using the Modernizr Drupal module, I added a conditional load into the .info file of my theme:

; Load CSS with Modernizr
modernizr[Modernizr.mq('screen and (min-width: 42em)')][yep][] = css/big.css

This tells Modernizr to output a Modernizr.load() statement with the test I specified. In this case, Modernizr will only load big.css if the test is true. My test checks the width of the window using a media query — .mq() — and returns true if the screen is at least 42em, causing the CSS to be fetched. Here’s the JavaScript output:

Modernizr.load({
  test: Modernizr.mq('screen and (min-width: 42em)'),
  yep : 'http://pressflow.org/sites/all/themes/pfo/css/big.css',
});

So that’s it, instant savings!

..oh what’s that? Always test your work? Thanks for keeping me honest.

Here’s some data.

I’ve got two network waterfalls here for comparison. They show a pretty stark difference following this one-line change to my code. If a screen isn’t big enough for the mountain, it’ll only take 20 HTTP requests and 193K total. If the screen is big enough, it takes 24 HTTP requests — for the CSS and then the images inside it — totalling 384KB total. That’s a savings of 191KB (almost exactly 50%) from a single change to my code. You’d have to remove 19 copies of jQuery 2.0 to achieve this kind of bandwidth savings.

(by the way, didja hear that jQuery 2.0 has small QSA-only custom builds?)

Small screens

Big screens

You can see in the second waterfall that the Initiator of big.css is modernizr.min.js, meaning that JavaScript loaded the file after running the test.

ThoughtContentLoaded

I hope this shows how easy it can be to reduce your page weight without worrying about shaving bytes of JavaScript that are supplying valuable functionality if you know how to use them right.

If you want to know more about the conditional loading API within Modernizr, head over to yepnope.js documentation and start reading. For more Drupal-specific examples check out the official documentation for conditional loading using the Modernizr module.

Hey, speeders! There have been some great presentation videos put up on the web recently, so I’ve got links to a couple videos this week.

Breaking the 1000ms time-to-glass barrier

Last week we checked out slides from Ilya Grigorik, so if you enjoyed those you have to check out his 45-min presentation recorded during the last SFHTML5 meetup. For mobile web developers this is required watching. He starts with the constraint that he wants to load a mobile web page in less than 1000ms, and proceeds backwards, walking through the realities of phone chips/towers, mobile network latency, and battery usage, in addition to more familiar concepts like image compression, the browser rendering stack, and reflow/repaint gotchas. Warning: you might need a cigarette after watching this video!

Steve Souders HTML5 DevConf keynote

Steve Souders delivered a great keynote at the HTML5 Developer Conference, and he makes it very clear that users want speed. One of the original powerhouses in the frontend performance world, Steve has created more resources and tools than is possible to list here, so I’ll just encourage you to check out his site if you’re not familiar with his work. His latest blog post also contains some errata from this video so if you like it please head over there to pick up the corrections.

I’ve been kicking around this idea for quite some time and finally sat down to get it published. Four Kitchens has always taken performance very seriously, but traditionally most people only focus on backend server performance when thinking about these types of issues. I’m not a hardcore backend dev, but I have plenty of experience making things fast and useful within a web browser.

To that end, I’d like to spread the word about frontend performance by publishing a periodic blog post series highlighting techniques, articles, and other webperf stuff that we’re talking about day-to-day in the office. So without further ado, here are this week’s links:

Building Faster Mobile Websites (.pdf)

Ilya Grigorik presents an extremely compelling case for putting speed at the top of your priority list, especially on responsive and otherwise mobile websites. He explores the realities of network latency and suggests methods that could be used to break the 1000ms “time-to-glass” barrier. It’s a low-level, deep dive into the effects of TCP/IP and HTTP requests on your site’s loading speed, and contains tons of great information to help you build fast sites. To make the most of these slides, you’ll have to work closely with backend and/or deployment people who help manage your sites, but the payoff is well worth it.

Double-whammy from Alex Sexton

Alex has recently published a couple great resources that can actually help you achieve the lofty goals in Ilya’s slides. First is a blog post entitled Deploying JavaScript Applications that talks about building a web application which matches user action patterns, relies on intelligent caching, and employs distributed deployment (like CDNs) whenever possible. He also lists some lower hanging fruit such as reducing use of images before shaving bytes of JavaScript.

Next are his slides from HTML5 Developer Conference that highlight the awesome new Modernizr v3 workflow. Very closely related to deploying JS apps, he outlines how the Modernizr v3 will help you build applications which adapt to the browser loading it, allowing for tiered, optimized experiences that match each browser’s capabilities while minimizing the loading of unnecessary assets. Having helped out with the v3 work, I’m especially excited to see people begin to use it.

Prerender in Chrome for Faster Page Loads

A couple weeks back I wrote about a feature of Chromium that Ilya has helped publicize in his upcoming book, High-Performance Browser Networking. It’s a <link> tag which lets you suggest pages that Chrome should prerender in cases where your traffic funnels are extremely predictable. This is a great example of using data to accurately predict user actions rather than directly altering the load time of your pages. Although they still load at the speed of the network, the end result can reduce apparent load times to 0.0s in some cases! Check out my example which is live on Pressflow.org.

Auf Wiedersehen

I hope you enjoyed these links! As I mentioned earlier, these posts won’t always be limited to links, so check back for code snippets, full implementation walkthroughs, and other goodies in the future.

Feel free to leave feedback or topic requests in the comments, and happy perfing!

As web applications become richer and more complicated the amount of JavaScript running them increases. More code means longer download times which means more waiting before your application or web site is usable. Thankfully there’s an easy solution that’s already widely used in the web development community: minification. Minified JavaScript strips whitespace and renames variables to produce a smaller download size.

Removing whitespace is pretty straightforward, but renaming variables is a more abstract concept. With tools like UglifyJS this is handled for you but here’s an example of what’s going on. Let’s look at this simple function that takes a parameter and alerts a message with that variable:

var sayHi = function(name) {
  alert('Hi ' + name + '!');
};

With script minification the above code will be changed to this:

var sayHi=function(a){alert("Hi "+a+"!")};

without altering the behavior of the function. As you can see, all whitespace that can be removed has been and the variable “name” was changed to “a”. The end result is the removal of 26 characters from the code. It’s not hard to see that the savings from minification will quickly add up!

You can take this one step further by wrapping your code in immediately-invoked function expressions (IIFEs). That’s a pretty scary looking name, but chances are you’ve seen these before, this is an example:

(function($) {
  $('h1').css('text-decoration', 'blink');
}(jQuery));

The function wrapped in parentheses is immediately invoked with jQuery as its argument. Within the function jQuery is accessed with the $ variable. Abstracting variables in IIFEs is not required though, consider this case:

(function(Drupal) {
  Drupal.behaviors.sayHi = {
    attach: function(name) {
      alert('Hi ' + name + '!');
    }
  };
}(Drupal));

Minifying this code produces the following:

(function(a){a.behaviors.sayHi={attach:function(a){alert("Hi "+a+"!")}}})(Drupal);

Keeping the “Drupal” variable in your script makes sense because that’s the namespace you’re used to accessing it with and it does have meaning in the code, but by including it as an argument to the IIFE UglifyJS will replace any instances of “Drupal” with a shortened variable.

What about Drupal?

So, this is pretty cool, right? Unfortunately Drupal 7 doesn’t provide minification out of the box. Thankfully, there are contrib solutions to this problem! The Speedy module provides pre-minified scripts but unfortunately needs to be re-released every time core is updated or you need to re-minify the files yourself on core updates.

I wasn’t completely satisfied with this approach so I built uglify.me and a companion Drupal module, UglifyJS, to do script minification on the fly. uglify.me isn’t the only “minifier-as-a-service” out there, but I wanted to be able to throw something up quickly and be fairly confident that it wouldn’t upset some other poor developer out there hosting their own service.

The uglify.me service accepts POST requests of un-minified JavaScript and returns the minified version. Simple! The UglifyJS module provides an API to expose Drupal scripts that should be minified:

/**
 * Implements hook_uglifyjs_info().
 */
function mymodule_uglifyjs_info() {
  return array(
    drupal_get_path('module', 'mymodule') . '/js/mymodule.js',
  );
}

The downside to this approach is that each script will create another request to the uglify.me service when the site cache is cold. If you expose a lot of scripts to the UglifyJS API this will be time consuming and could cause timeouts.

If you’re running Pressflow 7.20.1+ or apply this patch to core and you have JavaScript concatenation enabled (which you should, if you’re in production) the UglifyJS module will automatically minify the concatenated scripts. This greatly reduces the number of requests to the web service and overhead associated with minification.

Caveats

The biggest issue that I’ve encountered with this module is the requests made to the external web service. If there’s a problem connecting to the remote server the time spent waiting for the response is wasted and can cause timeouts on cold caches. This is mitigated if you’re using Pressflow 7.20.1+ since core will not request files to be rebuilt if the hash of the concatenated scripts did not change.

The uglify.me service also currently strips some header comments from scripts which would could remove any copyright or license information if they exist. This should be fixable from the uglify.me service, so if it’s bothering you and I haven’t had time to fix it before you need it, As of uglify.me v0.1.0 any comment containing the words “license” or “copyright” (case insensitive), or the common build tags “@preserve” and “@cc_on” will not be stripped from the source code. This is controlled by a regular expression so if you find a general case that’s not met by the current regular expression open a pull request to fix it!

Conclusion

Script minification is a great way to reduce your site’s download footprint and increase usability. If bandwidth and performance are concerns you should be minifying, regardless of your ultimate solution. Informal testing on the recently released Full Plate Living site showed reduction in the front page’s download weight by an average of 110KB*. Not monumental, but not too bad either.

Happy minifying!

* Note: The UglifyJS module is not currently in use on the production Full Plate Living site, but it likely will be soon!

The popularity of responsive web design is constantly exposing behaviors that web browsers have gotten away with over the years. Many of these behaviors — while brilliant for a single-screen web — become a direct hindrance when dealing with the multi-screen extravaganza that we face now.

One of the most hotly debated topics is responsive images. This was a topic widely exposed by Filament Group due to their work on the Boston Globe, and it continues to be an issue for every bandwidth-conscious site today.

History

The foundations of this topic have been hotly debated and oft explored in the past year, so here’s a list of articles that capture the history of responsive images and demonstrate the challenges developers face on a modern site:

• Mat Marquis, part of Filament Group and the Boston Globe team, took it upon himself to take the helm of this issue, writing about the initial challenges and creating the Responsive Images Community Group. The group has proposed a solution and is busy solidifying a web standard for responsive images, backed by the trial and error of developers worldwide.
• Jason Grigsby has written extensively about responsive image techniques, deftly balancing the need for simplicity on the authors’ side, while recognizing the underlying tech issues browsers inadvertently (and rightfully) created during our beloved single-screen golden era.
• Robert Paul Lloyd writes about how no solution is ideal, suggesting that we go back to the drawing board and create an entirely new image format. In the long term this is pretty likely. For certain types of imagery a bandwidth-cheap, scalable format already exists in SVG, icon fonts, and other vector formats. A similar vessel needs to exist for photos too.

Today’s solution

Whatever the final outcome of the standards process, if you find yourself needing responsive images today there are several approaches, the most successful of which is currently the <picture> element.

In a nutshell, <picture> is a brand new element with behavior that is — for now — controlled by JavaScript. This allows us to achieve all of the goals for a proper responsive image request: avoiding image prefetching, preferring author-controlled image selection, and ultimately only downloading a single image (instead of a fallback plus larger version)

Picturefill is its own open-source library, available on Github.

Implementing responsive images on a Drupal site

The latest responsive site that Four Kitchens launched, Full Plate Living, tightly integrates responsive images with Drupal content. The methods for doing so are thankfully rather easy, but as with all things Drupal there is a bit of a learning curve to get started. Responsive images can be added to content, views, etc. with the help of two contributed modules: Breakpoints and Picture. These two modules rely on Drupal’s core Image Styles module to output images in various sizes using a URL-specific, render-on-demand system.

Configuring breakpoints

In responsive web design breakpoints are used with media queries to determine when new CSS rules should apply to the site’s markup. As its name suggests, the Breakpoints module allows your site to define media queries on which you’d like to take action. The module provides a canonical list of breakpoints that are accessible from the database and thus surfaced to your site’s code. The Breakpoints module also has the notion of a “breakpoints group.” Groups are particularly useful for defining sets of breakpoints that are only relevant to certain parts of the site. These breakpoints can then be semantically referenced by their group name which allows you to reuse breakpoint names between groups, even if the media queries aren’t the same.

In the case of Full Plate Living we had one global set of breakpoints that was defined in the theme’s .info file:

breakpoints[smartphone] = (min-width: 320px)
breakpoints[fourhundred] = (min-width: 415px)
breakpoints[smartphone_only] = (min-width: 320px) and (max-width: 767px)
breakpoints[smartphone_landscape] = (min-width: 480px)
breakpoints[smartphone_landscape_only] = (min-width: 480px) and (max-width: 767px)
breakpoints[small_menu] = (min-width: 430px)
breakpoints[smaller_menu] = (min-width: 421px)
breakpoints[recipes_landing] = (min-wdith: 520px)
breakpoints[user_reg] = (min-width: 625px)
breakpoints[tablet] = (min-width: 768px)
breakpoints[tablet_only] = (min-width: 768px) and (max-width: 979px)
breakpoints[desktop] = (min-width: 980px)

This same set of breakpoints was used in our Sass and JavaScript files (with response.js) to provide a consistent list of breakpoints for all of our front end code to interact with. Unfortunately this list was duplicated in the three places; a future implementation might use a build tool like grunt to keep the global breakpoints list in one place and update the .info, Sass, and JavaScript definitions appropriately.

Additional breakpoint groups were added for content types so that the amount of markup required for responsive images could be kept down (one breakpoint means two picture options: the default, and the one defined for the breakpoint).

Configuring image styles

Core’s Image Styles are used to format the images that will be used in the picture element. It’s a good idea to give the image styles meaningful names so that configuring the Picture module is more straightforward. In the case of Full Plate Living we generally used two image styles per page: STYLE__full for images delivered to large viewports, and STYLE__mobile for images delivered to small viewports.

Configuring Picture

The Picture module ties information from Breakpoints and Image Styles together and ultimately provides the markup and picturefill JavaScript that will be used on the page. Each breakpoint group will have a tab on the Picture admin page that lists its breakpoints and provides a dropdown for each one to specify the image style that should be used.

Finally, when you’re defining the display style for your image fields you will choose “Picture” as the formatter for the image. The settings for this field will allow you to choose the “picture group” and a “fallback” image style. The fallback image style will be used if no breakpoint conditions are met. If you’re building mobile-first, this would be your base mobile style definition and should be the smallest image that will be delivered to the client.

Show me the data

To demonstrate the advantages of the <picture> element we captured data from the Full Plate Living recipe landing page. See those beautiful, gigantic images below the search? Such a huge image is not needed for the same effect on a small screen. Implementing <picture> resulted in a difference of 157K out of a max page weight of 746K. Here are the two network waterfalls:

Mobile

The three images in this screenshot that start with 2012 are the mobile-formatted images. They total approximately 40K at the default breakpoint.

Desktop

Looking again at those images with the same 2012 prefix, we see a total of 197K, meaning we are preventing the site from serving mobile users with approximately 157K of data that they don’t need.

You can see that the download “initiator” for the images is a script — not the HTML page — due to the picturefill script modifying the DOM and specifying the image that should be used. This DOM modification causes the markup on the page to differ based on the viewport, visible in this gist.

Div + data-attributes

The <picture> element was used on Full Plate Living, but for now it’s considered slightly more future-proof to use <div> elements with data attributes.

<div data-picture data-alt="A giant stone face at The Bayon temple in Angkor Thom, Cambodia">
    <div data-src="small.jpg"></div>
    <div data-src="medium.jpg"     data-media="(min-width: 400px)"></div>
    <div data-src="large.jpg"      data-media="(min-width: 800px)"></div>
    <div data-src="extralarge.jpg" data-media="(min-width: 1000px)"></div>
 
    <!-- Fallback content for non-JS browsers. Same img src as the initial, unqualified source element. -->
    <noscript>
        <img src="external/imgs/small.jpg" alt="A giant stone face at The Bayon temple in Angkor Thom, Cambodia">
    </noscript>
</div>

The picturefill script and Picture module both currently use the <div> method, so although your markup will look different, the end result will be the same.

In conclusion

We all recognize the need for a responsive image container on the web. Today’s solution might look very different from tomorrow’s, but the need for device-optimized media will continue to grow as fast as the variety of devices accessing the web. Just hang on, remember to test, and enjoy the ride!

Credits: this article was co-written by Elliott Foster.

Everyone does it.

There’s a piece of JavaScript that will only be used on one page, perhaps to provide some unique interactivity. It’s probably attached to a View or maybe a unique node ID. It’s so easy to toss in a drupal_add_js() and move on — or worse, throw the code in your theme. Wouldn’t it be nice if you could inline all these one-use scripts and make them appear only the page they’re needed?

Inline on the Fly

Here’s an easy way to inline scripts without losing the ability to edit them easily. We don’t want our code sitting in a PHP string so we create and maintain a real JS file, and use file_get_contents() to grab it whenever the appropriate page is built.

  // Ensure this JS ends up inline at the bottom of the page
  $options = array(
    'scope' => 'footer',
    'type' => 'inline',
  );
 
  // JS lives in its own file but is included inline when page renders
  $js_code = file_get_contents(drupal_get_path('module','my_module').'/my_code.js');
 
  // Add JS to page
  drupal_add_js($js_code, $options);

Optimized pages + organized code

I often find myself using hook_views_post_build() to apply this behavior when a specific Views display needs some custom JS to function properly. That way I don’t have to worry about the path, it just works anytime this View is used.

Avid Features users know it’s much more maintainable to keep the JS in its own file next to the View instead of stuffing it in a Views footer, or worse: tossing vital code for components into the theme’s “main” (read: only) JS file. Putting code in a theme file can seem swell until you copy a Feature for use in another project and just can’t figure out where that JavaScript went.

/**
 * Implements hook_views_post_build().
 */
function my_feature_views_post_build(&$view) {
  $has_run = &drupal_static(__FUNCTION__);
 
  if (!$has_run) {
    switch ($view->name) {
      // Check for the relevant View(s)
      case 'my_view':
        // Check for the relevant display(s)
        if ($view->current_display == 'my_block') {
          // Ensure this JS ends up inline at the bottom of the page
          $options = array(
          'scope' => 'footer',
          'type' => 'inline',
          );
 
          // JS lives in its own file but is included inline when page renders
          $js_code = file_get_contents(drupal_get_path('module','my_feature').'/my_code.js');
 
          // Add JS to page
          drupal_add_js($js_code, $options);
          $has_run = TRUE;
        }
        break;
    }
  }
}

Performance

Inlining a script avoids an http request and is great for frontend performance. However, if you have a page that is uncached and hit continuously, adding disk reads won’t be so great for the actual server’s performance. You can see in the second example there’s a reference to drupal_static(). This is a good way to avoid running a slow Drupal hook more than once per page request. Always make sure to cache the outcome of functions like this one in order to avoid too many disk reads.

This flew across my inbox/twitter/something a couple days back, and I spent all last evening having fun with it. It’s called ScriptCraft, a JavaScript API for building within Minecraft.

If you’re not familiar with Minecraft, it’s a world made of 1-meter cubes. You just dig around for minerals and build cities to protect your supplies. Then you get bored and start modding Minecraft.

How it works

ScriptCraft supplies an object called a Drone. Think of it like a turtle that can move in 3D space. Just like jQuery, the Drone always returns this, allowing you to chain commands together on the one-line console available in Minecraft. You can change the Drone’s position, create blocks of any material, create primitives, complex objects like trees, “bookmark” coordinates, and even copy/paste huge volumes of blocks. Built upon the Drone are other JS plugins, and the author has thrown in some good ones to get you started using the API, among them a couple buildings, a block-letters, and other goodies.

Say you want to avoid griefers and would like a truly impenetrable fortress. I would like an underground lair made of bedrock, for example. This is basically impossible in vanilla Minecraft, but using ScriptCraft you can do it in one comand:

/js down(11).box(7,20,1,20).up().box0(7,20,9,20).up(9).box(7,20,1,20).up().box(2,20,1,20)

…it’s not pretty, but this generates a hollow shell which you can use as scaffolding for your new fortress.

Creating a JS plugin in ScriptCraft

That command looks crazy all on one line, but here I broke it out into fortress.js:

load(__folder + "drone.js");
/**
 * Creates a fortress of bedrock and one layer of dirt on top.
 */
Drone.extend('fortress', function(material, sides, height) {
  // Drop the Drone down to the bottom of the structure
  this.down(height-1);
  // Lay down a solid layer for the floor
  this.box(material, sides, 1, sides);
  // Move up one level
  this.up();
  // Create all four walls. box0() creates hollow boxes
  this.box0(material, sides, (height-3), sides);
  // Move up above the walls
  this.up(height-3);
  // Another solid layer for the ceiling
  this.box(material, sides, 1, sides);
  // Move up one more time
  this.up();
  // Finally, a layer of dirt to remain inconspicuous ;)
  this.box(2, sides, 1, sides);
 
  // return this to allow further chaining
  return this;
});

You can chain the commands here too, but I chose not to in order to document each command clearly. When you’re happy with your creation, you’ll want to put the JS file inside ./js-plugins on your Minecraft server. Once you’re finished uploading it, you have to load the file within the game and get access to use the command:

/js load('js-plugins/path/to/fortress.js')

You’re ready to create some rooms in your fortress. Keep the material ID list handy while you assemble your commands. Materials with colons in them need to be strings.

// made of bedrock, 20 each side, 12 blocks tall
/js fortress(7,20,12)
 
// made of diamond blocks, 25 each side, 8 blocks tall
/js fortress(57, 25, 8)

Use Bukkit in JS

If all of that weren’t enough, the entire Bukkit API is available via JavaScript once ScriptCraft is loaded. That means anyone with JavaScript experience can now write whole Minecraft mods using ScriptCraft.

We haven’t installed it quite yet, but ScriptCraft will be up on Drupalcrafters.org very soon. Stop by and give it a shot!

Credits

ScriptCraft was created by Walter Higgins. He’s active on his Github project

If you’re a follower of this blog or Four Kitchens in general you might have noticed that making the web accessible to mobile devices is something we’re interested in. One of the biggest challenges with mobile computing is unreliable network connections that are inevitably encountered in the wild. This is particularly troublesome when your website becomes more of a web application that should behave the same regardless of connection status. There are many, many issues that crop up when the wire goes dark, but today we’re going to focus on one strategy for dealing with images, and a solution we created to help with this.

As web developers we’re all familiar with the <img> tag for embedding images in the document. Normally the source element of this tag points to a file on the server that the browser will fetch during a page load process. The source element can also contain what’s called a data URI. Data URIs allow you to embed base64 encoded data directly in the HTML document. In the case of image tags this means we can embed all data for an image in the document. A 1x1 gif file looks like this when it’s base64 encoded:

<img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///ywAAAAAAQABAAACAUwAOw==" />

Modern browsers will then interpret this data and display images as usual, avoiding another call to the server to fetch the image.

So what does this have to do with offline applications? Base64 encoding images and keeping them inline does two things:

1. Reduces the number of calls to the server that the client needs to make.
2. Moves the caching of these images into the cache entry for the document. If the document was cached by the browser, it’ll work offline and future loads won’t require server communication to fetch images. (Note: for simplicity’s sake I’m purposefully ignoring any JavaScript or CSS that might also be required for your application to function correctly offline).

It’s not all fun and games from here though, there are tradeoffs for base64 encoding images. Notably: file sizes tend to be a bit larger for base64+gzipped images than they would be for their binary+gzipped counterparts. There can also be a higher CPU cost on initial page loads while the browser processes the inline images. These problems can be mitigated by being careful about which images you base64 encode based on your use case.

So, now that we know about base64 encoded images and what they’re good for, let’s look at a way to replace non-base64 encoded images in an HTML document. There are lots of tools to do this, many of them online, but let’s make this more interesting and assume that we need to do the replacements on the fly before they’re served to clients. Let’s say that we have a RSS feed that contains embedded images that needs to have its contents parsed and distributed to client applications.

If you’re using node.js as a server backend or preprocessor of some sort, this is now incredibly easy thanks to the img64 module. img64 uses jsdom and jQuery to look for images in strings and replaces the source references with base64 encoded data URIs. Here’s an example of doing this:

var img64 = require('img64');
var string = "<p>Organic mattis pharetra eget lorem Brooklyn eget mattis sodales donec keytar vitae magna ut ipsum +1 sagittis. Congue proin auctor viral commodo pellentesque sem donec vinyl tellus sapien justo urna Austin non eros eros curabitur you probably haven't heard of them.</p><img src=\"http://upload.wikimedia.org/wikipedia/commons/2/23/1x1.GIF\" />";
 
img64.encodeImgs(string, function encoded(err, doc) {
  // Voici! Our 1x1 gif is now base64 encoded and
  // embedded as a data uri! The string is otherwise
  // unchanged.
  console.log(doc);
});

Now you can include content from anywhere in your web application without needing to worry about whether or not the content provider has already base64 encoded their images!

In our example case this means that the HTML delivered to the browser will now contain base64 encoded images for all the feed content, regardless of its origin. In addition to the advantages of base64 encoding images we outlined above this also means that the images hosted on other servers will not need to be fetched and downloaded. Assuming you have permission to use these images this means your application will not need to be concerned if an image’s host goes down or an image is moved, etc: your app will always behave the same as long as the base64 encoded image is present.

The img64 module was created as a proof of concept and while it’s definitely a viable solution it hasn’t been battle tested yet. Check it out; if it works for you, awesome! If not, open a pull request. Happy encoding!

A few weeks ago Pantheon sponsored a code sprint at their offices in San Francisco, the purpose of which was to extend support for the systemd journal. In attendance were the two maintainers of systemd: Lennart Poettering and Kay Sievers. We worked on creating native language bindings, and I headed up the JavaScript work - specifically creating a library that would allow asyncronous logging directly to the journal from node.js.

You can now do this in your application code:

var journald = require('journald').Log;
journald.log({
    MESSAGE: 'Hello world',
    ARG1: '<useful string here>',
    ARG2: '<another useful string here>',
    ARG3: '<yet another useful string here>'
 });

Systemd is the new default system manager in many major distributions. It handles starting and stopping services, provides socket activation capabilities, control over processes using cgroups, and much more. One of the new features it provides is the journal - a replacement for syslog that brings logging into the 21st century. There are many improvements over syslog, but one of the most obvious is the use of structured log entries, providing key/value fields instead of a string blob that often needs to be parsed again using regexes.

To add an entry to the journal from JavaScript, we need to break out of the interpreted world and call a C function that’s provided by the systemd API: sd_journal_sendv(). To do this, we create a V8 extension in C++, and implement the required functions to expose our new calls to JavaScript.

V8 provides neatly wrapped implementations of all JavaScript language components, so there’s a C++ version of the Function object, and a Number object, and every other piece you may require. You can easily build your JavaScript code from within C++, and return it back to JavaScript land for execution - an experience that feels something like being a backstage director at an elaborate costumed production.

The second part of the node.js API is the ability to hand off tasks to libuv, so that the main event loop doesn’t become blocked waiting for I/O. To do this, the uv_queue_work() function must be called with data about the job, a function to call to do the work, and a function to call once it’s completed. Since the V8 API isn’t available in the libuv worker threads, this part requires pure C++/C code.

The journald package is now available on npm, and it comes with a Winston transport plugin, allowing you to use Winston’s logging API to send to multiple places at once. The source is available on Github.

I found the following resources useful:

• The node.js addons documentation
• Konstantin Käfer’s presentation on building C++ extensions
• The v8 API reference
• Example C++ modules
• Writing native bindings
• v8 engine tutorial
• The v8 users mailing list