API development with X Autoload

As a native PHP Developer, I find coding in Drupal 7 sometimes limiting, because I want to use the advanced patterns I learned way back when working with Zend Framework, Code Igniter, and Symphony. Some may argue for performance reasons we should limit the number of ‘just for developer’ modules we use, but I find myself spending less time—and having more fun—by adding one in particular: X Autoload.

Make Drupal 7 development intuitive and fun!

The Challenge

Our client, Meredith Agrimedia, had a problem. They needed to get the information from the USDA-NASS Quickstats API into a map format that worked for the farmers and agribusiness—men and women who peruse their site. The challenge was that we are talking about a ton of data… megabytes even… of statistics about corn crops, their progress, yield, and the like.

Who better than the resident good ol’ country boy in overalls to whip up a custom API wrapping module? Some people would use feeds to bring in data, some people would maybe use some kind of custom migration using Migrate, but this one… this one was different with it’s own very non-REST like structure, and custom query string system that allows you to query data in what can be best described as giant spreadsheets of survey data.

The Proposed Solution

In situations like this, it is very easy to jump on the “Let’s Roll Out Our Own” bandwagon—you get all the joy of building something from scratch and can make a masterpiece, or end up with a brand new flavor of code spaghetti. I took a hint from several query builders in the past, and decided to go that direction, since the API could somewhat handle different conditions and what not. You can take a look at the documentation to see for yourself. If you are curious what I mean by “query builder,” take a look at the select portion of the Drupal Database API. Code-wise, I’m following the same ideals. But building this in D7, with include files (like ctools and views), just feels wrong… doesn’t it? Enter X Autoload.

Building a Drupal 7 Module Using PSR-4

If you have had a taste of Drupal 8 coding, and you are like me, it’s hard to go back. So let’s map out what I needed:

• overall Service connector object
• `Authentication` object to pass to the connector (basically a substitute for a configuration entity you would build in Drupal 8)
• `Query` object to house selection options any conditions and build a query
• `Condition` object to house the operations to do on specific properties in the query
• `Requester` object interface just for fun, because I considered extending it out and allowing for fixtures (but ran out of time)
• implementation of the requester interface

So how does that look?

So, in case you don’t know what PSR-4 is, it’s a Proposed Standards Recommendation that basically says after specifying a root namespace that maps to a directory (in this case defined by X Autoload which uses Drupal Coding standards), automatically load a PHP file that matches the same file structure as the namespace.

Example: in my code I create the Auth class under the Drupal\sfg_crop_maps\NassApi namespace. According to the rules X Autoload defines, we will load the file /path/to/sfg_crops_maps/src/NassApi/Auth.php. This is very exciting to me because it provides some serious structure to my code and keeps me from having to define more files in my sfg_crops_maps.info and I don’t have to worry about class/function name collisions nearly as much. I now have my own sandbox to work in. All of this is available simply by adding X Autoload as a dependency, no configuration needed.

Guidelines On Architecture for Query Builders

The query builder pattern is pretty much the hinge to this whole project. Following is a bit of a structure that you would want to build to and some hints to implement in your module

Main Service Class

Here, always build this to perform the global functions. In this case… the constructor should be initialized with an Auth and Requester object.

<?php
/**
* @file
* Houses NASS api wrapper class.
*
* @see https://quickstats.nass.usda.gov/api
*/

namespace Drupal\sfg_crop_maps;

use Drupal\sfg_crop_maps\NassApi\Auth;
use Drupal\sfg_crop_maps\NassApi\Query;
use Drupal\sfg_crop_maps\NassApi\Requester;

/**
* Wrapper and factory for querying the NASS API.
*/
class NassApi {

 protected $key;
 protected $requester;

 /**
  * Initialize the NASS Api.
  *
  * @param Auth $key
  *   The auth key for using the API.
  * @param Requester $requester
  *   The requester object that will preform the HTTP request.
  */
 public function __construct(Auth $key, Requester $requester) {
   $this->key = $key;
   $this->requester = $requester;
 }

 /**
  * Factory function for creating a query from the parameters in this instance.
  *
  * @return Query
  *   Query object for the NassAPI.
  */
 public function createQuery() {
   $query = new Query($this->key, $this->requester);
   return $query;
 }

}

Requester Class

The Requester will be used later on in the query when the query executes, so this class literally just holds instances for you to use in a singleton pattern. This allows you to do something like the following:

/**
* Helper function to get or initialize the NASS api.
*
* @return \Drupal\sfg_crop_maps\NassApi
*   NASS Api object.
*/
function sfg_crop_maps_get_api_service() {
 $api = &drupal_static(__FUNCTION__);
 if (!isset($api)) {
   $api = new NassApi(
     new \Drupal\sfg_crop_maps\NassApi\Auth(variable_get('sfg_crop_maps_nass_api_key')),
     new \Drupal\sfg_crop_maps\NassApi\Requester\DrupalHttpRequester()
   );
 }
 return $api;
}

Query Class

Your Query class should take your Auth, Requester, and any endpoint identifying bits that define the object you want to query. In my case, I only have one endpoint so I skipped that last bit. The  Query class should have a factory method in it that will either create conditions or allow you to add conditions and possibly condition groups. The Quick Stats API doesn’t allow anything more advanced than an AND operation, so I skipped condition groups.

<?php
/**
* @file
* Houses the Query class.
*/

namespace Drupal\sfg_crop_maps\NassApi;

/**
* Represents all the arguments of the query portion of the _GET endpoint.
*/
class Query {

 public $auth;
 public $conditions = array();

 /**
  * Initializes a Query.
  *
  * @param Auth $auth
  *   An Auth object containing a valid NASS Api key.
  * @param Requester $requester
  *   A Requester object used to send the HTTP request for the query.
  */
 public function __construct(Auth $auth, Requester $requester) {
   $this->auth = $auth;
   $this->requester = $requester;
 }

 /**
  * Factory for chain-able conditions.
  *
  * @param string $parameter
  *   Any valid value for a column. See https://quickstats.nass.usda.gov/api.
  * @param string $value
  *   Any appropriate value for the given column.
  * @param string $operator
  *   Any of =, !=, <=, <, >, >=, like, or not like.
  *
  * @return Query
  *   Returns this query for chaining.
  */
 public function condition($parameter, $value, $operator = '=') {
   $this->conditions[] = new Condition($parameter, $value, $operator);
   return $this;
 }

 /**
  * Sends the query object to a requester and returns the results.
  *
  * @return mixed
  *   Returns the data portion of the http request or FALSE upon error.
  */
 public function execute() {
   return $this->requester->executeQuery($this);
 }

}

Condition Class

Every query has a condition. It should handle a parameter, the value, and determine how operators are built. In hindsight, I could have moved the functionality that does the query string translation into the requester, but who hasn’t wanted to improve their code!?

<?php
/**
* @file
* Houses the Condition class.
*/

namespace Drupal\sfg_crop_maps\NassApi;

/**
* An object to house parameters and their operators when querying the API.
*/
class Condition {

 public $parameter;
 public $operator;
 public $value;

 /**
  * Initialize a query condition.
  *
  * @param string $parameter
  *   Any valid value for a column. See https://quickstats.nass.usda.gov/api.
  * @param string $value
  *   Any appropriate value for the given column.
  * @param string $operator
  *   Any of =, !=, <=, <, >, >=, like, or not like.
  */
 public function __construct($parameter, $value, $operator) {
   $this->parameter = $parameter;
   $this->operator = strtolower($operator);
   $this->value = $value;
 }

 /**
  * Translates the operator into something the api expects.
  *
  * @return string
  *   Returns the postfix for the query string based on operator.
  */
 public function operatorQueryString() {
   switch ($this->operator) {
     case '=':
       return '';

     case '!=':
       return '__NE';

     case '<':
       return '__LT';

     case '<=':
       return '__LE';

     case '>':
       return '__GT';

     case '>=':
       return '__GE';

     case 'like':
       return '__LIKE';

     case 'not like':
       return '__NOT_LIKE';

     default:
       throw new \Exception($this->operator . ' is an invalid operator string.');

   }
 }

 /**
  * Grab the full GET parameter string for the condition being queried.
  *
  * @return string
  *   parameter name plus operator. EX: year__GE for year, >=.
  */
 public function parameterQueryString() {
   return $this->parameter . $this->operatorQueryString();
 }

 /**
  * Grab the value formatted for a query string.
  *
  * @return string
  *   Transforms the value into the format to be used in the api.
  */
 public function valueQueryString() {
   return $this->value;
 }

}

Requester Classes

Lastly, the Requester object needs to translate and build the query object into something that it can then use to request information from the API.

<?php
/**
* @file
* Houses the NassAPI Requester interface.
*/

namespace Drupal\sfg_crop_maps\NassApi;

/**
* An interface allowing to dependency inject the query request runner.
*/
abstract class Requester {

 const NASS_URL = 'https://quickstats.nass.usda.gov';
 const NASS_QUERY_URL = self::NASS_URL . '/api/api_GET/';

 /**
  * Accepts a query object and runs an http request using it.
  *
  * @param Query $query
  *   Query object that will be used to build the http request.
  *
  * @return mixed
  *   Returns the `data` parameter of the JSON packet from the query request.
  *   Returns a request object that must have code and error properties.
  */
 public abstract function executeQuery(Query $query);

}

<?php
/**
* @file
* Houses the DrupalHttpRequester class.
*/

namespace Drupal\sfg_crop_maps\NassApi\Requester;

use Drupal\sfg_crop_maps\NassApi\Query;
use Drupal\sfg_crop_maps\NassApi\Requester;

/**
* Wraps the drupal_http_request() function to use it for hitting the NASS API.
*/
class DrupalHttpRequester extends Requester {

 /**
  * {@inheritdoc}
  */
 public function executeQuery(Query $query) {
   $parameters = array(
     'key' => $query->auth->key,
   );
   foreach ($query->conditions as $condition) {
     $parameters[$condition->parameterQueryString()] = $condition->valueQueryString();
   }
   $response = drupal_http_request(
     self::NASS_QUERY_URL . '?' . drupal_http_build_query($parameters),
     array('timeout' => 60)
   );
   if (isset($response->error)) {
     watchdog(
       'sfg_crop_maps',
       "@error<br /> <pre>@response</pre>",
       array(
         '@error' => $response->error,
         '@response' => print_r($response, TRUE),
       ),
       WATCHDOG_ERROR,
       url(self::NASS_QUERY_URL . '?' . drupal_http_build_query($parameters))
     );
     return $response;
   }
   return json_decode($response->data)->data;
 }

}

Ultimately, I went with drupal_http_request because I wasn’t using Guzzle for anything else and that would be yet another library for me to depend on and figure out how to load. Ideally, we could have used Guzzle by telling X Autoload to load libraries also, but I didn’t mess with that. If I was using Composer for this project, I would have definitely went that route.

What Does That Hard Work Do?

So, you’re wondering… “What does a query look like then? After all that hard work, what does that get me?”

Before I show you, keep a few things in mind:

• Always cache API data, even if it is for seconds. There is rarely a case where you are going to need up-to-date data every time someone hits the page.
• Document what’s going on. A newbie—or even you may not know what’s going on down the road. Do yourself a favor and document.
• Know your API’s limitations. There is nothing like building all of this and then realizing that the API only allows one of each parameter and therefore isn’t like a real query. Find out as much as you can early on and then plan on how to follow the query pattern above.

I ended up with something like this in my module getting specific data for the map shown at the top of the article:

/**
* Query the NASS api for the percentage planted records for specified year.
*
* @param string $crop
*   Crop we are querying status for. See Commodity parameter.
* @param string $year
*   Year to query status for. See Year Parameter.
*
* @return mixed
*   The data sent back by the api. An array of objects when successful. FALSE
*   when there is no data. An integer with the api fail code in the case of an
*   error.
*/
function sfg_crop_maps_get_planting_progress($crop, $year) {
 // Convert crop strings.
 $crop = sfg_crop_maps_convert_crops_string($crop);

 // Lets store this in memory in case we use it over and over.
 $data = &drupal_static(__FUNCTION__ . $crop . $year, array());
 if (empty($data)) {
   // For performance of page load, we will cache this until cache is cleared.
   if ($cache = cache_get(__FUNCTION__ . $crop . $year, 'cache_sfg_crop_maps')) {
     $data = $cache->data;
   }
   else {
     $api = sfg_crop_maps_get_api_service();
     $query = $api->createQuery();
     $statistic_categories = array(
       'PROGRESS',
       'PROGRESS, 5 YEAR AVG',
       'PROGRESS, PREVIOUS YEAR',
     );
     // Since you can't query for several parameters at once, we have to do
     // multiple calls.
     foreach ($statistic_categories as $statistic_category) {
       $result = $query->condition('commodity_desc', $crop)
         ->condition('source_desc', 'SURVEY')
         ->condition('sector_desc', 'CROPS')
         ->condition('group_desc', 'FIELD CROPS')
         ->condition('statisticcat_desc', $statistic_category)
         ->condition('unit_desc', 'PCT PLANTED')
         ->condition('year', $year)
         ->execute();
       // Handling any error we may see. We don't want to cache the error so
       // lets return here. Also you can't tell the difference between an empty
       // query and an error because they both come back as "BAD REQUEST IVALID
       // QUERY". The first query is the parent. If it fails, fail all.
       if (isset($result->error)) {
         if ($result->code > 400) {
           // Handling any error we may see. We don't want to cache the error
           // so lets return here.
           return $result->code;
         }
         elseif (empty($data) && $result->code == 400) {
           // If we get a 400, that means the main statistic is empty. We have
           // no data to give. All empty result sets show up as a "BAD REQUEST"
           // error from NASS.
           return FALSE;
         }
         else {
           $result = array();
         }
       }
       $data = array_merge($data, $result);
     }
     cache_set(__FUNCTION__ . $crop . $year, $data, 'cache_sfg_crop_maps');
   }
 }
 return $data;
}

Final Thoughts

So what do corn, autoloaders, and Drupal all have in common? Nothing so witty that this developer in overalls would burn a wheel in his head faster than a squirrel could hide a nut, but it sure as heck was a lot more fun, organized, and applicable to the everyday world of coding than making some flat module. Seriously, this could be pushed out as a library if I changed out the DrupalHTTPRequester object.

So, what have we’ve learned from this? Use X Autoload. Make it easier on yourself by using more modern practices and design patterns. Finally… really… have fun when you code.