Rescuing myself from the email monster with JavaScript

We here at Four Kitchens do love us some email. Last week, I self-awarded a prize for having achieved 4,000 unread items in my inbox, with another 3,000 read items sitting in my inbox for no reason. I could keep going for the record, but I thought I’d attempt to use nerdiness to take better control.

(Ultimately, it’s a chore that simply has to be done, but maybe it can be easier.)

TL;DR: If you just want the source code, grab it on GitHub.

Filters feel limited

I came to Gmail from Exchange; Filters have always seemed less powerful than Rules.

My gripes with Gmail Filters:

• Search is limited, seemingly, to word and number characters; special characters have no effect in searches, so I can’t isolate “[ category ]” strings we use in subject lines.
• Complex groups of boolean logic have a tendency to produce imprecise results and there’s no way to have “additional criteria” (even if multiple fields are used in the advanced search/filter builder, they’re flatted into a single query)
• Other header information or metadata can’t be used in the search criteria (like mailed-by to help weed out notifications “from” a user sent on their behalf by another service)
• Filters cannot be run on sent mail (for the purposes of auto-labeling)
• Filters cannot be run on a delay
• No regular expression matching

Ta-Daa! Gmail can be scripted with JavaScript!

Checkout Google Apps Script and create a new blank project.

This is just JavaScript, but it runs server-side within Google Apps and can be run on regular intervals or on specific triggers. You do not have to be logged in with a window open to make this work!

Step 1: Migrate filters to JavaScript for more power

Goal: Label everything, both incoming and outgoing. Additionally, some theads are starred, marked as /(un)?(important|read)/, or immediately auto-archived.

function autoTagMessages(thread, index, threads) {
  var msg     = thread.getMessages()[0],
      subject = thread.getFirstMessageSubject(),
      to      = [msg.getTo(), msg.getCc()].join(', '),
      from    = msg.getFrom(),
      any     = [to, from].join(', ');

A new function is created for tagging messages. I compile a list of useful variables and then move straight to categorizing:

Timely Messages generally require direct action, quickly. They are added to the label ‘~/Announcements’ by thread.addLabel() and also starred using message.star().

Gotchas:

• The addLabel() function requires a Label object, not a string. Such an object can be obtaind using GmailApp.getUserLabelByName().
  • Be sure to include ‘parent/child’ if your labels are hierarchical. (In this case, ‘Announcements’ is a child of ‘~’).
• When using string.match(), be sure to add the i flag at the end of the patte\r\n \to ignore case, since authors may be inconsistent with case.
• If you see an error like Cannot retrieve (line X, file "Code") where X is a line containing a getUserLabelByName call, the most likely cause is that Gmail couldn’t find that label.

// Immediate To-Do Items
if (subject.match(/[timely]/i) !== null) {
  msg.star();
  thread.addLabel( GmailApp.getUserLabelByName("~/Announcements") );
}

Whereabouts emails are generally uninteresting since I work remotely most of the time. They’re all flagged as ‘~/Whereabouts’ and, if they don’t appear to indicate that the sender will be unavailable, they are archived immediately:

// Whereabouts Info (except stuff I don't care about)
if (subject.match(/[(whereabouts|wfw*|ooo)]/i) !== null) {
  thread.addLabel( GmailApp.getUserLabelByName("~/Whereabouts") );

  // Most of this is just "I'm working at home today", but this may be
  // a poorly-imagined idea... We'll see...
  if (subject.match(/(ooo|offline|unavailable|errands)/i) === null) {
    thread.moveToArchive();
  }
}

Regex on line 2 visualized:

Google Calendar emails can be identified by what the subject line starts with:

// Google Calendar Stuff
if (subject.match(/^((Updated )?Invitation|Accepted|Canceled( Event)?):/) !== null) {
  thread.addLabel( GmailApp.getUserLabelByName("~/Calendaring") ).markUnimportant();
}

Regex visualized:

Client emails get sorted as well. We’re a little lax in the formatting of those tags, but regex makes that easier:

else if (any.indexOf('fullplateliving.org') > -1 || subject.match(/[f(ull)?s?p(late)?s?(l|living)?]/i)) {
  thread.addLabel( GmailApp.getUserLabelByName("#/Full Plate Living") );
}

Regex visualized:

This matches [fpl], [full plate], [full plate living], [fullplateliving], and various others, as well as any email sent to/from @fullplateliving.org.

In general, the function contains three pieces:

• If statements testing timeliness or general discussion topics
• If/else statements testing for one of any application notification (Google Calendar, GitHub, JIRA, Notable, etc.) since a thread won’t be from multiple
• If/else statements testing for one of any client name, since a thread is unlikely to pertain to multiple clients directly, although I may change this.

This allows a thread to end up with multiple labels at the expense of running a little slower, but the load is reduced by being conservative with the triggers (Step 3).

Step 2: Script email expirations

My second function will archive threads that have dated out. Since the autoTagMessages() function has nearly everything categorized, I’ll base retention and expiration off of labels, thread ages, and whether or not the thread is read. This can be done by executing Gmail searches programmatically using GmailApp.search().

Set up the searches as standard search queries:

// Archive anything matching these searches
var searches = [
  // General Stuff:
  'in:inbox label:~-whereabouts older_than:1d', // Highly timely
  'in:inbox label:~-calendaring older_than:3d', // Shows in Google Calendar
  '(in:inbox label:~-watercooler) AND ((is:read older_than:7d) OR (is:unread older_than:21d))',
  '(in:inbox label:~-announcements) AND ((is:read older_than:14d) OR (is:unread older_than:1m))',

  // Services Updates (timely; probably seen in-application)
  '(in:inbox) AND (label:~-jira OR label:~-notable OR label:~-harvest) AND ((is:read older_than:1d) OR (is:unread older_than:3d))',
  'in:inbox label:~-hipchat older_than:1d',

  // Catch all, don't keep anything stale:
  'in:inbox is:read older_than:2m'
];

Then run the searches and, in batches of 100 (batchSize), archive the resulting threads:

for (i = 0; i < searches.length; i++) {
  // Run the search, EXLUDING anything that is starred:
  var threads = GmailApp.search(searches[i] + ' AND (-is:starred)');

  // Batch through the results to archive:
  for (j = 0; j < threads.length; j+=batchSize) {
    GmailApp.moveThreadsToArchive(threads.slice(j, j+batchSize));
  }
}

Gotchas: That AND (-is:starred) at the end of the search string doesn’t always work. Sometimes threads with starred messages are archived anyway. But there is a way to fix that:

var threads = GmailApp.search('-in:inbox is:starred');
for (k = 0; k < threads.length; k+=batchSize) {
  GmailApp.moveThreadsToInbox(threads.slice(j, j+batchSize));
}

(I didn’t say it was a graceful way… Investigating better options…)

Step 3: Setup triggers (like Cron for your inbox)

Now I have two functions:

1. autoTagMessage() – given a thread, label it appropriately.
2. autoArchive() – search for email that can be archived and do so.

Trigger autoTagMessage hourly on new email only

Google Apps Scripts can be triggered routinely, but unlike Outlook, there is no ‘run as a message is received’ option. This can be emulated by:

• having Gmail filters assign one label to every incoming email, and then
• processing all messages in that label on a regular basis (5 minutes).

I assign the “Prefilter” label to all incoming messages by matching against having an @ in the to field. All other filters were exported and deleted. Using the Label settings, “Prefilter” can be hidden from your inbox view so you don’t see it.

Unexpected benefit: I noticed that this “Prefilter” label is applied to all outbound email as well (perhaps because it matches only against the to field), allowing messages I send to be auto-labeled with no additional work!

Then, in a new function, I get those threads and tag them:

function batchIncoming() {
  GmailApp.getUserLabelByName("Prefilter").getThreads().forEach(autoTagMessages);
}

Next, amend autoTagMessages() to remove that label, and, if a thread has multiple messages, abort. This will prevent re-labeling an entire thread for any new messages in it (which would only be annoying in the case that a message is starred; for example, replies to a [timely] thread would be starred otherwise).

thread.removeLabel( GmailApp.getUserLabelByName("Prefilter") );
if (thread.getMessageCount() > 1) { return; }

Now I have two functions that can be run on a regular basis, so let’s do so. Under the “Resources” menu, click “Current project’s triggers” and add these:

• autoArchive() can run hourly (or less frequently, honestly).
• batchIncoming() must run very frequently. I chose 5 minutes instead of 1 so that it wouldn’t start again before the last execution has finished.
  • Google Apps Scripts will timeout and abort at five minutes, although I haven’t hit that limitation.

Declare a reset, then profit

Be sure to warn folks when you’re about to purge a few thousand threads from your inbox. Then sit back, keep up with what you can using the auto-labeling help you’ve built, and let Google Apps Scripts help you.

Next steps

I’m still working to:

1. Find an efficient way to filter threads with starred messages out of a GmailApp.search() result, so that I don’t have to do that stupid “un-archive any starred threads” maneuver in autoArchive(). There is a method thread.hasStarredMessages(), but using that would require iterating over each thread in the result-set, which seems expensive for an otherwise batched process.

Additional reading

• Create time-based Gmail filters with Google Apps Script
• Awesome Things You Can Do With Google Scripts