Trigger Jenkins builds by pushing to Github

Connecting a Github private repository to a private instance of Jenkins can be tricky. In this post I’ll show you how to configure both services so that pushes to your Github repository will automatically trigger builds in Jenkins, all while keeping both safely hidden from the general public.

Step 1: Grant your server access to your private Github repository.

You may have Jenkins running on the same machine as your webhost, or they may be on separate machines with Jenkins configured to treat the webhost as a remote node. Either way, you’re going to want to SSH into the webhost and ensure that whichever Linux user Jenkins is building jobs as, can authenticate to Github. We have a robot user called ‘Bender’ (yeah, from Futurama) exactly for this purpose, so I’ll use that in the examples.

Instead of installing your own private key to the Bender account, create a new set of private/public keys, and then either create a Github user for Bender or use the Github deploy keys feature. Follow those links for the excellent guides from Github.

There are pros and cons to each approach which are discussed on the deploy-keys help page, but if you have multiple private repositories and don’t want a separate key for each, rather create a Github user for Bender.

Don’t proceed until you get the message “You’ve successfully authenticated” when executing ssh git@github.com as Bender.

Step 2: Install the Git and Github plugins.

Under ‘Manage Jenkins’ -> ‘Manage Plugins’, select and install both Github and Git plugins. Restart to finish the installation.

Configure both of these at ‘Manage Jenkins’ -> ‘Configure System’. Make sure the path to git is correctly set, and choose ‘Manually manage hook URLs” under the ‘Github Web Hook’ section.

Step 3: Configure a Jenkins job to use your repository.

The interface for configuring a job is peppered with references to Github, so it can be confusing.

Firstly, add the https:// url for your repository in the ‘GitHub project’ textfield under the general settings.

Then you’ll need to enable Git under ‘Source Code Management’. Use the SSH style syntax for the URL repository: git@github.com:user/repo.git (this is required as it’s a private repo), and specify a branch if needed. The advanced settings are optional.

Under ‘Build Triggers’, tick ‘Build when a change is pushed to Github’.

Save and build your job. You should get a successful build that correctly clones the repository to the webhost. Confirm by SSH’ing in and inspecting it.

Step 4: Grant Github access to your private Jenkins instance.

Unfortunately, this step will require you to store a plain-text user/password combination on Github, unless you’re using the Github OAuth plugin (see below). The good news is that you can lock down the user pretty tightly, so that in the event of a security breach on Github, an attacker would not be able to do anything more malicious than build your project and view previous builds.

There are a number of different authentication options in the ‘Security Realm’ section of ‘Manage Jenkins’ -> ‘Configure System’. Depending on your setup, these steps could differ, but in essence you need to create a new user for Github (I’ll just assume you used the username ‘Github’). If you’re using ‘Unix user/group database’ method, be sure to lock that new user down by restricting the shell so that SSH sessions are denied.

If you’re using the Github OAuth plugin for Jenkins to tightly tie your access to Github accounts, you can just tick the option to allow access to the POST webhook URL. However, this option is only available when using Github as the authentication server. I won’t go into detail but this allows you to skip this step entirely, as it allows anonymous access to the URL.

In the ‘Authorization’ section, choose ‘Project-based Matrix Authorization Strategy’, so that you can give project-level permissions to the Github user. You’ll probably deny access to everything for anonymous users, then grant just one permission here for Github: ‘Overall/Read’.

In the configuration for the job that will be automatically triggered, tick ‘Enable project-based security’ and then grant ‘Job/Read’ and ‘Job/Build’ to Github.

Test the Github user by logging into Jenkins with it’s credentials and ensuring that you can only see and build the relevant job.

Step 5: Add the hooks to Github.

Click the ‘Admin’ button on the main page of your private repository in Github. Under the ‘Service hooks’ -> ‘Post-Receive URLs’ section, add the URL of your Jenkins webhook, including the credentials of the Github user you created. For example:

https://USERNAME:PASSWORD@jenkins.example.com/github-webhook/

It’s great that Jenkins supports authentication in this format. You can test the hook from here, and confirm that there is a result under the ‘Github Hook Log’ section of your Jenkins project (it’s in the left column).

That’s it! Push some code to your repository and your project will gracefully begin building. As an added bonus, you get great information from the Github plugin, such as links to the diff for each build shown on the build page itself: