Creating a Custom Build Status Page Using Codeship API v2

UPDATE: As of January 10th, 2018, we released a Go client for Codeship’s API. To learn more, read the documentation article.

Codeship's API v1 will allow you to get data about your projects and builds that run on Codeship Basic, but if you've switched to Codeship Pro in the past few months, you probably noticed that the API won't work for your projects. Fortunately, that's all changed, as the Codeship API v2 allows you more powerful access to your projects and builds, including those running on Codeship Pro.

We use Codeship Pro for our Docker-based projects at The Graide Network, and one thing that we've wanted is the ability to show non-technical team members details about our build statuses. It's often helpful for our customer service team to know when or if a new feature or bug fix has been released to production yet -- showing the most recent production builds in a dashboard is a simple way to solve this problem.

In this tutorial, I'll walk you through the process of building a custom Codeship build status dashboard in Node and Express that displays the most recent build for each of your projects. You will also be able to drill down to individual projects and see the last several builds.

You can see a demo of a custom dashboard I set up for my open-source projects at builds.khughes.me, or you can get the whole codebase on GitHub. Let's get started!

Before we get started, let's touch on the requirements for this project.

1. It needs to authenticate with Codeship API v2. While viewers of the page won't need to have their own Codeship accounts, you will need to have a Codeship account with read access to some projects in order to get something from the API.

2. It should be able to display all our Codeship projects and the most recent build status of each. We'll also include the most recent commit message and date so that users can learn more if there's a problem. We'll use the Codeship API v2 to accomplish this task.

3. The project will use Node 8, Express 4, Bootstrap 4, and Handlebars. We mostly use PHP at The Graide Network, but Node and Express are great for lightweight projects like this.

Finally, here's a look at the final product we'll create:

Now that we have some parameters, we can start building the application.

We'll use Node Package Manager (commonly referred to as NPM) to install the required dependencies for our application. Create a new directory on your local machine called build-status, navigate to the directory, and create a package.json and index.js file.

The package.json file includes the external packages we will use to build this application as well as a start script for testing it locally:

{
  "name": "build-status",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "start": "node_modules/.bin/nodemon index.js"
  },
  "dependencies": {
    "express": "^4.15.4",
    "express-handlebars": "^3.0.0",
    "nodemon": "^1.11.0",
    "request": "^2.81.0",
    "request-promise-native": "^1.0.4"
  }
}

To install the NPM packages, just run npm install from your command line. Node will install the packages in the node_modules/ directory and create a package-lock.json file (if you're using NPM 5+). We can now use these packages to build our application.

The index.js file will set up our Express application, inject Handlebars, and define routes for our application. In a larger Express app, it might make sense to have multiple files for each of these functions, but because this is a simple demo, it's okay to put everything in one file. Add this to your index.js file:

'use strict';
const express = require('express');
// Set up express
const app = express();
// Create route
app.get('/', (req, res) => {
  res.send('Hello World!');
});
// Run the application
app.listen(3000, () => {
  console.log('Listening on port 3000');
});

This is not much more than the Express "Hello World" example, but let's go ahead and run it. Type npm start in your command line and point your browser to localhost:3000. You should see Hello World! in your browser.

Next, we want to create a new directory and file to house our application logic. While this is a simple app, it's still helpful to break controllers out from our routes to keep things cleaner.

Make a new directory called controllers/ and a file called projects.js. In the projects.js file, add:

'use strict';
function list(req, res) {
  res.send('Your projects here.');
}
module.exports = {list};

Update your index.js file to call the new projects controller:

'use strict';
const express = require('express');
const projects = require('./controllers/projects');
// Set up express
const app = express();
// Create endpoints
app.get('/', projects.list);
// Run the application
app.listen(3000, () => {
  console.log('Listening on port 3000');
});

You can again run the application with npm start. This time you should see the text, Your projects here. in the browser at localhost:3000.

In order to display anything meaningful in our build status page, we'll need to connect to the Codeship API. Codeship API v2 has three methods that we'll need for this application:

1. POST /auth In order to authenticate, we'll need to use our Codeship username and password passed as Basic Auth credentials. If successful, Codeship will return an access token, expiration timestamp, and an array of the organizations we belong to.

2. GET /organizations/{organization_uuid}/projects For the sake of this application, we're going to get all of the projects associated with our first organization. In my case, I only belong to one organization, so this works nicely, but if you belong to several, you may have to inject some logic to query specific organizations by UUID.

3. GET /organizations/{organization_uuid}/projects/{project_uuid}/builds Finally, we'll loop through each project and get a list of the latest builds and their statuses. This endpoint requires both an organization and project UUID.

Project UUIDs are different from the project IDs you see in your browser in Codeship's web interface. These UUIDs can be found on the project's settings page or via the API.

Let's create a new directory and file to hold our Codeship API code. Create a directory called clients/ and a file called codeship.js. In the new JavaScript file, we'll have a function for each of the API calls listed above:

'use strict';
const request = require('request-promise-native');
const apiUrl = process.env.CODESHIP_API_URL;
const username = process.env.CODESHIP_USERNAME;
const password = process.env.CODESHIP_PASSWORD;
let authorization;
async function postAuth() {
  if (accessTokenExpired()) {
    authorization = await request({
      uri: apiUrl + '/auth',
      method: 'POST',
      auth: {
        user: username,
        pass: password,
      },
      json: true,
      headers: {'Content-Type': 'application/json'},
    });
  }
  return authorization;
}
function listProjects(organizationId) {
  return request({
    uri: apiUrl + '/organizations/' + organizationId + '/projects',
    method: 'GET',
    json: true,
    headers: {
      'Content-Type': 'application/json',
      'Authorization': authorization.access_token,
    },
  });
}
function listBuilds(organizationId, projectId) {
  return request({
    uri: apiUrl + '/organizations/' + organizationId + '/projects/' + projectId + '/builds',
    method: 'GET',
    json: true,
    headers: {
      'Content-Type': 'application/json',
      'Authorization': authorization.access_token,
    },
  });
}
function accessTokenExpired() {
  const now = Math.round((new Date()).getTime() / 1000);
  if (authorization && authorization.access_token && authorization.expires_at) {
    return authorization.expires_at <= now;
  }
  return true;
}
module.exports = {postAuth, listProjects, listBuilds};

Next, we'll update the projects.js controller to call the Codeship API through our client and return the JSON as text:

'use strict';
const apiClient = require('../clients/codeship');
async function list(req, res) {
  // Authenticate the user
  const authorization = await apiClient.postAuth();
  // Use their first organization UUID
  const orgId = authorization.organizations[0].uuid;
  // Get the organization's projects
  let projects = (await apiClient.listProjects(orgId)).projects;
  // Embed the latest builds into each project
  projects = await Promise.all(projects.map(async (project) => {
    project.builds = (await apiClient.listBuilds(orgId, project.uuid)).builds;
    return project;
  }));
  // Return the results as text
  res.send(JSON.stringify(projects));
}
module.exports = {list};

I've elected to use the async/await functions available in Node 8. These functions are useful for ensuring that one API call is complete before moving on to the next without the messiness of callbacks or promises. If you're not running Node 8, you can use the Dockerfile included in the GitHub repo to run the application in a container or you can use Node Version Manager to choose your Node version.

If you refresh localhost:3000 at this point, you'll see an error like this in your command line:

(node:89254) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): RequestError: Error: Invalid URI "undefined/auth"
(node:89254) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

The problem is that our Codeship API Client file requires some environmental variables that we haven't set yet. In order to set these, stop the Node server and restart it with your Codeship credentials included in the command line. For example:

$ CODESHIP_API_URL="https://api.codeship.com/v2" CODESHIP_USERNAME="youremail@example.com" CODESHIP_PASSWORD="123456" node_modules/.bin/nodemon index.js

This embeds CODESHIP_API_URL, CODESHIP_USERNAME, and CODESHIP_PASSWORD as environmental variables that are then used in the Codeship client we created.

Now when you load localhost:3000, you should see a large JSON string with all your projects and builds. Some of this information is meant to be private, and some will be used to populate our views created in the next section.

Handlebars is a templating engine that allows us to embed JavaScript variables and perform some basic logic within view files. You can also create helper functions to be used in your application. To use Handlebars, first open up the index.js file and update it:

'use strict';
const express = require('express');
const exphbs = require('express-handlebars');
const projects = require('./controllers/projects');
// Set up express and handlebars
const app = express();
const handlebars = exphbs.create({
  defaultLayout: 'main',
  helpers: {
    ifEquals: function (arg1, arg2, options) {
      return (arg1 === arg2) ? options.fn(this) : options.inverse(this);
    },
  },
});
app.engine('handlebars', handlebars.engine);
app.set('view engine', 'handlebars');
app.get('/', projects.list);
app.listen(3000, () => {
  console.log('Listening on port 3000');
});

Notice that we defined one helper function called ifEquals. We'll use this later to compare the value of two strings in our view files.

Next, update the controllers/projects.js file to render a view instead of returning the JSON as text:

'use strict';
const apiClient = require('../clients/codeship');
async function list(req, res) {
  // Authenticate the user
  const authorization = await apiClient.postAuth();
  // Use their first organization UUID
  const orgId = authorization.organizations[0].uuid;
  // Get the organization's projects
  let projects = (await apiClient.listProjects(orgId)).projects;
  // Embed the latest builds into each project
  projects = await Promise.all(projects.map(async (project) => {
    project.builds = (await apiClient.listBuilds(orgId, project.uuid)).builds;
    return project;
  }));
  // Render the list view with projects
  res.render('list', {projects: projects});
}
module.exports = {list};

At this point, we haven't actually created the view files, so nothing will be rendered. Let's create a new views directory, a layouts subdirectory, a main layout, and a list view. The structure of the new files should be:

- views/
  - list.handlebars
  - layouts/
    - main.handlebars

The main.handlebars layout file should include the following:

<html lang="en">
<body>
  <div class="container-fluid">
    <div class="row">
      <div class="col-md-8 ml-auto mr-auto">
        <h1>Codeship Build Status</h1>
        <p class="lead">Here's the status of my Codeship builds. See a problem? Make a pull request!</p>
        {{{body}}}
      </div>
    </div>
  </div>
  <footer>
    <div class="container-fluid">
      <p>© 2017, <a href="https://www.karllhughes.com/">Karl Hughes</a>. <a href="https://github.com/karllhughes/build-status">This code is open source</a>.</p>
    </div>
  </footer>

<script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.11.0/umd/popper.min.js" integrity="sha384-b/U6ypiBEHpOf/4+1nzFpr53nxSS+GLCkfwBdFNTxtclqqenISfwAzpKaMNFNmj4" crossorigin="anonymous"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-beta/js/bootstrap.min.js" integrity="sha384-h0AbiXch4ZDo7tp9hKZ4TsHbi047NrKGLO3SEJAg45jXxnGIfYzk4Si90RDIqNm1" crossorigin="anonymous"></script>
</body>
</html>

And the list.handlebars file:

<section id="projects">
  {{#each projects}}
  <div class="card">
    <div class="card-body">
      <div class="row">
        <div class="col-sm-2 build-status-icon">
          {{#ifEquals this.builds.0.status "success"}}
            <i class="fa fa-check-circle success" aria-hidden="true"></i>
          {{/ifEquals}}
          {{#ifEquals this.builds.0.status "waiting"}}
            <i class="fa fa-clock-o testing" aria-hidden="true"></i>
          {{/ifEquals}}
          {{#ifEquals this.builds.0.status "testing"}}
            <i class="fa fa-clock-o testing" aria-hidden="true"></i>
          {{/ifEquals}}
          {{#ifEquals this.builds.0.status "failed"}}
            <i class="fa fa-times-circle failed" aria-hidden="true"></i>
          {{/ifEquals}}
          {{#ifEquals this.builds.0.status "stopping"}}
          <i class="fa fa-times-circle failed" aria-hidden="true"></i>
          {{/ifEquals}}
          {{#ifEquals this.builds.0.status "stopped"}}
          <i class="fa fa-times-circle failed" aria-hidden="true"></i>
          {{/ifEquals}}
        </div>
        <div class="col-sm-10">
          <h4 class="card-title">{{ this.name }}</h4>
          <p class="card-text">
            <strong>Commit Message:</strong> {{ this.builds.0.commit_message }}<br/>
            <strong>Status:</strong>
            <span style="text-transform: capitalize">{{ this.builds.0.status }}</span>
            <br/>
            {{#if this.builds.0.finished_at}}
            <strong>Built at:</strong> {{ this.builds.0.finished_at }}
            {{else}}
            {{#if this.builds.0.queued_at}}
            <strong>Queued at:</strong> {{ this.builds.0.queued_at }}
            {{/if}}
            {{/if}}
          </p>
        </div>
      </div>
    </div>
    <div class="card-body">
      <a href="{{ this.repository_url }}" target="_blank" class="card-link">
        {{ this.repository_provider }}
      </a>
    </div>
  </div>
  {{/each}}
</section>

Now if you refresh localhost:3000, you'll see a list of your Codeship Pro projects and the results of each project's most recent build.

While the build status dashboard we just created demonstrates how you can use the Codeship API v2, there are probably a few things that you'd want to do before using this to create a real production-ready project.

If you take a look at the demo application I created on GitHub, you'll see that I've also added a Dockerfile, instructions for running and deploying as a container, and a page to view the latest builds for a single project as well. You can see the whole application at builds.khughes.me.

There are also a few issues in the project that I would address if I were making a real project out of this status page, including caching results from Codeship's API (they currently rate-limit calls to 60 per minute), handling errors from the API, and unit testing, but I'll leave that for another demonstration.

Finally, take a look at the Codeship API v2 to learn more about what's possible. Now that Codeship is becoming a platform and not just a continuous integration solution, what kind of things are you interested in building? Let me hear about it on Twitter.