How to Build Rails APIs Following the json:api Spec

We've talked before about how to build a JSON API with Rails 5. We also discussed using Rails 5 in --api mode, serializing our JSON responses, caching, and also rate limiting/throttling.

These are all important topics, but even more important is producing a clear, standards-compliant API. We're going to look at how to build an API that conforms to the json:api spec.

The json:api spec isn't just limited to how the server should format JSON responses. It also speaks to how the client should format requests, how to handle sorting, pagination, errors, and how new resources should be created. It even speaks to which media types and HTTP codes should be used.

We're going to be implementing many of these things in our Rails app, and we'll also look at how we might test them. Lastly, we'll take a quick look at one possible authentication strategy and some tools we can use to produce beautiful documentation for our API.

The json:api spec is pretty big! It covers many of the possible features you may want to implement in a JSON API. May is the key word... you don't have to implement all of them for your API to be json:api compliant. For example: You don't have to have to implement sorting, but if you do, the json:api spec will tell you how it must be done.

One of the first things you'll notice with json:api is that it doesn't actually use the application/json media type. The reason for this is that the group behind json:api actually went through the proper process and channels to register their own media type, which is application/vnd.api+json. Part of the contract of the server is that it will respond with the Content-Type header set properly to this media type.

The easiest way I was able to find to get this working without having to set it manually in every action is to create an initializer to set them. I called it register_json_mime_types.rb, and it contains:

api_mime_types = %W(
  application/vnd.api+json
  text/x-json
  application/json
)
Mime::Type.register 'application/vnd.api+json', :json, api_mime_types

Now whenever a request is made it will automatically have Content-Type:application/vnd.api+json; charset=utf-8 set in the response headers.

One of the questions that came up from my last article was how we should go about creating a resource. What format should the data come in? What URL should the data be sent to? How should the server respond on success? On errors? Thankfully all of those answers are available, and my goal is to show how to create a resource while following the json:api spec.

For these examples, we'll be working with two models:

• User

• RentalUnit

Their fields don't matter too much, and they have the relationship of a RentalUnit belonging to a User and a User having many RentalUnits.

Routing is probably one of the easiest parts. The specification follows RESTful routing pretty much identically to what Rails gives you by default when you define the route with resources :rental_units.

The specific route we are interested in for creating a resource is POST /rental_units HTTP/1.1.

If you've done very much Rails development before, you're probably used to data being posted to the server in a fairly simple format, which may look somewhat like this:

{
  rental_unit: {
    price_cents: 100000,
    rooms: 2,
    bathrooms: 1
  }
}

We'll have to change this a little bit to conform to the specification. In the guide we see that it must be formatted like this:

data: {
  type: 'rental_units',
  attributes: {
    price_cents: 100000,
    rooms: 2,
    bathrooms: 1
  }
}

This matches how json:api is formatted on response, so it is at least familiar/consistent!

To ensure that our API responds correctly, let's write a test. The test will post the data to the correct URL, and will then verify that the server responded with a 201 HTTP status code (which means resource created). After that, we'll look for a Location header, which tells us where we can find this new resource that was created.

Lastly, we'll look to verify that it responded in the correct json:api response format using a custom matcher which I'll include below.

require 'rails_helper'
RSpec.describe "Rental Units", :type => :request do
  let(:user) { create(:user) }
  describe "POST create" do
    it "creates a rental unit" do
      post "/rental_units", {
        params: {
          data: {
            type: 'rental_units',
            attributes: {
              price_cents: 100000,
              rooms: 2,
              bathrooms: 1
            }
          }
        },
        headers: { 'X-Api-Key' => user.api_key }
      }
      expect(response.status).to eq(201)
      expect(response.headers['Location']).to match(/\/rental_units\/\d$/)
      expect(response.body).to be_jsonapi_response_for('rental_units')
    end
  end
end

Here is the custom rspec matcher I used. It does a simple check to make sure that the response conforms to the json:api spec when responding with a resource.

RSpec::Matchers.define :be_jsonapi_response_for do |model|
  match do |actual|
    parsed_actual = JSON.parse(actual)
    parsed_actual.dig('data', 'type') == model &&
      parsed_actual.dig('data', 'attributes').is_a?(Hash) &&
      parsed_actual.dig('data', 'relationships').is_a?(Hash)
  end
end

So if this is what our test looks like, what might the controller look like? It ends up looking fairly similar to how it might have before. The only difference here is that I have to dig a little deeper to get to the attributes coming in through the params object.

class RentalUnitsController < ApplicationController
  def create
    attributes = rental_unit_attributes.merge({user_id: auth_user.id})
    @rental_unit = RentalUnit.new(attributes)
    if @rental_unit.save
      render json: @rental_unit, status: :created, location: @rental_unit
    else
      respond_with_errors(@rental_unit)
    end
  end
  private
  def rental_unit_params
    params.require(:data).permit(:type, {
      attributes: [:address, :rooms, :bathrooms, :price_cents]
    })
  end
  def rental_unit_attributes
    rental_unit_params[:attributes] || {}
  end
end

You might have noticed above that I have a special method called respond_with_errors for when the @rental_unit object is unable to save. But before we get to that, let's take a look at how json:api expects us to format the errors:

{
  errors: [
    status: 422,
    source: {pointer: "/data/attributes/rooms"},
    detail: "Must be present."
  ]
}

To help with this, I've created a small method that lives in the ApplicationController to help respond with errors:

def respond_with_errors(object)
  render json: {errors: ErrorSerializer.serialize(object)}, status: :unprocessable_entity
end

The ErrorSerializer class it is referring to is just a simple module which helps format the errors in the correct way.

module ErrorSerializer
  def self.serialize(object)
    object.errors.messages.map do |field, errors|
      errors.map do |error_message|
        {
          status: 422,
          source: {pointer: "/data/attributes/#{field}"},
          detail: error_message
        }
      end
    end.flatten
  end
end

I've written a small test to make sure that the API responds correctly. It again uses a custom rspec matcher which I'll include below. What I am looking for here is that it responds with the 422 HTTP code (unprocessable entity) and that it contains an error for a specific field.

it "responds with errors" do
  post "/rental_units", {
    params: {
      data: {
        type: 'rental_units',
        attributes: {}
      }
    },
    headers: { 'X-Api-Key' => user.api_key }
  }
  expect(response.status).to eq(422)
  expect(response.body).to have_jsonapi_errors_for('/data/attributes/rooms')
end

Here is the custom matcher which looks to see if there is an error for a specific field, in this case the rooms field:

RSpec::Matchers.define :have_jsonapi_errors_for do |pointer|
  match do |actual|
    parsed_actual = JSON.parse(actual)
    errors = parsed_actual['errors']
    return false if errors.empty?
    errors.any? do |error|
      error.dig('source', 'pointer') == pointer
    end
  end
end

The next thing we are going to look at is how to sort results within our API. I've grouped this together with pagination because the way I coded it they go hand in hand within the same class.

You may be thinking at this point why I mentioned class. Doesn't this normally happen within the index action of the RentalUnitsController? Normally yes, but while doing this, I found that it was a bit more code than I was comfortable with to leave it all inside the controller. It's also a good example of how you might extract some complicated logic out of the controller into its own class or module.

The action itself is dead simple:

def index
  rental_units_index = RentalUnitsIndex.new(self)
  render json: rental_units_index.rental_units, links: rental_units_index.links
end

The RentalUnitsIndex class has one job, to handle preparing the queries and data necessary to respond to the the GET /rental_units HTTP/1.1 request. It receives self (the controller) so that it can access things such as the params object as well as the URL helpers.

class RentalUnitsIndex
  DEFAULT_SORTING = {created_at: :desc}
  SORTABLE_FIELDS = [:rooms, :price_cents, :created_at]
  PER_PAGE = 10
  delegate :params, to: :controller
  delegate :rental_units_url, to: :controller
  attr_reader :controller
  def initialize(controller)
    @controller = controller
  end
  def rental_units
    @rental_units ||= RentalUnit.includes(:user).
      order(sort_params).
      paginate(page: current_page, per_page: PER_PAGE)
  end
  def links
    {
      self:  rental_units_url(rebuild_params),
      first: rental_units_url(rebuild_params.merge(first_page)),
      prev:  rental_units_url(rebuild_params.merge(prev_page)),
      next:  rental_units_url(rebuild_params.merge(next_page)),
      last:  rental_units_url(rebuild_params.merge(last_page))
    }
  end
  private
    def current_page
      (params.to_unsafe_h.dig('page', 'number') || 1).to_i
    end
    def first_page
      {page: {number: 1}}
    end
    def next_page
      {page: {number: [total_pages, current_page + 1].min}}
    end
    def prev_page
      {page: {number: [1, current_page - 1].max}}
    end
    def last_page
      {page: {number: total_pages}}
    end
    def total_pages
      @total_pages ||= rental_units.total_pages
    end
    def sort_params
      SortParams.sorted_fields(params[:sort], SORTABLE_FIELDS, DEFAULT_SORTING)
    end
    def rebuild_params
      @rebuild_params ||= begin
        rejected = ['action', 'controller']
        params.to_unsafe_h.reject { |key, value| rejected.include?(key.to_s) }
      end
    end
end

The nice thing about the way this is written is that it would be quite easy to test what might end up being complicated logic to perform sorting, pagination, and maybe at some point in the future, filtering of the rental units.

If this were an API I was developing for real, I would most likely extract a lot of these generic methods into a parent class.

Sorting in json:api is done through a single query param called sort which comes through the URL. It might look like this ?sort=-rooms,price_cents, which would sort descending by the rooms field, and then ascending by the price_cents field.

This functionality is handled by the sort_params method, which farms out the work to a module called SortParams. This module has the job of taking a string such as -rooms,price_cents and converting it into the usual Hash that the order method wants to receive. From -rooms,price_cents to {rooms: :desc, price_cents: :asc}.

module SortParams
  def self.sorted_fields(sort, allowed, default)
    allowed = allowed.map(&:to_s)
    fields = sort.to_s.split(',')
    ordered_fields = convert_to_ordered_hash(fields)
    filtered_fields = ordered_fields.select { |key, value| allowed.include?(key) }
    filtered_fields.present? ? filtered_fields : default
  end
  def self.convert_to_ordered_hash(fields)
    fields.each_with_object({}) do |field, hash|
      if field.start_with?('-')
        field = field[1..-1]
        hash[field] = :desc
      else
        hash[field] = :asc
      end
    end
  end
end

We've been trying to test functionality as we build it, so here are the tests to ensure that it is sorting correctly when the sort param is included.

We've created some rental units and are requesting that they return in descending order based on the amount of rooms they have.

describe "GET index" do
  it "returns sorted results" do
    create(:rental_unit, rooms: 4)
    create(:rental_unit, rooms: 5)
    create(:rental_unit, rooms: 3)
    get "/rental_units", {
      params: { sort: '-rooms' },
      headers: { 'X-Api-Key' => user.api_key }
    }
    expect(response.status).to eq(200)
    parsed_body = JSON.parse(response.body)
    rental_unit_ids = parsed_body['data'].map{ |unit| unit['attributes']['rooms'].to_i }
    expect(rental_unit_ids).to eq([5,4,3])
  end
end

With pagination, most of the details are left up to the programmer to decide. That is because pagination could be done quite differently from app to app, either page by page or by where your cursor is on the screen (say in an infinite scroll view). What is dictated by the spec is that you should pass this information through query params in the page key.

It may end up looking like this: ?page[number]=1. You are also required to include a links key in the response which provides links to the current page (self), the first, previous, next, and last links. Most of these details are handled by the RentalUnitsIndex class in combination with the will_paginate gem.

These are included in the rendered response from within the controller which calls out to our helper class:

render json: rental_units_index.rental_units, links: rental_units_index.links

Here we will test to ensure that it is returning paginated results along with the links correctly.

it "returns paginated results" do
  (RentalUnitsIndex::PER_PAGE + 1).times { create(:rental_unit) }
  get "/rental_units", {
    params: { sort: '-rooms', page: {number: 2} },
    headers: { 'X-Api-Key' => user.api_key }
  }
  expect(response.status).to eq(200)
  parsed_body = JSON.parse(response.body)
  expect(parsed_body['data'].size).to eq(1)
  expect(URI.unescape(parsed_body['links']['first'])).to eq('http://www.example.com/rental_units?page[number]=1&sort=-rooms')
  expect(URI.unescape(parsed_body['links']['prev'])).to eq('http://www.example.com/rental_units?page[number]=1&sort=-rooms')
  expect(URI.unescape(parsed_body['links']['next'])).to eq('http://www.example.com/rental_units?page[number]=2&sort=-rooms')
  expect(URI.unescape(parsed_body['links']['last'])).to eq('http://www.example.com/rental_units?page[number]=2&sort=-rooms')
end

One interesting but slightly unrelated thing I discovered while working on this article was that params has changed in Rails 5. It is now a different object instead of the old HashWithIndifferentAccess object. Thanks to Eileen Uchitelle for a great article pointing out what has changed.

The type of authentication you need depends on the type of application you are building. If it is a public API, you might not need it at all, but more and more I have seen a simple API key given out, allowing the server to track your usage and giving them more control over its access.

Alternatively, it may be an API which powers a single page application or a mobile app. Here you will still go with a token solution, perhaps like the one that devise_token_auth gives you, but it is a little bit more complicated to implement because there is more security involved (the token may change on every single request).

The main thing to take away is that there are no cookies involved... there is no magical state that the browser handles for us. In fact, because we've built this app using the --api version of Rails 5, the cookies functionality doesn't even exist.

We're going to go with the first solution I mentioned, a simple API key that is attached to the user's account. When they request a page, they'll include that in the request headers under the X-Api-Key value.

Here is what our User model looks like. It will assign each new user a key (which could later be regenerated and/or modified if we need to shut this account out because of abuse).

class User < ActiveRecord::Base
  has_many :rental_units
  before_create :set_initial_api_key
  private
  def set_initial_api_key
    self.api_key ||= generate_api_key
  end
  def generate_api_key
    SecureRandom.uuid
  end
end

Inside of the ApplicationController, we'll include a few small methods which will help us validate that the token is in fact correct, who it belongs to, and to respond with a 401 unauthorized response if it is invalid.

class ApplicationController < ActionController::API
  before_action :authenticate
  private
    def authenticate
      authenticate_api_key || render_unauthorized
    end
    def authenticate_api_key
      api_key = request.headers['X-Api-Key']
      @auth_user = User.find_by(api_key: api_key)
    end
    def auth_user
      @auth_user
    end
    def render_unauthorized
      render json: 'Bad credentials', status: 401
    end
end

Here we have a test that ensures it is working correctly:

it "responds with unauthorized" do
  post "/rental_units", {
    params: {},
    headers: { 'X-Api-Key' => '12345' }
  }
  expect(response.status).to eq(401)
end

As far as I can tell, json:api doesn't speak much or at all about authentication, and it is left up to the implementor to decide what works best for their specific API and its use case.

Part of what makes APIs great -- such as the ones from Stripe, Twilio, or GitHub -- is how good their documentation is. I won't go into too much detail here other than to recommend two of the best approaches I've seen.

The first approach, a gem called apipie-rails, is done by adding extra details to your tests, which automatically generates documentation. This has the benefit of being easy to keep in sync with your code as it is right there alongside it, but I find that it can tend to clutter up the tests a little bit.

The second approach is one called slate which allows you to create some really beautiful API documentation, much like the ones I listed above. This is done in markdown, and it ends up generating static HTML/CSS files that you can host.

In this article, we expanded upon a previous article about creating JSON APIs within Rails. We tackled some common problems that occur when creating APIs, such as wanting pagination, sorting, errors, and authentication. We looked at how we can develop these features in keeping with the json:api spec.

By doing so, we avoid needing to have conversations about how to implement each feature, and we can instead focus on the specifics of our application.