An Introduction to APIs with Phoenix

Phoenix is taking the internet by storm, with good reason. It's productive, fault-tolerant, concurrent, safe as a compiled language, and blazing fast. It shares many of Rails' core values, such as convention over configuration, restful resources, and a focus on developer happiness.

The cherry on the top: Phoenix was designed from the ground up for WebSockets connections -- so you know it's ready for today's web.

Today, we will build a restful API with Phoenix and compare it to a Ruby on Rails API. Before we get our hands dirty, however, let's make a case for learning Phoenix. Why should you switch?

If you're looking for performance, prepare to be delighted. Phoenix is built on top of Elixir, which uses BEAM (the Erlang virtual machine). Elixir compiles directly to BEAM byte code. Because of this, it can take advantage of the 30 years of stability and performance optimizations that Erlang offers.

Elixir is by no means a fast language. If you want to crunch numbers or manipulate images, then you should probably pick Rust, Go, or C over Elixir. However, Elixir is extremely performant when handling thousands or even millions of connected devices. This is why Phoenix responds to calls in a few milliseconds, if not microseconds.

You might say that performance is not a good reason to switch. After all, if you use Rails, that means that fast time to market is more important to you than performance. That's what Rails is best at: getting out of your way as you focus on building software.

The good news is that Phoenix is great at that as well. So you don't have to pick either performance or productivity: Phoenix delivers on both fronts.

In traditional languages (imperative, procedural, object-oriented), the construct for parallel or concurrent execution of code is the thread.

Web servers or frameworks typically have some type of thread pool, and when a request comes in, the server/framework decides which thread should handle the request. This can be problematic for multiple reasons, memory management and error recovery being the two main problems.

Heroku restarts its dynos daily to prevent memory from skyrocketing. Recently, memory leaks have been found in mainstream open-source projects like Sidekiq. This is because thread management is difficult. And memory management doesn't even deal with problems associated with a thread that has a segfault and brings down the whole server.

These problems simply don't exist in Elixir. The construct used for concurrent execution is the process.

Processes are independent applications that have their own garbage collector. They are completely isolated. If a process has a catastrophic failure, only that process is affected, and the application continues to work normally.

This is exactly how Phoenix servers operate. When a request comes in, it isn't being shared with a thread pool. Instead, each request uses its own independent web server. If that process (web server) dies, the application is unaffected. And when that request is finished, that process garbage-collects itself.

Everything so far makes for a rock-solid web framework, but it still says nothing about a developer's ability to ship products.

Rails has been the framework that stands apart in this aspect. Phoenix was written by rock-solid Rails developers. It takes all the good lessons that 10 years of Rails brought but leaves behind the things that Rails can't/won't let go of.

The easiest way to explain this is to look at some code and talk about the differences. The following assumes the reader has Elixir (1.2.0), Phoenix (1.1.2), and PostgreSQL installed. The Phoenix installation guides can be found here.

One of the first problems a young Padawan Rails developer faces is knowing when to type rails and when to type rake. And when teaching young developers, that's surprisingly difficult to explain.

The rails command is used to run generators, start the server, create new apps, and so on, but the rake command is used to migrate the database, run the tests, etc.

There is no rhyme or reason to it. A developer will type rails generate migration to create the migration but rake db:migrate to make the migration happen. It's confusing. Thankfully much of this will be fixed in Rails 5; most commands will use rails, so look for that in a few months.

The Elixir language has a tool called mix. The mix command is always used. If you ever want to know what commands are available, you just type mix help.

In a command prompt, grep for "phoenix":

$ mix help | grep -i phoenix
mix phoenix.new       # Create a new Phoenix v1.1.2 application

Now use mix to create a new Phoenix API:

$ mix phoenix.new todo_api
...
Fetch and install dependencies? [Yn] y
* running mix deps.get
We are all set! Run your Phoenix application:
    $ cd todo_api
    $ mix phoenix.server
You can also run your app inside IEx (Interactive Elixir) as:
    $ iex -S mix phoenix.server
Before moving on, configure your database in config/dev.exs and run:
    $ mix eco.create

The option --no-brunch does not install brunch.io, which is the build tool used to build assets; it requires Node.js. And the --no-html removes code involved in creating traditional websites.

If you don't use those options, a hybrid app is generated: an app that can have API routes and traditional HTML routes. There is no performance penalty for this. The only reason I skipped those options is because they are outside the scope of this blog post.

The much anticipated Rails 5 --api option is an all-or-nothing approach. If you need HTML pages later, you cannot add them. This makes it almost unusable. Most APIs eventually have some type of traditional HTML pages for administrative use.

The instructions from the last command are nice. They tell the user how to run the server and create the database using Ecto. Ecto is an ORM much like Active Record or LINQ.

Following the instruction given by the phoenix.new command:

$ cd todo_list

Modify the config/dev.exs and config/test.exs files to have the proper credentials. On a Mac where PostgreSQL was installed with Homebrew, just remove the username and password.

config :todo_api, TodoApi.Repo,
  adapter: Ecto.Adapters.Postgres,
  # username: "postgres", # remove
  # password: "postgres", # remove
  database: "todo_api_dev",
  hostname: "localhost",
  pool_size: 10

Create the database:

$ mix ecto.create

Run the tests:

$ mix test
...
Finished in 0.1 seconds (0.1s on load, 0.00s on tests)
3 tests, 0 failures

Mix compiled the app for the first time. Don't worry, mix only recompiles when files change and then only the files that changed. Type mix test again. It's instantaneous!

Three tests for an app that does nothing. What are they?

# /test/views/error_view_test.exs
defmodule TodoApi.ErrorViewTest do
  use TodoApi.ConnCase, async: true
  # Bring render/3 and render_to_string/3 for testing custom views
  import Phoenix.View
  test "renders 404.json" do
    assert render(TodoApi.ErrorView, "404.json", []) ==
           %{errors: %{detail: "Page not found"}}
  end
  test "render 500.json" do
    assert render(TodoApi.ErrorView, "500.json", []) ==
           %{errors: %{detail: "Server internal error"}}
  end
  test "render any other" do
    assert render(TodoApi.ErrorView, "505.json", []) ==
           %{errors: %{detail: "Server internal error"}}
  end
end

These tests assert that out-of-the-box errors return proper JSON. But more importantly they test views, one of the core differences in Phoenix. A view is a group of functions used to render JSON, or if this was an HTML view, these functions would be available in the template.

Open the view and notice that it's just a module with functions:

# /web/views/error_view.ex
defmodule TodoApi.ErrorView do
  use TodoApi.Web, :view
  def render("404.json", _assigns) do
    %{errors: %{detail: "Page not found"}}
  end
  def render("500.json", _assigns) do
    %{errors: %{detail: "Server internal error"}}
  end
  # In case no render clause matches or no
  # template is found, let's render it as 500
  def template_not_found(_template, assigns) do
    render "500.json", assigns
  end
end

The important takeaway for a JSON API is this: Views create dictionaries that represent the JSON for an action. And views are easy to test. This comes in handy if views have complex logic.

However, most of the time, a controller test that asserts the proper response is good enough. Those types of tests are considered integration tests.

Frameworks aren't really mature until they have generators. Luckily Phoenix has plenty. Listing them is easy:

$ mix help | grep -i phoenix
mix phoenix.digest      # Digests and compress static files
mix phoenix.gen.channel # Generates a Phoenix channel
mix phoenix.gen.html    # Generates controller, model and views for an HTML based resource
mix phoenix.gen.json    # Generates a controller and model for a JSON based resource
mix phoenix.gen.model   # Generates an Ecto model
mix phoenix.gen.secret  # Generates a secret
mix phoenix.new         # Create a new Phoenix v1.1.2 application
mix phoenix.routes      # Prints all routes
mix phoenix.server      # Starts applications and their servers

Use the phoenix.new.json generator to scaffold out a JSON resource:

$ mix phoenix.gen.json Todo todos description:string
complete:boolean
* creating web/controllers/todo_controller.ex
* creating web/views/todo_view.ex
* creating test/controllers/todo_controller_test.exs
* creating web/views/changeset_view.ex
* creating priv/repo/migrations/20160120010759_create_todo.exs
* creating web/models/todo.ex
* creating test/models/todo_test.exs
Add the resource to your api scope in web/router.ex:
    resources "/todos", TodoController, except: [:new, :edit]
Remember to update your repository by running migrations:
    $ mix ecto.migrate

The generator says to add the a line to the router. Add it to the /api scope:

defmodule TodoApi.Router do
  use TodoApi.Web, :router
  pipeline :api do
    plug :accepts, ["json"]
  end
  scope "/api", TodoApi do
    pipe_through :api
    resources "/todos", TodoController, except: [:new, :edit]
  end
end

Routes initially feel very familiar to a seasoned Rails developer:

# Rails
resources :todos, except: [:new, :edit]

# Phoenix
resources "/todos", TodoController, except: [:new, :edit]

They can be written very similar to Rails, with a few exceptions. The controller is specified in the route. There are no plurals. Anyone who has ever lost several hours tracking down a bug, only find out they didn't pluralize correctly, should be ecstatic about this.

Lastly, routes have pipelines, which are composable middleware stacks. They are composed of plugs which are middleware similar to Rack but with several advantages.

Run the migration and then run the tests:

$ mix eco.migrate
// stuff happens ...
$ mix test
.............
Finished in 0.2 seconds (0.1s on load, 0.07s on tests)
13 tests, 0 failures

Awesome, another passing test suite for free. For now, ignore the tests. Instead focus on the similarities and differences in the Phoenix framework. Open the model first:

# web/models/todo.ex
defmodule TodoApi.Todo do
  use TodoApi.Web, :model
  schema "todos" do
    field :description, :string
    field :complete, :boolean, default: false
    timestamps
  end
  @required_fields ~w(description complete)
  @optional_fields ~w()
  @doc """
  Creates a changeset based on the `model` and `params`.
  If no params are provided, an invalid changeset is returned
  with no validation performed.
  """
  def changeset(model, params \\ :empty) do
    model
    |> cast(params, @required_fields, @optional_fields)
  end
end

This is another major difference from Rails. In Rails, the mantra is "fat model, skinny controller." This is not the case with Phoenix.

Models are a data type and are treated as such. They are just vanilla structs that use the special schema macro to map to the database. There is one exception to this. Functions called "changesets" are used to scrub params and validate data. In the above example, the cast\3 method is being used to scrub the params.

Now open the view:

# web/views/todo_view.ex
defmodule TodoApi.TodoView do
  use TodoApi.Web, :view
  def render("index.json", %{todos: todos}) do
    %{data: render_many(todos, TodoApi.TodoView, "todo.json")}
  end
  def render("show.json", %{todo: todo}) do
    %{data: render_one(todo, TodoApi.TodoView, "todo.json")}
  end
  def render("todo.json", %{todo: todo}) do
    %{id: todo.id,
      description: todo.description,
      complete: todo.complete}
  end
end

Exactly what was expected. A set of functions that's used to generate JSON. This is great -- it will be easy to modify this code later.

Now open the controller:

# web/controllers/todo_controller.ex
defmodule TodoApi.TodoController do
  use TodoApi.Web, :controller
  alias TodoApi.Todo
  plug :scrub_params, "todo" when action in [:create, :update]
  def index(conn, _params) do
    todos = Repo.all(Todo)
    render(conn, "index.json", todos: todos)
  end
  def create(conn, %{"todo" => todo_params}) do
    changeset = Todo.changeset(%Todo{}, todo_params)
    case Repo.insert(changeset) do
      {:ok, todo} ->
        conn
        |> put_status(:created)
        |> put_resp_header("location", todo_path(conn, :show, todo))
        |> render("show.json", todo: todo)
      {:error, changeset} ->
        conn
        |> put_status(:unprocessable_entity)
        |> render(TodoApi.ChangesetView, "error.json", changeset: changeset)
    end
  end
  def show(conn, %{"id" => id}) do
    todo = Repo.get!(Todo, id)
    render(conn, "show.json", todo: todo)
  end
  def update(conn, %{"id" => id, "todo" => todo_params}) do
    todo = Repo.get!(Todo, id)
    changeset = Todo.changeset(todo, todo_params)
    case Repo.update(changeset) do
      {:ok, todo} ->
        render(conn, "show.json", todo: todo)
      {:error, changeset} ->
        conn
        |> put_status(:unprocessable_entity)
        |> render(TodoApi.ChangesetView, "error.json", changeset: changeset)
    end
  end
  def delete(conn, %{"id" => id}) do
    todo = Repo.get!(Todo, id)
    # Here we use delete! (with a bang) because we expect
    # it to always work (and if it does not, it will raise).
    Repo.delete!(todo)
    send_resp(conn, :no_content, "")
  end
end

The syntax is different from Ruby, but this is the exact same logic one would expect to find in a Rails controller.

One difference is that the create and update actions return the changeset when the request is invalid, instead of the model. Given that models are just data objects that map to the database, this makes sense. A different data type, the changeset, is used to carry information about valid and invalid data.

If you've been dreaming of a framework that is blazing fast, fault-tolerant, compiled, and concurrent, but you've always come back to Rails because of its productivity... dream no more. The next blog post in this series will tackle CORS, authentication from scratch, and dependency management. Until then, enjoy your new-found productivity and performance!