Running a Rails Application on Deis

In the last couple of years, we have seen a lot of development in the devops field. It's becoming much easier for developers to provision servers and deploy their applications on those servers just with a couple of key strokes. Since the start of the SaaS and PaaS products (even before we knew them as that), we have seen a vast number of companies and communities try to make our lives easier by developing smart tools that will fit into our workflow seamlessly.

As a side effect, or perhaps intentionally, we've also seen the open-source community create tools to not just make the workflow easier but to cut costs and enable us to make great software without thinking about the costs of the hardware.

Amazon with its huge selection of web services (AWS) and the like have made our lives much easier when it comes to incorporating infrastructure and tools in the cloud. Then there are Chef and Puppet that made provisioning servers a breeze. Then came containerization in the form of Docker, although some would argue that containers, or at least the idea around them, existed well before Docker came along.

The newest addition to this collection of tools is Deis. This lightweight application platform deploys and scales Twelve-Factor apps as Docker containers across a cluster of CoreOS machines. Deis will wrap a release of your application in a Docker container and deploy it to one (or multiple) machines running on CoreOS.

After that, you will be able to scale, manage, limit, and do much more to your application. Also, because Deis is an application platform, you as an administrator can change and customize the platform itself. It's basically having your own, self-hosted Heroku. The folks at Engine Yard, and all the contributors that created Deis, have done a marvelous job.

To get some things straight from the start, this tutorial will be about running a Rails application on Deis. If you haven't seen the concepts and architecture that powers Deis, I recommend checking out the Understanding Deis page. If you do not have Deis installed and would like to follow this tutorial as you read, you can head over to the installation guide.

To run Deis smoothly, you'll need a cluster of servers, three at minimum. Although Deis comes with Vagrantfiles that can provision and install the platform on virtual machines, I strongly recommend using servers in the cloud if you don't have a very powerful computer. I tried using Vagrant on my MacBook Air 2013, and let's just say it didn't go well. For this tutorial, I used AWS, but the documentation has instructions on installing Deis on any of the most popular cloud providers.

Before we start, there's a bit to know about deploying applications on Deis. Any applications or service that can be run inside a Docker container can be deployed to Deis. In order to be scaled horizontally, applications must follow Heroku’s Twelve-Factor methodology and store state in external backing services.

This means that data should be written to a database, images should not be persisted to the local filesystem (they should be uploaded to a cloud provider), dependencies should be isolated, logging should be enabled, and so on. By complying to these 12 factors, you can easily scale your application inside Deis on demand.

On the bright side, most modern applications are stateless, have proper logging, and have nicely isolated external services and dependencies. This makes them horizontally scalable inside Deis.

The application that we'll be using for this tutorial is a sample made by Michael Hartl, for his book Ruby On Rails Tutorial (3rd ed). It's basically a minimal Twitter clone, where users can register, follow other users, and post microposts. Users can also change their profile settings and view their profiles and the profiles of other users on the application.

Deis as a PaaS allows multiple ways of deploying an application.

The first is using buildpacks. These pieces of software are basically scripts that prepare your code for execution by the Deis controller. Buildpacks are useful if you’re interested in following Heroku’s best practices for building applications or if you're deploying an application that already runs on Heroku. This can make the transition from Heroku to Deis seamless.

Deployment is very easy, just like on Heroku. From the project directory, we need to register a new application on Deis. This is done by running:

deis create

You will see an output similar to this:

Creating Application... done, created aerial-nonesuch
Git remote deis added
remote available at ssh://git@deis.mydomain.com:2222/aerial-nonesuch.git

Note that the URL and the application name might be different in your case. When you run the deis create command, Deis will not only register a new application, but it will also add a new git remote to your repo with the name deis. This is the remote where we will push our code to be deployed. It's just like Heroku, where the Heroku toolbelt adds a new remote called heroku.

After creating the application on Deis, the next thing to do is to deploy the application by Git push:

sample_app_rails_4 git:(master) git push deis master
Counting objects: 948, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (484/484), done.
Writing objects: 100% (948/948), 429.01 KiB | 0 bytes/s, done.
Total 948 (delta 428), reused 941 (delta 424)
-----> Ruby app detected
-----> Compiling Ruby/Rails
-----> Using Ruby version: ruby-2.0.0
-----> Installing dependencies using bundler 1.9.7
       Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
       Fetching gem metadata from https://rubygems.org/...........
       Fetching version metadata from https://rubygems.org/...
       Fetching dependency metadata from https://rubygems.org/..
       * gems omitted *
       Bundle complete! 28 Gemfile dependencies, 51 gems now installed.
       Gems in the groups development and test were not installed.
       Bundled gems are installed into ./vendor/bundle.
       Bundle completed (29.38s)
       Cleaning up the bundler cache.
-----> Writing config/database.yml to read from DATABASE_URL
       Detected manifest file, assuming assets were compiled locally
       ###### WARNING:
       No Procfile detected, using the default web server (webrick)
       https://devcenter.heroku.com/articles/ruby-default-web-server
-----> Discovering process types
       Default process types for Ruby -> rake, console, web, worker
-----> Compiled slug size is 26M
-----> Building Docker image
remote: Sending build context to Docker daemon 28.13 MB
Step 0 : FROM deis/slugrunner
# Executing 3 build triggers
Trigger 0, RUN mkdir -p /app
Step 0 : RUN mkdir -p /app
 ---> Using cache
Trigger 1, WORKDIR /app
Step 0 : WORKDIR /app
 ---> Using cache
Trigger 2, ADD slug.tgz /app
Step 0 : ADD slug.tgz /app
 ---> 3da1998b04a2
Removing intermediate container 3286b3de2ee2
Step 1 : ENV GIT_SHA 0f9caa12bf8315633b6ff366e638d29f349c8ee2
 ---> Running in 3eb69115c0d9
 ---> 475d97558564
Removing intermediate container 3eb69115c0d9
Successfully built 475d97558564
-----> Pushing image to private registry
-----> Launching...
       done, sample:v2 deployed to Deis
       http://sample.mydomain.com
       To learn more, use `deis help` or visit http://deis.io
To ssh://git@deis.mydomain.com:2222/sample.git
 * [new branch]      master -> master

A small disclaimer: Parts of the output have been omitted to save up some space. Also, note that your output might be different. Nevertheless, let's analyze the output we received from the deploy.

First, we can see that we pushed our code to a git repository. Then, Deis recognized our application as a Rails app and used a buildpack for Ruby on Rails applications. It created our bundle by fetching the required gems for our app from RubyGems (the output is omitted to save space).

After it completes the bundle, it changes the DATABASE_URL variable in the config/database.yml file. It basically expects our application to have a database set up. If you haven't noticed, we still haven't mentioned anything about a database. We will look into this in a moment.

The next step is booting the server. Since the sample application does not have a Procfile, it boots our application using Webrick by default. If we wanted to boot our application using a different server, we could have added a Procfile with the commands needed to boot our app on that particular server.

This is where it gets interesting. In the next step, Deis containerizes our application with Docker. The Docker image that it creates is based off of deis/slugrunner. Slugrunner takes a gzipped tarball of a "compiled" application via stdin or from a URL, letting you run a command in that application environment or start a process defined in the application Procfile.

If you take a look at the logs above, you'll notice that slugrunner first creates the /app path and changes the working directory to it. Then it adds the slug.tgz file to /app. With each step, it creates an intermediate container, which it removes at the end. When the container build is finished, slugrunner boots up the application.

At the end of the output, we get an informative message with the URL where the application resides, with some other informative links in addition. But if we point our browser to the URL, we'll get an error. Why, what's going on?

With the last step, we deployed the application. Although the deploy went smooth, the application is still not working. How can we debug this?

Just like with Heroku, viewing the logs on Deis is really easy. In the application directory, you can run: deis logs -n 100

This will send the last 100 lines of the logs of the application to STDOUT. In the logs, you'll notice the following lines:

sample[web.1]: => Booting WEBrick
sample[web.1]: => Rails 4.0.8 application starting in production on http://0.0.0.0:5000
sample[web.1]: => Run `rails server -h` for more startup options
sample[web.1]: => Ctrl-C to shutdown server
sample[web.1]: Exiting
sample[web.1]: (erb):9:in `rescue in <main>': Invalid DATABASE_URL (RuntimeError) 

This tells us that Deis tried to boot the application, but it's missing the DATABASE_URL environment variable which crashes the application. This means that we just need to add an additional configuration to our Deis application. Adding a new environment variable is done via:

deis config:set ENV_VAR_NAME=value

Or, in our case, we need to run:

deis config:set DATABASE_URL="some-url-here"

But since we don't have a database running, we don't know the database url. Let's spin up a database instance for this application.

There's a slight problem with this. At the moment of writing this tutorial, Deis does not have an addon/plugin system like Heroku's Add-ons. This means that we can use any "database as a service" provider. Another approach is using the official Postgres Docker image and deploying it to a Amazon EC2 instance. There's a plethora of options, but for this blog post we will take a very simple (or lazy, if you will) approach.

We will use Heroku's Postgres platform. The Postgres platform allows us to create a Postgres database with literally two clicks.

When a new database is created, we can grab the database URL from the settings page and add it as an environment variable to our Deis-hosted application:

deis config:set DATABASE_URL="postgres://username:password@ec2.compute-1.amazonaws.com:some-port/dbname"

The whole flow looks just like adding the Postgres plugin to a Heroku application. After the environment variable has been set, you can simply run:

deis run "rake db:migrate"

This will execute the migrations, and our application is ready to be used! Head over to http://sample.<your-domain-here>.com to see the application deployed to your Deis cluster.

Another option supported by Deis is deployment via a Dockerfile. This is a powerful way to define a portable execution environment built on a base OS of your choosing.

A Dockerfile automates the steps for the creation of a Docker image. However, Dockerfiles must conform to some Deis rules:

• The Dockerfile must EXPOSE only one port.

• The port must be listening for an HTTP connection.

• A default CMD must be specified for running the container.

Since our sample application does not have a Dockerfile that we can use, we need to create one. If you do not know how to build a Dockerfile for your Rails application, check out these two posts:

• Running a Rails Development Environment in Docker

• Deploying Your Docker Rails App

For this Rails application we will use the following Dockerfile:

FROM ruby:2.0
# Install apt based dependencies required to run Rails as
# well as RubyGems. As the Ruby image itself is based on a
# Debian image, we use apt-get to install those.
RUN apt-get update && apt-get install -y \
  build-essential \
  nodejs
# Configure the main working directory. This is the base
# directory used in any further RUN, COPY, and ENTRYPOINT
# commands.
RUN mkdir -p /app
WORKDIR /app
# Copy the Gemfile as well as the Gemfile.lock and install
# the RubyGems. This is a separate step so the dependencies
# will be cached unless changes to one of those two files
# are made.
COPY Gemfile Gemfile.lock ./
RUN gem install bundler && bundle install --without development test --jobs 20 --retry 5
# Set environment to production
ENV RAILS_ENV production
ENV RACK_ENV production
# Copy the main application.
COPY . ./
# Copy config/database.yml.prod to config/database.yml
COPY config/database.yml.prod config/database.yml
# Precompile Rails assets
RUN bundle exec rake assets:precompile
# Expose port 3000 to the Docker host, so we can access it
# from the outside.
EXPOSE 3000
# The main command to run when the container starts. Also
# tell the Rails dev server to bind to all interfaces by
# default.
ENTRYPOINT ["bundle", "exec", "rails", "server", "-e", "production"]

The Dockerfile is quite simple; the base image is ruby:2.0. Next, we install the build-essential package, which is a reference for all the packages needed to compile a Debian package. Then, we install a JavaScript runtime: nodejs.

After that, the Dockerfile will create the /app directory which will be the home of our application and will change the working directory to it. Then the Gemfile and the Gemfile.lock files are copied, and the bundle is installed. The whole source code is copied to the image, the 3000 port is exposed, and the Rails server is booted.

If we commit our Dockerfile to the repository and push it to Deis, you'll notice that Deis will pick up the Dockerfile, and it will not use the Rails buildpack to deploy the application. Instead, Deis will use all of the instructions from the Dockerfile.

This is a neat feature of Deis, because Dockerfiles provide more flexibility to the deployment itself. As you can see, you can do pretty much anything to the base operating system, by changing configurations, exposing ports, installing additional packages, and running any command.

A third option for deploying Rails applications in Deis is using a Docker image. As the documentation states, this is useful for integrating Deis into Docker-based CI/CD pipelines. This means that you can build the image locally and ship the image wherever you want. Deis can pull the image from both public and private registries.

Let's build an image using the Dockerfile from the last step. The command to build an image from a Dockerfile is:

docker build -t <your-username>/<image-name> .

In our case, we can run it as:

docker build -t <your-username>/sample-rails-application .

When Docker finishes building the image, we can push it to DockerHub or a private registry:

docker push <your-username>/sample-rails-application

Since the hard work with the Dockerfile is already done and the image is pushed to DockerHub, we can just ask Deis to pull the image from the registry and deploy it:

deis pull <your-username>/sample-rails-application:latest

Deis makes configuring your applications quite easy. One way that I already mentioned is to set environment variables.

From the command line, you can see all the available commands for modifying environment variables:

deis help config
Valid commands for config:
config:list        list environment variables for an app
config:set         set environment variables for an app
config:unset       unset environment variables for an app
config:pull        extract environment variables to .env
config:push        set environment variables from .env
Use 'deis help [command]' to learn more.

When we were attaching the database as a backing service, we used the following command:

deis config:set DATABASE_URL="some-url-here"

As you can see above, Deis even allows you to download all of the environment variables in a .env file, so you can start the application locally with the same configuration.

Using a custom domain for a Deis app is very easy. You can use the deis domains command to configure custom domains for an application. Adding a custom domain is done via:

deis domains:add new-domain.com

Next, you need to head over to your DNS registrar and set up a CNAME for the new domain to the old one. For example, if the old subdomain for the application was sample.your-domain.com, you need make new-domain.com be a CNAME to sample.your-domain.com.

Deis is a super powerful platform, and it's super configurable. You can do backups and data restores, configure the load balancers, view the logs, monitor the platform with external tools, add SSL to the platform, and so much more. If you want to dig deeper in Deis and see all its customizability and power, I recommend reading the very detailed and helpful documentation.