Getting Started with Minitest

Minitest is a beautifully made test suite for writing tests to verify your expectations from your code base. According to the website, it "provides a complete suite of testing facilities supporting TDD, BDD, mocking, and benchmarking." The way it's implemented is rather straightforward, as you can see in creator Ryan Davis' walk-through of Minitest's design/implementation.

Fundamentally, it has been built from the ground up with small, focused steps in design. As each step was completed, the code was then evaluated and refactored to provide a separation between different concerns. This has resulted in Minitest being both easy to understand and to build upon, enabling you to write your own plugins or reporters.

To run a Minitest test, the only setup you really need is to require the autorun file at the beginning of a test file: require 'minitest/autorun'. This is good if you'd like to keep the code small. A better way to get started with Minitest is to have Bundler create a template project for you.

bundle gem myproject

There are three files generated that are specifically for your test suite:

Rakefile test/test_helper.rb test/myproject_test.rb

The Rakefile maps what files should be included when you run your tests via rake. Update the three lines in the gemspec file myproject.gemspec that have TODO written in them (you may delete the homepage line if you wish), and this will allow you to run your tests. You can simply run rake on the command line or rake test.

The test/test_helper.rb is the file where you initialize all the Minitest specific configurations, plugins, and reporters. This file is to be included at the top of each test file in the test directory with require 'test_helper'.

The myproject_test.rb file is an example test file with one passing test and one failing test. Any new test files you create need to have _test at the end of the file name, e.g. stuff_test.rb. This is what the Rakefile is mapped to load. Let's discuss the test suite execution path.

Rake::TestTask.new(:test) do |t|
  t.libs << "test"
  t.libs << "lib"
  t.test_files = FileList['test/**/*_test.rb']
end
task :default => :test

Here the Rakefile creates a task :test, which is where all the file inclusions are defined for that task. This is how it's selected when you run the command rake test. The default setting at the bottom of the Rakefile allows you to simply type rake to run the tests.

As I mentioned earlier, the only thing you need to run a Minitest test is to include the autorun file. This is why it's important to include the test_helper.rb in every test file; it's where autorun is included from.

The autorun file sets up an at_exit hook that will run all the tests after everything is loaded and done. Each test block gets registered in a list that will be iterated over with the at_exit hook. This happens for both the assert-style testing with the inherited Minitest::Test class and the spec-style testing with the describe block.

Assert-style tests are the most basic kind of test. They are the primary choice for most major Ruby projects online. Looking at the test file generated from Bundler, we have:

require 'test_helper'
class MyprojectTest < Minitest::Test
  def test_that_it_has_a_version_number
    refute_nil ::Myproject::VERSION
  end
  def test_it_does_something_useful
    assert false # this will result in a failure
  end
end

This is the basic outline: You'll have a class inherit from Minitest::Test and then define all your tests on methods that start with test_. In each test, you need to either use a form of assert or refute for any code you're testing.

There are quite a few different methods to choose from as you can see in the documentation. The assertion used most often tends to be assert_equal, which takes two values to compare and an optional failure message.

Beyond these assertions, you may also use setup and teardown methods. Both of these methods will run with every test method called; setup runs before and teardown after. If all of this is foreign to you, it's important to remember that it's just Ruby. You can set up objects and variables to use here as you would anywhere else. With this, it's simplest and best to just experiment with it.

Spec style is the test style I prefer. It's slightly more readable and looks less like code.

require 'test_helper'
describe "My Project" do
  it "has a version number" do
    value(::Myproject::VERSION).wont_be_nil
  end
  it "does something useful" do
    value(4 + 4).must_equal 8
  end
end

In addition to assert methods being available, including setup/teardown, there are spec versions of them: before and after. One additional method you can use for spec style tests is let, a lazy named initializer (it remains unevaluated until called).

let(:my_number) { 4 + 4 }
it "does something useful" do
  value( my_number ).must_equal 8
end

As of Minitest 5.6.0, expectations (spec assertions) are moving into value monads. (Note that the older style will still work as of this writing). Until now, all spec-style methods have been included directly into the Object class, meaning everything has methods like must_equal and wont_be_nil on it. At some point in the future, these will be removed and be explicitly available on value monads. value and expect are aliases for the same method which is written for underscore _.

Here's an excerpt from the code documentation:

#         _(1 + 1).must_equal 2
#     value(1 + 1).must_equal 2
#    expect(1 + 1).must_equal 2
#
# This method of expectation-based testing is preferable to
# straight-expectation methods (on Object) because it stores its
# test context, bypassing our hacky use of thread-local variables.

For other expectation methods, see the documentation.

To get started with benchmarking your code, let's add an entry for benchmarks in your Rakefile.

Rake::TestTask.new(:bench) do |t|
  t.libs = %w(lib test)
  t.pattern = 'test/**/*_benchmark.rb'
end

Any file ending in _benchmark in your test folder or subdirectory will run when you execute rake bench. Now basic assert-style benchmarks can look as follows.

require "test_helper"
require "minitest/benchmark" if ENV["BENCH"]
class MyprojectBenchmark < Minitest::Benchmark
  def bench_my_linear
    assert_performance_linear do |n|
      n.times do
        n*n
      end
    end
  end
  def bench_my_constant
    assert_performance_constant do |n|
      n*n
    end
  end
end

Each benchmark method name needs to begin with bench_. The two benchmark tests go through the default range of [1, 10, 100, 1_000, 10_000] and assert that the method performance is linear in the first case and consistent in the second one. For spec style, you can replace the methods prefix of assert_ with bench_ and drop the def line before it like this:

describe "My Project" do
  if ENV["BENCH"] then
    bench_performance_linear "my_linear" do |n|
      n.times do
        n*n
      end
    end
  end
end

Reporters process a test's results and dictate the output details and style. Obviously, having easy-to-read output can really help improve the process of TDD. To add your own custom reporters, you'll need to include the minitest-reporters gem in your Gemfile or gemspec file. You can test one of the included reporters by adding this to the end of your test_helper.rb file:

require 'minitest/reporters'
Minitest::Reporters.use! [Minitest::Reporters::SpecReporter.new]

This is just one of many styles available for reporters. For example, Aaron Patterson has created an emoji reporter minitest-emoji, and I've created a reporter called color_pound_spec_reporter which has customized the spec-style reporter for a much cleaner layout and topic coloring.

When you start working in frameworks such as Rails, there are plugins available that add additional methods and DSLs to accomplish a higher domain of testing. For Rails, there is minitest-rails and capybara.

When using these, it's best to get familiar with their DSL and be cautious should you choose to intermix them. They're each designed to work well on their own, but you may come across some difficulties mixing test contexts across DSLs.

Minitest boasts that it's smaller and simpler than rSpec. It's the chosen test framework for both the Ruby language and Rails. That doesn't mean it's for everyone though. Many people are fond of the test suites they're most familiar with, and rSpec has a strong community around it. I'm not going to debate whether one is better than the other; they each have their own advantages.

What's important when testing is to practice good coding habits. Clean, readable, and understandable code with continual learning and integration of best practices should be every developer's daily discipline.

One method for ingraining new techniques I would recommend is code katas. See Jim Weirich's TDD video on Roman Numeral Kata for a superb example.

Testing is a particular area of interest and concern when it comes to following best practice in coding. Many people aren't testing simply because they haven't dared to dive in. The truth of the matter is that testing is as simple as you make it.

Often testing will become complicated when you try to be clever. Being clever should be considered a code smell. It can mean that you're adding some high level of complexity for some extra convenience that, if you're honest with yourself, you probably don't need. writing tests is just as much Ruby development as the main part of your code. Any unnecessary complexity is just going to add overhead for you and others.

For more help with writing tests with Minitest, I highly recommend The Minitest Cheat Sheet by Chris Kottom. It includes both assert-style and spec-style method references with descriptions and examples. If you're interested in digging even deeper into Minitest testing, you really must check out his book The Minitest Cookbook.