Unit Testing in R: The Bare Minimum

Introduction

This week I decided to start unit testing my R code, so I taught myself the bare minimum about the RUnit and testthat packages to be able to use them. Here’s what I found necessary to get started writing tests with both packages.

RUnit Basic Example

I’m going to assume that you’ve got a bunch of functions in sample.R that you want to test. For example, sample.R might contain a definition of the na├»ve factorial function:

1
2
3
4
5
6
7
8
9
10
11
factorial <- function(n)
{
  if (n == 0)
  {
    return(1)
  }
  else
  {
    return(n * factorial(n - 1))
  }
}

To test your functions, create a directory called tests that will store all of your test cases. In tests, create a file called 1.R that will contain your first set of tests. Each set of tests will go in a function inside of 1.R, named according to the convention test.*. For example, you might have this:

1
2
3
4
5
6
7
8
9
10
11
12
13
test.examples <- function()
{
  checkEquals(6, factorial(3))
  checkEqualsNumeric(6, factorial(3))
  checkIdentical(6, factorial(3))
  checkTrue(2 + 2 == 4, 'Arithmetic works')
  checkException(log('a'), 'Unable to take the log() of a string')
}
 
test.deactivation <- function()
{
  DEACTIVATED('Deactivating this test function')
}

To run this set of tests, we need to create a file called run_tests.R that will act as a test suite and invoke all of the tests in your tests directory:

1
2
3
4
5
6
7
8
9
10
11
library('RUnit')
 
source('sample.R')
 
test.suite <- defineTestSuite("example",
                              dirs = file.path("tests"),
                              testFileRegexp = '^\\d+\\.R')
 
test.result <- runTestSuite(test.suite)
 
printTextProtocol(test.result)

Here you inform the defineTestSuite() function that you’re creating a test suite called “example” and that the test files are located in a directory called “tests” where all of the files match the regular expression ‘^\\d+\\.R’. Then you run the suite explicitly and print out the results in a text format.

That’s it. With those ideas, you can write your own test suite for your R code.

Using the check*() Functions

In general, with RUnit, you use a function named something like check* to test the following conditions:

  • checkEquals: Are two objects equal, including named attributes?
  • checkEqualsNumeric: Are two numeric values equal?
  • checkIdentical: Are two objects exactly the same?
  • checkTrue: Does an expression evaluate to TRUE?
  • checkException: Does an expression raise an error?

In addition to these functions, there’s also a DEACTIVATED() function that lets you turn off a test function during its execution if you need to do that.

testthat Basic Example

As above, I’m going to assume that you’ve got a bunch of functions in sample.R that you want to test. And, as before, to test your functions, you should create a directory called tests that will store all of your test cases. In tests, create a file called 1.R that will contain your first set of tests. We’ll use expect_that() for all of our tests, as in the example below:

1
2
3
4
5
6
7
8
9
10
11
12
13
expect_that(1 ^ 1, equals(1))
expect_that(2 ^ 2, equals(4))
 
expect_that(2 + 2 == 4, is_true())
expect_that(2 == 1, is_false())
 
expect_that(1, is_a('numeric'))
 
expect_that(print('Hello World!'), prints_text('Hello World!'))
 
expect_that(log('a'), throws_error())
 
expect_that(factorial(16), takes_less_than(1))

To run this set of tests, we need to create a file called run_tests.R that will act as a test suite and invoke all of the tests in your tests directory:

1
2
3
4
5
library('testthat')
 
source('sample.R')
 
test_dir('tests', reporter = 'Summary')

To get output, you have to inform test_dir() to use the SummaryReporter, which provides more than enough information for my purposes. See the testthat docs for other reporters you could use.

Using expect_that()

In general, you can ask expect_that() to test the following conditions:

  • is_true: Does the expression evaluate to TRUE?
  • is_false: Does the expression evaluate to FALSE?
  • is_a: Did the object inherit from a specified class?
  • equals: Is the expression equal within numerical tolerance to your expected value?
  • is_equivalent_to: Is the object equal up to attributes to your expected value?
  • is_identical_to: Is the object exactly equal to your expected value?
  • matches: Does a string match the specified regular expression?
  • prints_text: Does the text that’s printed match the specified regular expression?
  • throws_error: Does the expression raise an error?
  • takes_less_than: Does the expression take less than a specified number of seconds to run?

More testthat Tricks

There are some other tricks that testthat can do as well. It can automatically rerun tests on a directory of code whenever the code is edited using a function called auto_test() and it can set up contexts to separate tests using a function context(). I haven’t really explored either, so I can’t comment more on them.

Which to Use?

While I don’t think the arguments on behalf of either RUnit or testthat are unquestionable, I’m inclined to think that testthat has a brighter future, especially since it’s written by ggplot2’s author, Hadley Wickham.

9 responses to “Unit Testing in R: The Bare Minimum”

  1. Mike Dewar

    I would like

    expect_that(x, is_distributed_like(runif))

    or some other distribution. I guess one could cook this up, though…

  2. Jeromy Anglim

    Awesome post! I’ve been looking for a friendly introduction to unit testing in R for a while.

  3. jcborras

    Mike Dewar,
    expect_that() semantics are like assert_equals() methods found in traditional xUnit-inspired unit test frameworks. Therefore your example is a little overfetched IMHO.
    It makes more sense expect_that(distribution_distance(x, runif(…)) < k) where distribution_distance() has to be cooked somehow.

    I don't want to sound pedantic here but some aspects of R do not match well traditional unit testing approaches. Sure, it glues well when doing unit testing on deterministic algorithms but once you get into "MonteCarlo testing methods" you'll lose immediate testing feedback…

  4. Bernd

    Hi John,

    as always I enjoyed reading your post. However, I tried to run your little testthat example but to no avail. In my understanding your “expect_that” expressions should be surrounded by

    test_that(“test”, {
    expect_that(…)

    }
    )

    At least in its current version testthat expects that all “[t]est files start with test and are executed in alphabetical order”, i.e. valid file names are test_1.R, test_2.R etc (test_dir() contains the following regex “^test.*\\.[rR]$”).

  5. Vadim Vinichenko

    Thank you for the post, John! Can you please e-mail me an example of your testing code using testthat too? Like Bernd, I was unable to get “expect_that” calls working without wrapping them in “test_that” (was getting some errors tracing back to SummaryReporter$end_reporter() method). With “test_that” everything works fine.

  6. Idris

    Awesome! I just started unit testing Python with nose and was jonesing for an equivalent in R. I just don’t get that warm fuzzy feeling if I don’t have unit tests for my functions.

    Thanks!