[LRUG] Number of tests over time?

[Graham Ashton]

On Wednesday 25 March, Tim Cowlishaw wrote:

> (1) How long has your project been in development?

3 years (it's Agile Planner)

> (2) How many people work on it?

1

> (3) How many specs / unit tests do you have, and
> (4) How many integration / acceptance tests or cukes do you have?

    $ for folder in $(find test -type d -mindepth 1 -maxdepth 1); do
    >   count=$(egrep -r '^\s+it\b.*(do|->)\s*$' $folder | wc -l)
    >   printf "%16s: %5d\n" $folder $count
    > done
    test/controllers:   306
        test/helpers:    20
    test/integration:    51
    test/javascripts:    23
        test/mailers:    26
         test/models:   312

It's a Rails app, tested with test/unit-style assertions, organised with describe/it blocks.

I'm surprised I've got that many helper tests!

I consider the tests in test/controllers to be integration tests (they test integration of model, view and controller). Mine always access the database and check the HTML or JSON rendered, rather than whether or not an instance variable has been set -- are people still following that style?

The tests in test/integration are written with capybara, and serve two purposes.

1. If I feel I need an acceptance test (i.e. does the app work the way a user might expect?) they'll go in there.

2. I also make sure that important JavaScript functionality is executing without errors in the context of the HTML that the app actually produced here (i.e. with a headless browser). Specific behaviour of JavaScript code is generally only tested with unit tests though (via Mocha, Chai and Sinon in test/javascripts).

The whole lot takes around 90 seconds to run on my fastest machine, with a fairly even split between models, controllers and integration.

I'm not doing much mocking (in Ruby; there's more mocking going on with my JavaScript code). I've got a lot of small models (not AR instances; mostly just simple classes), which has helped keep the tests fast. It's helped me avoid significant reliance on mocks too.

I also follow an approach of "test what's likely to break", rather than aiming to test everything. I like to think of each test as something that should save (or make) my company/client some money. It's a judgement call, to be sure, and one that involves lots of factors. The question of what to test is simplified for me because I've been the only developer for the lifetime of the project (so far).