[LRUG] Large Slow Test Suites

[Jon Leighton]

Hi Tom,

On 04/12/13 22:56, Mr Jaba wrote:
> First up, thanks for Spring (amongst your many other contributions!)
> it's a really useful gem!

Glad you like it!

> With regards to the factories, what do you recommend instead? Simpler
> objects with less coupling to Rails? 

Yeah I don't like the factories approach at all. Apart from often being
a source of test slowness, they encourage you down the path of high
coupling and make it too trivial to create a boat load of
database-backed objects. Bo Jeanes wrote a post about this:
http://bjeanes.com/2012/02/factories-breed-complexity

There are certainly situations in a Rails app where you need to test
against the database. For this I use fixtures, which I find perfectly
adequate and much faster than factories. You do need to ensure your
fixtures don't get out of control, but generally I think of fixtures as
a handful of pre-inserted records that I might modify in specific tests
in order to achieve a certain thing (rather than creating a new fixture
entry for each thing, which can get out of hand).

Apart from that, it's obviously wise to avoid inserting stuff in the
database as much as possible, e.g. if you just want to test the logic of
a certain method which only interacts with a model's attributes, you can
just create an in-memory instance.

I have been down the path of making my "unit tests" independent of the
Rails app so as to avoid the boot process, and found it impractical to
be honest:

1) You end up with require and require_dependency everywhere, and it's
hard to get this right in a way that plays nicely with Rails' class
reloading
2) Even if you make some tests independent of Rails, you're still going
to need some that are not independent of Rails, so you only solve part
of the problem

This was part of my motivation for spring - if we can't make get rid of
having to boot the Rails app, let's make booting the Rails app less painful.

Not sure any of these comments help with your current predicament, but
hopefully they're vaguely useful anyway!

Jon

> 
> Thanks for your input!
> 
> Tom
> 
> 
> On 4 December 2013 22:43, Jon Leighton <j at jonathanleighton.com
> <mailto:j at jonathanleighton.com>> wrote:
> 
>     Hello there,
> 
>     On 04/12/13 22:20, Mr Jaba wrote:
>     > I've recently taken ownership of a new project with a large test suite
>     > (2000ish tests), and the overall run time is around 30 minutes
>     which is
>     > certainly less than ideal!
>     >
>     > Now I know the general approach to "Fast Rails Tests" but taking the
>     > time to refactor the whole test suite is a bit too much right now. I'm
>     > wondering if anyone has experience of transforming a test suite of
>     this
>     > magnitude to something a bit speedier? If so how did you go about it?
>     > What tips, tricks and techniques can you share?
>     >
>     > A bit more info:
>     >
>     > - Test::Unit tests, moving to Minitest
>     > - Rails 3.2.16
>     > - Running parallel_tests shaved 5 mins off the time
>     > - Using Spring to reduce Rails load time.
>     >
>     > Any advice gratefully received!
> 
>     I think you need to provide some more information :) What's making it
>     take so long - the sheer number of tests or what the tests are doing? Do
>     you have certain types of tests that are taking a large portion of the
>     time (e.g. Capybara tests?) Are you using factories? In my experience
>     factories are an easy cause of a slow test suite.
> 
>     One thing to think about is whether you can break up your test suite.
>     For example in the application I work on, we have unit/integration tests
>     which run in about 25 seconds on my machine. I frequently run these
>     during development and they often alert me to problems. But I rarely run
>     the full set of capybara tests which takes several minutes - I let the
>     CI do that and run failing tests individually where necessary. This
>     achieves a decent balance of fast feedback for development without
>     throwing away the safety net of proper full-stack tests.
> 
>     Jon
>     _______________________________________________
>     Chat mailing list
>     Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>
>     http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> 
>