[LRUG] Large Slow Test Suites

[Jon Leighton]

Hi Tom,

On 05/12/13 09:28, Mr Jaba wrote:
> Interesting points on making your unit tests independent of Rails, I
> hadn't considered the class reloading aspect. Does this interfere very
> badly? Is it avoidable at all? Or was the cost of maintaining all the
> requires simply too high? 

If you want to run your tests outside of Rails, you can't rely on the
rails autoloading mechanism (ActiveSupport::Dependencies). But if you
put requires all over the place, it will prevent AS::Dependencies from
working when you *do* want it to (i.e. code reloading in dev mode won't
work, because the code was loaded by a require rather than a missing
constant).

The solution I ended up with was basically using require_dependency
rather than require, and stubbing out require_dependency for my "outside
Rails" tests. It was nasty.

In might be possible to *just* use AS::Dependencies in your unit tests,
without loading the full Rails app. That's something I didn't try, but
with spring loading the full app is not such a big deal for me these
days anyway, so I don't see the benefit of having tests that can run
outside of the Rails app.

Jon

> 
> Thanks again
> 
> Tom
> 
> 
> On 5 December 2013 09:04, Jon Leighton <j at jonathanleighton.com
> <mailto:j at jonathanleighton.com>> wrote:
> 
>     Hi Tom,
> 
>     On 04/12/13 22:56, Mr Jaba wrote:
>     > First up, thanks for Spring (amongst your many other contributions!)
>     > it's a really useful gem!
> 
>     Glad you like it!
> 
>     > With regards to the factories, what do you recommend instead? Simpler
>     > objects with less coupling to Rails?
> 
>     Yeah I don't like the factories approach at all. Apart from often being
>     a source of test slowness, they encourage you down the path of high
>     coupling and make it too trivial to create a boat load of
>     database-backed objects. Bo Jeanes wrote a post about this:
>     http://bjeanes.com/2012/02/factories-breed-complexity
> 
>     There are certainly situations in a Rails app where you need to test
>     against the database. For this I use fixtures, which I find perfectly
>     adequate and much faster than factories. You do need to ensure your
>     fixtures don't get out of control, but generally I think of fixtures as
>     a handful of pre-inserted records that I might modify in specific tests
>     in order to achieve a certain thing (rather than creating a new fixture
>     entry for each thing, which can get out of hand).
> 
>     Apart from that, it's obviously wise to avoid inserting stuff in the
>     database as much as possible, e.g. if you just want to test the logic of
>     a certain method which only interacts with a model's attributes, you can
>     just create an in-memory instance.
> 
>     I have been down the path of making my "unit tests" independent of the
>     Rails app so as to avoid the boot process, and found it impractical to
>     be honest:
> 
>     1) You end up with require and require_dependency everywhere, and it's
>     hard to get this right in a way that plays nicely with Rails' class
>     reloading
>     2) Even if you make some tests independent of Rails, you're still going
>     to need some that are not independent of Rails, so you only solve part
>     of the problem
> 
>     This was part of my motivation for spring - if we can't make get rid of
>     having to boot the Rails app, let's make booting the Rails app less
>     painful.
> 
>     Not sure any of these comments help with your current predicament, but
>     hopefully they're vaguely useful anyway!
> 
>     Jon
> 
>     >
>     > Thanks for your input!
>     >
>     > Tom
>     >
>     >
>     > On 4 December 2013 22:43, Jon Leighton <j at jonathanleighton.com
>     <mailto:j at jonathanleighton.com>
>     > <mailto:j at jonathanleighton.com <mailto:j at jonathanleighton.com>>>
>     wrote:
>     >
>     >     Hello there,
>     >
>     >     On 04/12/13 22:20, Mr Jaba wrote:
>     >     > I've recently taken ownership of a new project with a large
>     test suite
>     >     > (2000ish tests), and the overall run time is around 30 minutes
>     >     which is
>     >     > certainly less than ideal!
>     >     >
>     >     > Now I know the general approach to "Fast Rails Tests" but
>     taking the
>     >     > time to refactor the whole test suite is a bit too much
>     right now. I'm
>     >     > wondering if anyone has experience of transforming a test
>     suite of
>     >     this
>     >     > magnitude to something a bit speedier? If so how did you go
>     about it?
>     >     > What tips, tricks and techniques can you share?
>     >     >
>     >     > A bit more info:
>     >     >
>     >     > - Test::Unit tests, moving to Minitest
>     >     > - Rails 3.2.16
>     >     > - Running parallel_tests shaved 5 mins off the time
>     >     > - Using Spring to reduce Rails load time.
>     >     >
>     >     > Any advice gratefully received!
>     >
>     >     I think you need to provide some more information :) What's
>     making it
>     >     take so long - the sheer number of tests or what the tests are
>     doing? Do
>     >     you have certain types of tests that are taking a large
>     portion of the
>     >     time (e.g. Capybara tests?) Are you using factories? In my
>     experience
>     >     factories are an easy cause of a slow test suite.
>     >
>     >     One thing to think about is whether you can break up your test
>     suite.
>     >     For example in the application I work on, we have
>     unit/integration tests
>     >     which run in about 25 seconds on my machine. I frequently run
>     these
>     >     during development and they often alert me to problems. But I
>     rarely run
>     >     the full set of capybara tests which takes several minutes - I
>     let the
>     >     CI do that and run failing tests individually where necessary.
>     This
>     >     achieves a decent balance of fast feedback for development without
>     >     throwing away the safety net of proper full-stack tests.
>     >
>     >     Jon
>     >     _______________________________________________
>     >     Chat mailing list
>     >     Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>
>     <mailto:Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>>
>     >     http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>     >
>     >
> 
>