[LRUG] Large Slow Test Suites

[Mr Jaba]

Hi Jon,

That's all very useful stuff, thanks very much. Interesting to hear people
are going from Fixtures, to Factories and back to Fixtures again! Factories
are definitely slow, but I guess as long as your fixtures are managed
carefully they're not exactly as evil as they've been made out to be.

Interesting points on making your unit tests independent of Rails, I hadn't
considered the class reloading aspect. Does this interfere very badly? Is
it avoidable at all? Or was the cost of maintaining all the requires simply
too high?

Thanks again

Tom


On 5 December 2013 09:04, Jon Leighton <j at jonathanleighton.com> wrote:

> Hi Tom,
>
> On 04/12/13 22:56, Mr Jaba wrote:
> > First up, thanks for Spring (amongst your many other contributions!)
> > it's a really useful gem!
>
> Glad you like it!
>
> > With regards to the factories, what do you recommend instead? Simpler
> > objects with less coupling to Rails?
>
> Yeah I don't like the factories approach at all. Apart from often being
> a source of test slowness, they encourage you down the path of high
> coupling and make it too trivial to create a boat load of
> database-backed objects. Bo Jeanes wrote a post about this:
> http://bjeanes.com/2012/02/factories-breed-complexity
>
> There are certainly situations in a Rails app where you need to test
> against the database. For this I use fixtures, which I find perfectly
> adequate and much faster than factories. You do need to ensure your
> fixtures don't get out of control, but generally I think of fixtures as
> a handful of pre-inserted records that I might modify in specific tests
> in order to achieve a certain thing (rather than creating a new fixture
> entry for each thing, which can get out of hand).
>
> Apart from that, it's obviously wise to avoid inserting stuff in the
> database as much as possible, e.g. if you just want to test the logic of
> a certain method which only interacts with a model's attributes, you can
> just create an in-memory instance.
>
> I have been down the path of making my "unit tests" independent of the
> Rails app so as to avoid the boot process, and found it impractical to
> be honest:
>
> 1) You end up with require and require_dependency everywhere, and it's
> hard to get this right in a way that plays nicely with Rails' class
> reloading
> 2) Even if you make some tests independent of Rails, you're still going
> to need some that are not independent of Rails, so you only solve part
> of the problem
>
> This was part of my motivation for spring - if we can't make get rid of
> having to boot the Rails app, let's make booting the Rails app less
> painful.
>
> Not sure any of these comments help with your current predicament, but
> hopefully they're vaguely useful anyway!
>
> Jon
>
> >
> > Thanks for your input!
> >
> > Tom
> >
> >
> > On 4 December 2013 22:43, Jon Leighton <j at jonathanleighton.com
> > <mailto:j at jonathanleighton.com>> wrote:
> >
> >     Hello there,
> >
> >     On 04/12/13 22:20, Mr Jaba wrote:
> >     > I've recently taken ownership of a new project with a large test
> suite
> >     > (2000ish tests), and the overall run time is around 30 minutes
> >     which is
> >     > certainly less than ideal!
> >     >
> >     > Now I know the general approach to "Fast Rails Tests" but taking
> the
> >     > time to refactor the whole test suite is a bit too much right now.
> I'm
> >     > wondering if anyone has experience of transforming a test suite of
> >     this
> >     > magnitude to something a bit speedier? If so how did you go about
> it?
> >     > What tips, tricks and techniques can you share?
> >     >
> >     > A bit more info:
> >     >
> >     > - Test::Unit tests, moving to Minitest
> >     > - Rails 3.2.16
> >     > - Running parallel_tests shaved 5 mins off the time
> >     > - Using Spring to reduce Rails load time.
> >     >
> >     > Any advice gratefully received!
> >
> >     I think you need to provide some more information :) What's making it
> >     take so long - the sheer number of tests or what the tests are
> doing? Do
> >     you have certain types of tests that are taking a large portion of
> the
> >     time (e.g. Capybara tests?) Are you using factories? In my experience
> >     factories are an easy cause of a slow test suite.
> >
> >     One thing to think about is whether you can break up your test suite.
> >     For example in the application I work on, we have unit/integration
> tests
> >     which run in about 25 seconds on my machine. I frequently run these
> >     during development and they often alert me to problems. But I rarely
> run
> >     the full set of capybara tests which takes several minutes - I let
> the
> >     CI do that and run failing tests individually where necessary. This
> >     achieves a decent balance of fast feedback for development without
> >     throwing away the safety net of proper full-stack tests.
> >
> >     Jon
> >     _______________________________________________
> >     Chat mailing list
> >     Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>
> >     http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20131205/b5a45713/attachment-0003.html>