[LRUG] Large Slow Test Suites

[Sam Livingston-Gray]

I just put a require statement at the top of the fast test file for what I'm testing, and another one for anything it depends on. When that gets to be annoying, I know I have too many dependencies. ;)

--
(Sent from phone. Please excuse: brevity, top posting, hilarious autocorrections.)

> On Dec 5, 2013, at 7:29 AM, Jon Leighton <j at jonathanleighton.com> wrote:
> 
> Hi Tom,
> 
>> On 05/12/13 09:28, Mr Jaba wrote:
>> Interesting points on making your unit tests independent of Rails, I
>> hadn't considered the class reloading aspect. Does this interfere very
>> badly? Is it avoidable at all? Or was the cost of maintaining all the
>> requires simply too high?
> 
> If you want to run your tests outside of Rails, you can't rely on the
> rails autoloading mechanism (ActiveSupport::Dependencies). But if you
> put requires all over the place, it will prevent AS::Dependencies from
> working when you *do* want it to (i.e. code reloading in dev mode won't
> work, because the code was loaded by a require rather than a missing
> constant).
> 
> The solution I ended up with was basically using require_dependency
> rather than require, and stubbing out require_dependency for my "outside
> Rails" tests. It was nasty.
> 
> In might be possible to *just* use AS::Dependencies in your unit tests,
> without loading the full Rails app. That's something I didn't try, but
> with spring loading the full app is not such a big deal for me these
> days anyway, so I don't see the benefit of having tests that can run
> outside of the Rails app.
> 
> Jon
> 
>> 
>> Thanks again
>> 
>> Tom
>> 
>> 
>> On 5 December 2013 09:04, Jon Leighton <j at jonathanleighton.com
>> <mailto:j at jonathanleighton.com>> wrote:
>> 
>>    Hi Tom,
>> 
>>>    On 04/12/13 22:56, Mr Jaba wrote:
>>> First up, thanks for Spring (amongst your many other contributions!)
>>> it's a really useful gem!
>> 
>>    Glad you like it!
>> 
>>> With regards to the factories, what do you recommend instead? Simpler
>>> objects with less coupling to Rails?
>> 
>>    Yeah I don't like the factories approach at all. Apart from often being
>>    a source of test slowness, they encourage you down the path of high
>>    coupling and make it too trivial to create a boat load of
>>    database-backed objects. Bo Jeanes wrote a post about this:
>>    http://bjeanes.com/2012/02/factories-breed-complexity
>> 
>>    There are certainly situations in a Rails app where you need to test
>>    against the database. For this I use fixtures, which I find perfectly
>>    adequate and much faster than factories. You do need to ensure your
>>    fixtures don't get out of control, but generally I think of fixtures as
>>    a handful of pre-inserted records that I might modify in specific tests
>>    in order to achieve a certain thing (rather than creating a new fixture
>>    entry for each thing, which can get out of hand).
>> 
>>    Apart from that, it's obviously wise to avoid inserting stuff in the
>>    database as much as possible, e.g. if you just want to test the logic of
>>    a certain method which only interacts with a model's attributes, you can
>>    just create an in-memory instance.
>> 
>>    I have been down the path of making my "unit tests" independent of the
>>    Rails app so as to avoid the boot process, and found it impractical to
>>    be honest:
>> 
>>    1) You end up with require and require_dependency everywhere, and it's
>>    hard to get this right in a way that plays nicely with Rails' class
>>    reloading
>>    2) Even if you make some tests independent of Rails, you're still going
>>    to need some that are not independent of Rails, so you only solve part
>>    of the problem
>> 
>>    This was part of my motivation for spring - if we can't make get rid of
>>    having to boot the Rails app, let's make booting the Rails app less
>>    painful.
>> 
>>    Not sure any of these comments help with your current predicament, but
>>    hopefully they're vaguely useful anyway!
>> 
>>    Jon
>> 
>>> 
>>> Thanks for your input!
>>> 
>>> Tom
>>> 
>>> 
>>> On 4 December 2013 22:43, Jon Leighton <j at jonathanleighton.com
>>    <mailto:j at jonathanleighton.com>
>>> <mailto:j at jonathanleighton.com <mailto:j at jonathanleighton.com>>>
>>    wrote:
>>> 
>>>    Hello there,
>>> 
>>>>    On 04/12/13 22:20, Mr Jaba wrote:
>>>> I've recently taken ownership of a new project with a large
>>    test suite
>>>> (2000ish tests), and the overall run time is around 30 minutes
>>>    which is
>>>> certainly less than ideal!
>>>> 
>>>> Now I know the general approach to "Fast Rails Tests" but
>>    taking the
>>>> time to refactor the whole test suite is a bit too much
>>    right now. I'm
>>>> wondering if anyone has experience of transforming a test
>>    suite of
>>>    this
>>>> magnitude to something a bit speedier? If so how did you go
>>    about it?
>>>> What tips, tricks and techniques can you share?
>>>> 
>>>> A bit more info:
>>>> 
>>>> - Test::Unit tests, moving to Minitest
>>>> - Rails 3.2.16
>>>> - Running parallel_tests shaved 5 mins off the time
>>>> - Using Spring to reduce Rails load time.
>>>> 
>>>> Any advice gratefully received!
>>> 
>>>    I think you need to provide some more information :) What's
>>    making it
>>>    take so long - the sheer number of tests or what the tests are
>>    doing? Do
>>>    you have certain types of tests that are taking a large
>>    portion of the
>>>    time (e.g. Capybara tests?) Are you using factories? In my
>>    experience
>>>    factories are an easy cause of a slow test suite.
>>> 
>>>    One thing to think about is whether you can break up your test
>>    suite.
>>>    For example in the application I work on, we have
>>    unit/integration tests
>>>    which run in about 25 seconds on my machine. I frequently run
>>    these
>>>    during development and they often alert me to problems. But I
>>    rarely run
>>>    the full set of capybara tests which takes several minutes - I
>>    let the
>>>    CI do that and run failing tests individually where necessary.
>>    This
>>>    achieves a decent balance of fast feedback for development without
>>>    throwing away the safety net of proper full-stack tests.
>>> 
>>>    Jon
>>>    _______________________________________________
>>>    Chat mailing list
>>>    Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>
>>    <mailto:Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>>
>>>    http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org