[LRUG] Large Slow Test Suites
James McCarthy
james2mccarthy at gmail.com
Thu Dec 5 10:58:34 PST 2013
Just in case some low hanging fruit hasn't been covered;
I’d first turn your network connection off and run the test suit to catch any stupid tests coupled to api’s and put VCR on there for the moment. I’ve picked up several apps where this has happened.
If you’ve got Devise in there make sure your test_helper puts the stretch time down to one; Devise.stretches = 1
If Factory Girl is being used then move as many things to use builld_stubbed; Use Factory Girl's build_stubbed for a Faster Test Suite
What version of Ruby? I’ve found vanilla 1.8.7 benefitted from moving to EE and manually controlling GC YMMV;
http://fredwu.me/post/60441991350/protip-ruby-devs-please-tweak-your-gc-settings-for
https://gist.github.com/JamesChevalier/4735441
http://labs.goclio.com/tuning-ruby-garbage-collection-for-rspec/
I have gone down the route of taking my least rails coupled tests require a skinny_test_helper.rb that doesn’t load the whole rails stack and provides just enough unit test running, leaving the slow stuff to integration and controller tests, and at this stage, only the stuff you can move over really easily or you are re-factoring the wrong thing.
Controller tests often get a bad rap, they can add value. Ask yourself if the goal of an integration test could be achieved by testing further down the stack where it consumes less resources e.g. is a test for the presence of a widgets description actually a proxy for the controller instantiating a variable. Is an integration test actually a proxy for testing a helper, or an indication that the view code should be re-factored into some kind of view [helper | view object | presenter | whatever you call it]?
But now I’m getting into re-factoring which I didn’t set out to comment on: http://blog.codeclimate.com/blog/2012/10/17/7-ways-to-decompose-fat-activerecord-models/
On 5 Dec 2013, at 10:54, Sam Livingston-Gray <geeksam at gmail.com> wrote:
> I just put a require statement at the top of the fast test file for what I'm testing, and another one for anything it depends on. When that gets to be annoying, I know I have too many dependencies. ;)
>
> --
> (Sent from phone. Please excuse: brevity, top posting, hilarious autocorrections.)
>
>> On Dec 5, 2013, at 7:29 AM, Jon Leighton <j at jonathanleighton.com> wrote:
>>
>> Hi Tom,
>>
>>> On 05/12/13 09:28, Mr Jaba wrote:
>>> Interesting points on making your unit tests independent of Rails, I
>>> hadn't considered the class reloading aspect. Does this interfere very
>>> badly? Is it avoidable at all? Or was the cost of maintaining all the
>>> requires simply too high?
>>
>> If you want to run your tests outside of Rails, you can't rely on the
>> rails autoloading mechanism (ActiveSupport::Dependencies). But if you
>> put requires all over the place, it will prevent AS::Dependencies from
>> working when you *do* want it to (i.e. code reloading in dev mode won't
>> work, because the code was loaded by a require rather than a missing
>> constant).
>>
>> The solution I ended up with was basically using require_dependency
>> rather than require, and stubbing out require_dependency for my "outside
>> Rails" tests. It was nasty.
>>
>> In might be possible to *just* use AS::Dependencies in your unit tests,
>> without loading the full Rails app. That's something I didn't try, but
>> with spring loading the full app is not such a big deal for me these
>> days anyway, so I don't see the benefit of having tests that can run
>> outside of the Rails app.
>>
>> Jon
>>
>>>
>>> Thanks again
>>>
>>> Tom
>>>
>>>
>>> On 5 December 2013 09:04, Jon Leighton <j at jonathanleighton.com
>>> <mailto:j at jonathanleighton.com>> wrote:
>>>
>>> Hi Tom,
>>>
>>>> On 04/12/13 22:56, Mr Jaba wrote:
>>>> First up, thanks for Spring (amongst your many other contributions!)
>>>> it's a really useful gem!
>>>
>>> Glad you like it!
>>>
>>>> With regards to the factories, what do you recommend instead? Simpler
>>>> objects with less coupling to Rails?
>>>
>>> Yeah I don't like the factories approach at all. Apart from often being
>>> a source of test slowness, they encourage you down the path of high
>>> coupling and make it too trivial to create a boat load of
>>> database-backed objects. Bo Jeanes wrote a post about this:
>>> http://bjeanes.com/2012/02/factories-breed-complexity
>>>
>>> There are certainly situations in a Rails app where you need to test
>>> against the database. For this I use fixtures, which I find perfectly
>>> adequate and much faster than factories. You do need to ensure your
>>> fixtures don't get out of control, but generally I think of fixtures as
>>> a handful of pre-inserted records that I might modify in specific tests
>>> in order to achieve a certain thing (rather than creating a new fixture
>>> entry for each thing, which can get out of hand).
>>>
>>> Apart from that, it's obviously wise to avoid inserting stuff in the
>>> database as much as possible, e.g. if you just want to test the logic of
>>> a certain method which only interacts with a model's attributes, you can
>>> just create an in-memory instance.
>>>
>>> I have been down the path of making my "unit tests" independent of the
>>> Rails app so as to avoid the boot process, and found it impractical to
>>> be honest:
>>>
>>> 1) You end up with require and require_dependency everywhere, and it's
>>> hard to get this right in a way that plays nicely with Rails' class
>>> reloading
>>> 2) Even if you make some tests independent of Rails, you're still going
>>> to need some that are not independent of Rails, so you only solve part
>>> of the problem
>>>
>>> This was part of my motivation for spring - if we can't make get rid of
>>> having to boot the Rails app, let's make booting the Rails app less
>>> painful.
>>>
>>> Not sure any of these comments help with your current predicament, but
>>> hopefully they're vaguely useful anyway!
>>>
>>> Jon
>>>
>>>>
>>>> Thanks for your input!
>>>>
>>>> Tom
>>>>
>>>>
>>>> On 4 December 2013 22:43, Jon Leighton <j at jonathanleighton.com
>>> <mailto:j at jonathanleighton.com>
>>>> <mailto:j at jonathanleighton.com <mailto:j at jonathanleighton.com>>>
>>> wrote:
>>>>
>>>> Hello there,
>>>>
>>>>> On 04/12/13 22:20, Mr Jaba wrote:
>>>>> I've recently taken ownership of a new project with a large
>>> test suite
>>>>> (2000ish tests), and the overall run time is around 30 minutes
>>>> which is
>>>>> certainly less than ideal!
>>>>>
>>>>> Now I know the general approach to "Fast Rails Tests" but
>>> taking the
>>>>> time to refactor the whole test suite is a bit too much
>>> right now. I'm
>>>>> wondering if anyone has experience of transforming a test
>>> suite of
>>>> this
>>>>> magnitude to something a bit speedier? If so how did you go
>>> about it?
>>>>> What tips, tricks and techniques can you share?
>>>>>
>>>>> A bit more info:
>>>>>
>>>>> - Test::Unit tests, moving to Minitest
>>>>> - Rails 3.2.16
>>>>> - Running parallel_tests shaved 5 mins off the time
>>>>> - Using Spring to reduce Rails load time.
>>>>>
>>>>> Any advice gratefully received!
>>>>
>>>> I think you need to provide some more information :) What's
>>> making it
>>>> take so long - the sheer number of tests or what the tests are
>>> doing? Do
>>>> you have certain types of tests that are taking a large
>>> portion of the
>>>> time (e.g. Capybara tests?) Are you using factories? In my
>>> experience
>>>> factories are an easy cause of a slow test suite.
>>>>
>>>> One thing to think about is whether you can break up your test
>>> suite.
>>>> For example in the application I work on, we have
>>> unit/integration tests
>>>> which run in about 25 seconds on my machine. I frequently run
>>> these
>>>> during development and they often alert me to problems. But I
>>> rarely run
>>>> the full set of capybara tests which takes several minutes - I
>>> let the
>>>> CI do that and run failing tests individually where necessary.
>>> This
>>>> achieves a decent balance of fast feedback for development without
>>>> throwing away the safety net of proper full-stack tests.
>>>>
>>>> Jon
>>>> _______________________________________________
>>>> Chat mailing list
>>>> Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>
>>> <mailto:Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>>
>>>> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>> _______________________________________________
>> Chat mailing list
>> Chat at lists.lrug.org
>> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20131205/becdd0e9/attachment-0003.html>
More information about the Chat
mailing list