[LRUG] Large Slow Test Suites
Viktor Trón
viktor.tron at gmail.com
Thu Dec 5 13:46:15 PST 2013
IMHO zeus, vcr, database cleaner truncate, db on ramdisk, and frugal
use of factory create (as opposed to build) go a very long way with
clean non-hacky solutions
On 5 December 2013 18:58, James McCarthy <james2mccarthy at gmail.com> wrote:
> Just in case some low hanging fruit hasn't been covered;
>
> I’d first turn your network connection off and run the test suit to catch
> any stupid tests coupled to api’s and put VCR on there for the moment. I’ve
> picked up several apps where this has happened.
>
> If you’ve got Devise in there make sure your test_helper puts the stretch
> time down to one; Devise.stretches = 1
>
> If Factory Girl is being used then move as many things to use
> builld_stubbed; Use Factory Girl's build_stubbed for a Faster Test Suite
>
> What version of Ruby? I’ve found vanilla 1.8.7 benefitted from moving to EE
> and manually controlling GC YMMV;
>
> http://fredwu.me/post/60441991350/protip-ruby-devs-please-tweak-your-gc-settings-for
> https://gist.github.com/JamesChevalier/4735441
> http://labs.goclio.com/tuning-ruby-garbage-collection-for-rspec/
>
> I have gone down the route of taking my least rails coupled tests require a
> skinny_test_helper.rb that doesn’t load the whole rails stack and provides
> just enough unit test running, leaving the slow stuff to integration and
> controller tests, and at this stage, only the stuff you can move over really
> easily or you are re-factoring the wrong thing.
>
> Controller tests often get a bad rap, they can add value. Ask yourself if
> the goal of an integration test could be achieved by testing further down
> the stack where it consumes less resources e.g. is a test for the presence
> of a widgets description actually a proxy for the controller instantiating a
> variable. Is an integration test actually a proxy for testing a helper, or
> an indication that the view code should be re-factored into some kind of
> view [helper | view object | presenter | whatever you call it]?
>
> But now I’m getting into re-factoring which I didn’t set out to comment on:
> http://blog.codeclimate.com/blog/2012/10/17/7-ways-to-decompose-fat-activerecord-models/
>
>
>
> On 5 Dec 2013, at 10:54, Sam Livingston-Gray <geeksam at gmail.com> wrote:
>
> I just put a require statement at the top of the fast test file for what I'm
> testing, and another one for anything it depends on. When that gets to be
> annoying, I know I have too many dependencies. ;)
>
> --
> (Sent from phone. Please excuse: brevity, top posting, hilarious
> autocorrections.)
>
> On Dec 5, 2013, at 7:29 AM, Jon Leighton <j at jonathanleighton.com> wrote:
>
> Hi Tom,
>
> On 05/12/13 09:28, Mr Jaba wrote:
> Interesting points on making your unit tests independent of Rails, I
> hadn't considered the class reloading aspect. Does this interfere very
> badly? Is it avoidable at all? Or was the cost of maintaining all the
> requires simply too high?
>
>
> If you want to run your tests outside of Rails, you can't rely on the
> rails autoloading mechanism (ActiveSupport::Dependencies). But if you
> put requires all over the place, it will prevent AS::Dependencies from
> working when you *do* want it to (i.e. code reloading in dev mode won't
> work, because the code was loaded by a require rather than a missing
> constant).
>
> The solution I ended up with was basically using require_dependency
> rather than require, and stubbing out require_dependency for my "outside
> Rails" tests. It was nasty.
>
> In might be possible to *just* use AS::Dependencies in your unit tests,
> without loading the full Rails app. That's something I didn't try, but
> with spring loading the full app is not such a big deal for me these
> days anyway, so I don't see the benefit of having tests that can run
> outside of the Rails app.
>
> Jon
>
>
> Thanks again
>
> Tom
>
>
> On 5 December 2013 09:04, Jon Leighton <j at jonathanleighton.com
> <mailto:j at jonathanleighton.com>> wrote:
>
> Hi Tom,
>
> On 04/12/13 22:56, Mr Jaba wrote:
> First up, thanks for Spring (amongst your many other contributions!)
> it's a really useful gem!
>
>
> Glad you like it!
>
> With regards to the factories, what do you recommend instead? Simpler
> objects with less coupling to Rails?
>
>
> Yeah I don't like the factories approach at all. Apart from often being
> a source of test slowness, they encourage you down the path of high
> coupling and make it too trivial to create a boat load of
> database-backed objects. Bo Jeanes wrote a post about this:
> http://bjeanes.com/2012/02/factories-breed-complexity
>
> There are certainly situations in a Rails app where you need to test
> against the database. For this I use fixtures, which I find perfectly
> adequate and much faster than factories. You do need to ensure your
> fixtures don't get out of control, but generally I think of fixtures as
> a handful of pre-inserted records that I might modify in specific tests
> in order to achieve a certain thing (rather than creating a new fixture
> entry for each thing, which can get out of hand).
>
> Apart from that, it's obviously wise to avoid inserting stuff in the
> database as much as possible, e.g. if you just want to test the logic of
> a certain method which only interacts with a model's attributes, you can
> just create an in-memory instance.
>
> I have been down the path of making my "unit tests" independent of the
> Rails app so as to avoid the boot process, and found it impractical to
> be honest:
>
> 1) You end up with require and require_dependency everywhere, and it's
> hard to get this right in a way that plays nicely with Rails' class
> reloading
> 2) Even if you make some tests independent of Rails, you're still going
> to need some that are not independent of Rails, so you only solve part
> of the problem
>
> This was part of my motivation for spring - if we can't make get rid of
> having to boot the Rails app, let's make booting the Rails app less
> painful.
>
> Not sure any of these comments help with your current predicament, but
> hopefully they're vaguely useful anyway!
>
> Jon
>
>
> Thanks for your input!
>
> Tom
>
>
> On 4 December 2013 22:43, Jon Leighton <j at jonathanleighton.com
>
> <mailto:j at jonathanleighton.com>
>
> <mailto:j at jonathanleighton.com <mailto:j at jonathanleighton.com>>>
>
> wrote:
>
>
> Hello there,
>
> On 04/12/13 22:20, Mr Jaba wrote:
> I've recently taken ownership of a new project with a large
>
> test suite
>
> (2000ish tests), and the overall run time is around 30 minutes
>
> which is
>
> certainly less than ideal!
>
> Now I know the general approach to "Fast Rails Tests" but
>
> taking the
>
> time to refactor the whole test suite is a bit too much
>
> right now. I'm
>
> wondering if anyone has experience of transforming a test
>
> suite of
>
> this
>
> magnitude to something a bit speedier? If so how did you go
>
> about it?
>
> What tips, tricks and techniques can you share?
>
> A bit more info:
>
> - Test::Unit tests, moving to Minitest
> - Rails 3.2.16
> - Running parallel_tests shaved 5 mins off the time
> - Using Spring to reduce Rails load time.
>
> Any advice gratefully received!
>
>
> I think you need to provide some more information :) What's
>
> making it
>
> take so long - the sheer number of tests or what the tests are
>
> doing? Do
>
> you have certain types of tests that are taking a large
>
> portion of the
>
> time (e.g. Capybara tests?) Are you using factories? In my
>
> experience
>
> factories are an easy cause of a slow test suite.
>
> One thing to think about is whether you can break up your test
>
> suite.
>
> For example in the application I work on, we have
>
> unit/integration tests
>
> which run in about 25 seconds on my machine. I frequently run
>
> these
>
> during development and they often alert me to problems. But I
>
> rarely run
>
> the full set of capybara tests which takes several minutes - I
>
> let the
>
> CI do that and run failing tests individually where necessary.
>
> This
>
> achieves a decent balance of fast feedback for development without
> throwing away the safety net of proper full-stack tests.
>
> Jon
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>
>
> <mailto:Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>>
>
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
>
>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
More information about the Chat
mailing list