[LRUG] Large Slow Test Suites
Mr Jaba
the.jaba at gmail.com
Fri Dec 6 00:17:16 PST 2013
Thanks again for all the responses everyone, hugely useful!
So far we've managed to shave 10 minutes off the test run using a variety
of tricks:
1 - spring
2 - parallel_tests
3 - GC tuning
4 - Deferred GC
Not a trick, but upgrading to Ruby 2.0 has helped as well, although this
has caused a few test failures as well.
To get the rest of the suite down, i'll be looking at implementing lots of
people's suggestions
Thanks
Tom
On 5 December 2013 21:46, Viktor Trón <viktor.tron at gmail.com> wrote:
> IMHO zeus, vcr, database cleaner truncate, db on ramdisk, and frugal
> use of factory create (as opposed to build) go a very long way with
> clean non-hacky solutions
>
> On 5 December 2013 18:58, James McCarthy <james2mccarthy at gmail.com> wrote:
> > Just in case some low hanging fruit hasn't been covered;
> >
> > I’d first turn your network connection off and run the test suit to catch
> > any stupid tests coupled to api’s and put VCR on there for the moment.
> I’ve
> > picked up several apps where this has happened.
> >
> > If you’ve got Devise in there make sure your test_helper puts the stretch
> > time down to one; Devise.stretches = 1
> >
> > If Factory Girl is being used then move as many things to use
> > builld_stubbed; Use Factory Girl's build_stubbed for a Faster Test Suite
> >
> > What version of Ruby? I’ve found vanilla 1.8.7 benefitted from moving to
> EE
> > and manually controlling GC YMMV;
> >
> >
> http://fredwu.me/post/60441991350/protip-ruby-devs-please-tweak-your-gc-settings-for
> > https://gist.github.com/JamesChevalier/4735441
> > http://labs.goclio.com/tuning-ruby-garbage-collection-for-rspec/
> >
> > I have gone down the route of taking my least rails coupled tests
> require a
> > skinny_test_helper.rb that doesn’t load the whole rails stack and
> provides
> > just enough unit test running, leaving the slow stuff to integration and
> > controller tests, and at this stage, only the stuff you can move over
> really
> > easily or you are re-factoring the wrong thing.
> >
> > Controller tests often get a bad rap, they can add value. Ask yourself if
> > the goal of an integration test could be achieved by testing further down
> > the stack where it consumes less resources e.g. is a test for the
> presence
> > of a widgets description actually a proxy for the controller
> instantiating a
> > variable. Is an integration test actually a proxy for testing a helper,
> or
> > an indication that the view code should be re-factored into some kind of
> > view [helper | view object | presenter | whatever you call it]?
> >
> > But now I’m getting into re-factoring which I didn’t set out to comment
> on:
> >
> http://blog.codeclimate.com/blog/2012/10/17/7-ways-to-decompose-fat-activerecord-models/
> >
> >
> >
> > On 5 Dec 2013, at 10:54, Sam Livingston-Gray <geeksam at gmail.com> wrote:
> >
> > I just put a require statement at the top of the fast test file for what
> I'm
> > testing, and another one for anything it depends on. When that gets to be
> > annoying, I know I have too many dependencies. ;)
> >
> > --
> > (Sent from phone. Please excuse: brevity, top posting, hilarious
> > autocorrections.)
> >
> > On Dec 5, 2013, at 7:29 AM, Jon Leighton <j at jonathanleighton.com> wrote:
> >
> > Hi Tom,
> >
> > On 05/12/13 09:28, Mr Jaba wrote:
> > Interesting points on making your unit tests independent of Rails, I
> > hadn't considered the class reloading aspect. Does this interfere very
> > badly? Is it avoidable at all? Or was the cost of maintaining all the
> > requires simply too high?
> >
> >
> > If you want to run your tests outside of Rails, you can't rely on the
> > rails autoloading mechanism (ActiveSupport::Dependencies). But if you
> > put requires all over the place, it will prevent AS::Dependencies from
> > working when you *do* want it to (i.e. code reloading in dev mode won't
> > work, because the code was loaded by a require rather than a missing
> > constant).
> >
> > The solution I ended up with was basically using require_dependency
> > rather than require, and stubbing out require_dependency for my "outside
> > Rails" tests. It was nasty.
> >
> > In might be possible to *just* use AS::Dependencies in your unit tests,
> > without loading the full Rails app. That's something I didn't try, but
> > with spring loading the full app is not such a big deal for me these
> > days anyway, so I don't see the benefit of having tests that can run
> > outside of the Rails app.
> >
> > Jon
> >
> >
> > Thanks again
> >
> > Tom
> >
> >
> > On 5 December 2013 09:04, Jon Leighton <j at jonathanleighton.com
> > <mailto:j at jonathanleighton.com>> wrote:
> >
> > Hi Tom,
> >
> > On 04/12/13 22:56, Mr Jaba wrote:
> > First up, thanks for Spring (amongst your many other contributions!)
> > it's a really useful gem!
> >
> >
> > Glad you like it!
> >
> > With regards to the factories, what do you recommend instead? Simpler
> > objects with less coupling to Rails?
> >
> >
> > Yeah I don't like the factories approach at all. Apart from often being
> > a source of test slowness, they encourage you down the path of high
> > coupling and make it too trivial to create a boat load of
> > database-backed objects. Bo Jeanes wrote a post about this:
> > http://bjeanes.com/2012/02/factories-breed-complexity
> >
> > There are certainly situations in a Rails app where you need to test
> > against the database. For this I use fixtures, which I find perfectly
> > adequate and much faster than factories. You do need to ensure your
> > fixtures don't get out of control, but generally I think of fixtures as
> > a handful of pre-inserted records that I might modify in specific tests
> > in order to achieve a certain thing (rather than creating a new fixture
> > entry for each thing, which can get out of hand).
> >
> > Apart from that, it's obviously wise to avoid inserting stuff in the
> > database as much as possible, e.g. if you just want to test the logic
> of
> > a certain method which only interacts with a model's attributes, you
> can
> > just create an in-memory instance.
> >
> > I have been down the path of making my "unit tests" independent of the
> > Rails app so as to avoid the boot process, and found it impractical to
> > be honest:
> >
> > 1) You end up with require and require_dependency everywhere, and it's
> > hard to get this right in a way that plays nicely with Rails' class
> > reloading
> > 2) Even if you make some tests independent of Rails, you're still going
> > to need some that are not independent of Rails, so you only solve part
> > of the problem
> >
> > This was part of my motivation for spring - if we can't make get rid of
> > having to boot the Rails app, let's make booting the Rails app less
> > painful.
> >
> > Not sure any of these comments help with your current predicament, but
> > hopefully they're vaguely useful anyway!
> >
> > Jon
> >
> >
> > Thanks for your input!
> >
> > Tom
> >
> >
> > On 4 December 2013 22:43, Jon Leighton <j at jonathanleighton.com
> >
> > <mailto:j at jonathanleighton.com>
> >
> > <mailto:j at jonathanleighton.com <mailto:j at jonathanleighton.com>>>
> >
> > wrote:
> >
> >
> > Hello there,
> >
> > On 04/12/13 22:20, Mr Jaba wrote:
> > I've recently taken ownership of a new project with a large
> >
> > test suite
> >
> > (2000ish tests), and the overall run time is around 30 minutes
> >
> > which is
> >
> > certainly less than ideal!
> >
> > Now I know the general approach to "Fast Rails Tests" but
> >
> > taking the
> >
> > time to refactor the whole test suite is a bit too much
> >
> > right now. I'm
> >
> > wondering if anyone has experience of transforming a test
> >
> > suite of
> >
> > this
> >
> > magnitude to something a bit speedier? If so how did you go
> >
> > about it?
> >
> > What tips, tricks and techniques can you share?
> >
> > A bit more info:
> >
> > - Test::Unit tests, moving to Minitest
> > - Rails 3.2.16
> > - Running parallel_tests shaved 5 mins off the time
> > - Using Spring to reduce Rails load time.
> >
> > Any advice gratefully received!
> >
> >
> > I think you need to provide some more information :) What's
> >
> > making it
> >
> > take so long - the sheer number of tests or what the tests are
> >
> > doing? Do
> >
> > you have certain types of tests that are taking a large
> >
> > portion of the
> >
> > time (e.g. Capybara tests?) Are you using factories? In my
> >
> > experience
> >
> > factories are an easy cause of a slow test suite.
> >
> > One thing to think about is whether you can break up your test
> >
> > suite.
> >
> > For example in the application I work on, we have
> >
> > unit/integration tests
> >
> > which run in about 25 seconds on my machine. I frequently run
> >
> > these
> >
> > during development and they often alert me to problems. But I
> >
> > rarely run
> >
> > the full set of capybara tests which takes several minutes - I
> >
> > let the
> >
> > CI do that and run failing tests individually where necessary.
> >
> > This
> >
> > achieves a decent balance of fast feedback for development without
> > throwing away the safety net of proper full-stack tests.
> >
> > Jon
> > _______________________________________________
> > Chat mailing list
> > Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>
> >
> > <mailto:Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>>
> >
> > http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> >
> > _______________________________________________
> > Chat mailing list
> > Chat at lists.lrug.org
> > http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> >
> > _______________________________________________
> > Chat mailing list
> > Chat at lists.lrug.org
> > http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> >
> >
> >
> > _______________________________________________
> > Chat mailing list
> > Chat at lists.lrug.org
> > http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> >
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20131206/3bf9d95e/attachment-0003.html>
More information about the Chat
mailing list