[LRUG] ActiveRecord::Relation#as_batches - like find_each but honours order, limit, offset

[Viktor Tron]

Hi Daniel,

Thanks for the great gem.
Indeed testing by parsing logs seems just as volatile as it is an overkill.

I suggest that to test batchwise calling you should just replicate rails  
find_each or find_in_batches tests,
which is checking how many times sql query was called simply using the  
assert_queries method:

https://github.com/rails/rails/blob/9066e3402548d365dc7cf2abda8e7cd64370675c/activerecord/lib/active_record/test_case.rb

Testing correct order and offset restrictions is easy by checking on the  
models manipulated directly without even looking at sql calls.
Roughly:

- define a test model with an ordered_attr
- populate with instances @first, ... @nth with ordered_attr as 0, ..., n
- invoke batches like

@invoked = []
Model.order(:ordered_attr,:desc).as_batches do |m|
  @invoked << m.ordered_attr
end

- and test that assert_equal(@invoked, (0..n).to_a.reverse)
- (and maybe test order without the order clause to make sure it is not by  
pure chance :)

HTH

Vik


On Sat, 05 May 2012 17:44:53 +0100, Daniel Barlow <dan at telent.net> wrote:

> An almost[*]-a-gem announcement, and a question:
>
> The internet confirms that I'm not the only person who's attempted to
> use find_each to get only part of a result set, and been disappointed
> to find that it doesn't support order, limit or offset - any or all of
> which might be useful if you want to consume a large but unknown
> number of records without potentially instantiating several million
> ActiveRecord::Base objects at once.  So  I wrote something that does.
>
> ar-as-batches : https://github.com/telent/ar-as-batches/ lets you do e.g.
>
>     User.where(country_id:
> 44).order(:joined_at).offset(200).as_batches do |user|
>       user.party_all_night!
>     end
>
> and it works much as you'd expect
>
> And now the question: to test it I create an in-memory sqlite database
> and tell activerecord to log its sql queries to a Logger wrapping a
> StringIO, then afterwards I match regexps on the string to see what
> queries were actually sent.  I cannot help feeling that there must be
> a Better Way: is there, and what?
>
> https://github.com/telent/ar-as-batches/blob/master/test/as-batches-test.rb
>
> This is the first time I've really done anything with ActiveRecord
> innards, so all suggestions welcome
>
>
> -dan
>
> [*] "almost" meaning there's a gemspec in the repo and you can include
> it with a :git entry in your Gemfile