[LRUG] Advice: manipulating ActiveRecord objects [newbie]

[Ed Saunders]

For a bit of background, the use of the "includes" is all to do with the
N+1 problem - read this for more info:
http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations

I don't think the order of the "includes" or "where" will make a difference
though.

Fred's suggestion of loading into separate hashes may work, but to be
honest it sounds like you're going to need to look at rewriting this stuff
anyway.  Presumably either the FeedBuilder or ResponseBuilder is going to
call into the FeedDetail's associations which will make that difficult.
Without seeing those classes I'm not sure what else to suggest other than a
potential rewrite...

Alternatively, if you're getting that much data in a single request it
might be worth investing time in paginating the results, instead of a
complete rewrite, as Julius has just suggested whilst I type this :)

Cheers,

Ed

On 14 December 2015 at 13:50, Frederick Cheung <frederick.cheung at gmail.com>
wrote:

>
>
>
> On 14 December 2015 at 12:47:21, Marco Iannone (marco.iannone at gmail.com)
> wrote:
>
> ​Hello,
>
> ​
> Ruby newbie question alert
> ​: many apologies if it's too basic, or not the type of questions for this
> forum.
> At least I can say we tried hard before going to StackOverflow
> <http://stackoverflow.com/questions/34227034/manipulating-activerecord-objects-to-build-json>
> ​, and now here.​
>
>
> We're trying to
> ​ensure a webpage loads faster. The main issue
> ​relates to a few activerecord-generated queries on that page.
>
> If you look at the SQL they generate, the queries are as follows (all code
> a bit simplified):
>
> select * from feed_details where account_id = 5
> select * from feeds where id in (VERY_LONG_LIST_OF_IDS_FROM_FIRST_QUERY)
> select * from feeds_metadata where feed_id in
> (VERY_LONG_LIST_FROM_FIRST_QUERY)
> select * from documents where feed_id in (VERY_LONG_LIST_FROM_FIRST_QUERY)
>
> All the indexes are in place, but we verified that they'd take less than
> half the time by changing
> ​each of them to:
>
> select * from feeds, feed_details where id = feed_id and account_id = 5
> select * from feeds_metadata fm, feed_details fd where fm.feed_id =
> fd.feed_id and account_id = 5
> select * from documents where d, feed_details fd where d.feed_id =
> fd.feed_id and account_id = 5
>
>
>
> That in a nutshell is the difference between .preloads and .eager_load
> (.includes defaults to preloads), except that .eager_load runs one query
> to load all the associations. This can perform badly when there are
> multiple has_many being loaded (as in your case) , since you end up pretty
> much with the cartesian  product. Worth a try though.
>
> This generates an active record object with the data from the
> above-mentioned queries, that is used in the controller to build the JSON
> that is sent to the webpage as in below (see feeds_data line):
>
>   def index
>     consumer          = ConsumerWithAccount.new(current_user.account)
>     feed_builder      = FeedBuilder
>     feeds_data        = FeedDetail.feeds_data_for_account(
> current_user.account.id)
>
>     render json: ResponseBuilder.new(consumer, feed_builder, feeds_data)
>   end
>
> To use the faster queries, we were hoping to build the feeds_data
> activerecord object
> ​​by using the faster queries; this gets passed to the JSON builder​. That
> would
> ​seem to require getting rid of the "involves", which (seemingly) uses the
> IN ().
>
> If useful, we have the
> ​more effective activerecord for the 3 queries, they look like:
> Feed.joins(:feed_details).where(feed_document_sets: {account_id:
> current_account_id})
>
> But we couldn't figure out how to build that
> ​ ​
> ​activerecord object
> ​, or one that can be parsed in a similar way by the responsebuilder​
> : therein lies the proverbial rub.
>
> Intense googling and referring to rails docs hasn't helped.​
>
>
>
>
> Sometimes you can crowbar this sort of stuff in: load the objects for each
> of the tables in whatever way is the fastest, keep them in a hash so that
> you can easily identify which ones belongs to which feed detail. Then
> fiddle with some_feed_detail.association(:association_name).target to side
> load in the data. This may not work if response builder expects to have an
> active record relation rather than an array (i.e. it wants to chain more
> stuff onto the scope)
>
> Fred
>
>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> Archives: http://lists.lrug.org/pipermail/chat-lrug.org
> Manage your subscription: http://lists.lrug.org/options.cgi/chat-lrug.org
> List info: http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20151214/78274067/attachment-0002.html>