[LRUG] Enumerator shenanigans

[Oliver Legg]

Andy,

I think there are really two issues here. First, having a method returning an Enumerator or yielding depending on whether a block is passed to the method or not. Second, how to do lazy enumeration in Ruby 2.0.

You're right, that approach is a little suboptimal. Ruby provides the `enum_for`[1] method for just this use case. See this gist[2] for your complete code, but the trick is to use this  line at the beginning of the method.

    return enum_for(:find_each) unless block_given?

You, also then, don't need to explicitly create the Enumerator. Effectively `enum_for` will use the method in-place of the block when initialising the Enumerator.

With regards to lazy enumerators, neither of those examples are using lazy enumeration. IIRC both of those examples will work in Ruby 1.9 and 2.0.

Also, I don't think this example will work as I suspect `reduce`, in this case, doesn't return an `Enumerable`. So `take` will raise an error.

   Item.find_each.map(&:price).reduce(:+).take(75)

In Ruby 2.0 you need to use the `lazy`[3] method to get a lazy enumerator. So for example:

    Item.find_each.lazy.take(75).map(&:price).reduce(:+)

although in this specific case it doesn't change the result – and in all likelihood will make things slower.

I did a talk on Enumerators at last months LRUG, the video[4] and slides[5] are online. There's also an example of where I used a similar pattern[6].

Hope thats useful.

Cheers, Olly

[1] http://ruby-doc.org/core-2.0.0/Object.html#method-i-enum_for
[2] https://gist.github.com/olly/7466900
[3] http://ruby-doc.org/core-2.0.0/Enumerable.html#method-i-lazy
[4] http://skillsmatter.com/podcast/home/enumerators-in-ruby
[5] https://speakerdeck.com/ollylegg/enumerators
[6] https://github.com/olly/grim_repo/blob/master/lib/grim_repo/paginated_collection.rb


On 14 Nov 2013, at 13:24, Andrew Stewart <boss at airbladesoftware.com> wrote:

> Hello El Rug,
> 
> I have written a method that fetches items in turn from a paged collection.  When called with a block it yields each item to the block.  When called without a block, it returns an Enumerator that spits out each item in turn as lazily as possible.
> 
> Let's say the collection has 1,000 items and we can fetch pages of 50 at a time.  This is what I want:
> 
>    # yields as it goes, retrieving pages as needed behind the scenes
>    Item.find_each do |item|
>      puts item.name
>    end
> 
>    # only retrieves the first two pages
>    # Ruby 1.9 doesn't do lazy map or reduce, I think?
>    Item.find_each.take(75).map(&:price).reduce(:+)
> 
>    # only retrieves the first two pages
>    # Ruby 2 allows this?
>    Item.find_each.map(&:price).reduce(:+).take(75)
> 
> I'd like the code to work on Ruby 1.9 and 2.
> 
> So I have a working implementation but it looks, er, suboptimal.  What's the proper way to do this?
> 
>    class Item
>      def self.find_each(params = {})
>        if block_given?
>          page = nil
>          while page.nil? || page.current_page < page.total_pages
>            if page.nil?
>              page = all params
>            else
>              page = all params.merge(page: page.current_page + 1)
>            end
>            page.each do |item|
>              yield item
>            end
>          end
> 
>        else
>          Enumerator.new do |yielder|
>            page = nil
>            while page.nil? || page.current_page < page.total_pages
>              if page.nil?
>                page = all params
>              else
>                page = all params.merge(page: page.current_page + 1)
>              end
>              page.each do |item|
>                yielder << item
>              end
>            end
>          end
>        end
>      end
> 
>      def self.all(params = {})
>        # snip
>      end
>    end
> 
> Thanks in advance,
> 
> Andy Stewart
> 
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org