[LRUG] Downloading images quickly

Andrew Stewart boss at airbladesoftware.com
Tue Feb 5 03:07:06 PST 2013


Hola El Rug,

I want to download a number of images over HTTP as quickly as possible.  I thought the simplest way would be to use Typhoeus (please tell me if there's a simpler way):

    https://gist.github.com/airblade/4713705

To test that it was working, I added a `sleep 3` inside the `request.on_complete` block (lines 17-19).  I then gave it an array of 40 URLs of assets on Amazon S3 and ran it.

I was expecting this to take ~40s but it actually took 120s.  What am I doing wrong?

I'm on OS X 10.6.8, Ruby 1.9.3p286, typhoeus 0.5.4, ethon 0.5.7, ffi 1.2.1.

Thanks!

Cheers,
Andy Stewart
-------
http://airbladesoftware.com






More information about the Chat mailing list