[LRUG] Downloading images quickly

[Tom Taylor]

Ah, sorry, realised you want to parallelise requests, not introduce a delay. Ignore me!

On 5 Feb 2013, at 11:40, Tom Taylor <tom at tomtaylor.co.uk> wrote:

> You're probably better off using something like wget for this. Something like `wget -i list_of_files --wait 3` would do the trick.
> 
> On 5 Feb 2013, at 11:07, Andrew Stewart <boss at airbladesoftware.com> wrote:
> 
>> Hola El Rug,
>> 
>> I want to download a number of images over HTTP as quickly as possible.  I thought the simplest way would be to use Typhoeus (please tell me if there's a simpler way):
>> 
>>   https://gist.github.com/airblade/4713705
>> 
>> To test that it was working, I added a `sleep 3` inside the `request.on_complete` block (lines 17-19).  I then gave it an array of 40 URLs of assets on Amazon S3 and ran it.
>> 
>> I was expecting this to take ~40s but it actually took 120s.  What am I doing wrong?
>> 
>> I'm on OS X 10.6.8, Ruby 1.9.3p286, typhoeus 0.5.4, ethon 0.5.7, ffi 1.2.1.
>> 
>> Thanks!
>> 
>> Cheers,
>> Andy Stewart
>> -------
>> http://airbladesoftware.com
>> 
>> 
>> 
>> _______________________________________________
>> Chat mailing list
>> Chat at lists.lrug.org
>> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> 
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org