[LRUG] Downloading images quickly

[Tom Stuart]

On 5 Feb 2013, at 11:07, Andrew Stewart wrote:
> To test that it was working, I added a `sleep 3` inside the `request.on_complete` block (lines 17-19).  I then gave it an array of 40 URLs of assets on Amazon S3 and ran it.
> 
> I was expecting this to take ~40s but it actually took 120s.  What am I doing wrong?

Typhoeus uses non-blocking I/O to perform simultaneous requests — it's still single-threaded, like EventMachine. If you sleep the (only) interpreter thread 40 times, you'll wait 120 seconds regardless of any I/O you're doing.

If you want to test it for real, you'll need to arrange for each HTTP request to actually take 3 seconds to return. Maybe write a little Rack app that sleeps for 3 seconds before responding, run it locally in a multithreaded application server (e.g. Puma), and point your code at that instead of S3?

Cheers,
-Tom