[LRUG] Distributed delayed_job / resque

[Glenn Gillen]

I've used them both in a distributed fashion.

Resque - Worked better of the two when there were a hundred or so workers attached. My only problem with it is that it's not guaranteed to be durable. So if my Redis instance dies/restarts/whatever I need to be able to rebuild the queue. Easy enough done, just be mindful of it.
DJ - Meh. Ran into problems under heavy load. Ensuring failed jobs were removed and that there was proper indexing on the priority, run_at, etc. fields improved the situation. 

I've been queue_classic for a number of months now without issue. It uses the NOTIFY event within postgres to hand work out to workers. For some reason I have more faith in something implemented at the DB level to work when I need it most.

G


On Jul 3, 2012, at 8:30 AM, Ed James (Alt) wrote:

> Hi all
> 
> I've used both DJ and Resque before, so I'm familiar with both. However, I've never used them in a distributed architecture, where you have workers running on multiple machines. Resque seems to tout this as a feature a bit more then DJ, although the DJ docs say it is possible.
> 
> Has anyone had any experience in running either Resque or DJ in a distributed setup?
> 
> The primary reason for wanting to do this is for the ability to auto-scale (on AWS).
> 
> Thanks.
> 
> -- 
> Ed James
> Sent with Sparrow
> 
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20120703/9d111ac2/attachment-0003.html>