[LRUG] Queuing systems

[Chris Rode]

Ultimately it comes down to how scalable and how important each message is to you. 

If losing a message here or there is not palatable, you will need to look at an AMQP compliant solution (we have clocked reads and writes on RabbitMQ at 1000's per second).

Another option is the redis-backed resque which is great. It is not guaranteed to be fault tolerant though due to the fact that redis is not designed as a message queuing system, although it goes to pains to reduce the loss. 

I hope that helps

Chris

On 07/09/2011, at 11:18, Neil Middleton <neil.middleton at gmail.com> wrote:

> Hundreds, possibly thousands of new jobs per second.
> 
> Neil
> On Wednesday, 7 September 2011 at 10:17, Graham Ashton wrote:
> 
>> On 7 Sep 2011, at 10:14, Neil Middleton wrote:
>> 
>>> a crap load of data from an API
>> 
>> How much is a crap load?
>> _______________________________________________
>> Chat mailing list
>> Chat at lists.lrug.org
>> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> 
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20110907/0cfc5bdf/attachment-0003.html>