<div dir="ltr"><i>"On my production servers, the job is queued in RabbitMq, processed perfectly and the notification is published apparently."</i><br><div><i><br></i></div><div>Is this apparent because you have acknowledgements on the RabbitMQ server for the sent messages?</div>
<div>I believe the ACK config and UI for RabbitMQ will report what was sent and received using acknowledgements from the client.</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 21 March 2014 11:47, Simon Morley <span dir="ltr"><<a href="mailto:simon@polkaspots.com" target="_blank">simon@polkaspots.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Morning All<div><br></div><div>Having asked SO, I remembered we were supposed to be encouraging more questions here! I hope you can help (sorry this is a long message).</div>
<div><br></div><div>Relatively new to the world of RabbitMQ, I'm getting stuck debugging the problem of missing messages. All was / is fine in my dev. environment, the problems arise when we start chucking thousand of messages at things. </div>
<div><br></div><div>I've narrowed the problem down to messages that are sent from within another job. For example, we have a job that moves a customer's subscription when they sign up. We publish this to our rabbit servers using the AMQP gem and EM using something like this:</div>
<div><br></div><div><a href="https://gist.github.com/simonmorley/9684010" target="_blank">https://gist.github.com/simonmorley/9684010</a><br></div><div><br></div><div>def queue_process_subscription </div><div> user. process_subscription</div>
<div>end</div><div><br></div><div>def process_subscription</div><div> hammer_time</div><div> MESSAGE_QUEUE.push(json_message, 'ps.routing.key') </div><div>end</div><div><br></div><div>After the job's been completed, we use MESSAGE_QUEUE to push a message to our notification servers (which run NodeJS, node-amqp and SocketIO).</div>
<div><br></div><div>On my lonely laptop, the jobs process perfectly, the notification are sent and my SocketIO client receives them. Happy days.</div><div><br></div><div>On my production servers, the job is queued in RabbitMq, processed perfectly and the notification is published apparently. The problem is, it doesn't appear to be received by the node amqp consumer 100% of the time. Maybe 1 in 10 get delivered (and sometimes they don't look like the full message). The rest just vanish.. It gets worse as we load the servers up.</div>
<div><br></div><div>I thought it was the AMQP on the Node servers but I created a loop to run MESSAGE_QUEUE 10,000 times and they were all consumed..</div><div><br></div><div>So, I'm assuming the problem is that we're sending a message from within another thread. At least, that's what I think it is. My knowledge of such things has now come to an end...</div>
<div><br></div><div>We've now replaced the last notification with Redis publish which works a treat. But that's another cog to think about.</div><div><br></div><div>Like I said, I'm new to this messaging / threading stuff, and would appreciate it if someone can point me in the right direction, tell me what to read or suggestion something else to try. Or tell me to give up and go home.</div>
<div><br></div><div>Cheers</div><div><br></div><div>Simon</div><div><br></div><div><br></div><div><br></div><div><div><div dir="ltr">Simon Morley<br><span style="font-family:arial;font-size:small">Big Chief</span> | PolkaSpots Supafly Wi-Fi<div>
<br><div>simon@PolkaSpots.com<br><div>020 7183 1471<br>@polkaspots</div><div><br><img src="http://ps-ec2-web-1.polkaspots.com/global/images/polkaspots.png" width="200" height="104"><br><br><br><br><div><br></div></div></div>
</div></div></div>
</div></div>
<pre>---------------------------------------------------------------------------------------
The information contained in this email and its attachments is confidential and may be the subject of legal, professional or other privilege. It is intended only for the named addressees and may not be disclosed to anyone else without consent from PolkaSpots Limited. If you are not the named addressee you must not use, disclose, distribute, copy, print or rely on the contents of this email and should destroy it immediately. Whilst PolkaSpots Limited takes care to protect its systems from electronic virus attack or other harmful event, the firm gives no warranty that this email message (including any attachments to it) is free of any virus or other harmful matter and accepts no responsibility for any loss or damage resulting from the recipient receiving, opening or using it. If you need any further information, please contact the originator of this message on <a href="tel:%2B44%20%280%29%2020%207183%201471" value="+442071831471" target="_blank">+44 (0) 20 7183 1471</a>.
PolkaSpots Limited is registered in the UK, number 05508105. Registered Office: 62 Britton Street (behind the bike sheds), Clerkenwell, London EC1M 5UY
Website: <a href="http://polkaspots.com" target="_blank">polkaspots.com</a>
WiFi Shop: <a href="http://shop.polkaspots.com" target="_blank">shop.polkaspots.com</a>
Blog: <a href="http://polkaspots.com/blog" target="_blank">polkaspots.com/blog</a>
---------------------------------------------------------------------------------------
</pre><br>_______________________________________________<br>
Chat mailing list<br>
<a href="mailto:Chat@lists.lrug.org">Chat@lists.lrug.org</a><br>
<a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><a href="http://uk.linkedin.com/in/stephenlaing" target="_blank">http://uk.linkedin.com/in/stephenlaing</a>
</div>