<font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif">For the past 6 months I've been working for GoSquared, a real-time website analytics service, and the available tech choices are pretty straightforward.</font></font></font><div>
<font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif"><br></font></font></font></div><div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif"><b>Redis</b> for real-time, short-term storage. It shines for aggregates and counters in particular.</font></font></font></div>
<div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif"><br></font></font></font></div><div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif">I've spent a fair amount of time with <b>Graphite</b> and there is no other open source project which gives you more time series goodness out of the box. MongoDB is nice and easy up to a certain point, but you find yourself needing to write a lot of code to do something that Graphite does brilliantly by default. <a href="https://github.com/gchef/graphite-cookbook">This should help you get started</a>.</font></font></font></div>
<div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif"><br></font></font></font></div><div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif">Your biggest challenge will be the long-term storage and post-processing (ie. map reduce). <b>Cassandra</b> is my favourite, but <b>MongoDB</b> will do the job. Pre Mongo 1.8 map reduce was buggy and very difficult to work with, but I've heard that since it got a lot better. Pick whatever you're most comfortable with, they're both fine choices.</font></font></font></div>
<div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif"><br></font></font></font></div><div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif">The challenge ahead of you is immense. Teams of many sharp developers have been working on the same problems for many years now. If you'll want some of the work cut out for you, there's always services with good APIs which you can build on. My first recommendation will have to be <a href="http://gosquared.com">gosquared.com</a> (in particular the February release), but <a href="http://chartbeat.com">chartbeat.com</a> and <a href="http://gaug.es">gaug.es</a> have their strengths too. For something more DIY-ish, I would check out <a href="https://metrics.librato.com/">Librato's metrics</a> and <a href="http://www.splunk.com/view/whats-new/SP-CAAAFD2">Splunk</a>.</font></font></font></div>
<div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif"><br></font></font></font></div><div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif">Good luck!</font></font></font></div>
<div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif"><br></font></font></font></div><div><font color="#000000"><font size="4"><font face="arial,helvetica,sans-serif">Gerhard<br clear="all"></font></font></font><br>
<hr style="font-family:arial,helvetica,sans-serif"><font color="#3333ff"><font face="arial, helvetica, sans-serif"><a href="http://twitter.com/#!/gerhardlazu" target="_blank">Twitter</a> <a href="https://github.com/gerhard" target="_blank">Github</a> </font><a href="http://gerhardlazu.com/" style="font-family:arial,helvetica,sans-serif" target="_blank">Blog</a></font><br>
<br><br><div class="gmail_quote">On Wed, Jan 25, 2012 at 12:34 PM, Adam Carlile <span dir="ltr"><<a href="mailto:adam@benchmedia.co.uk">adam@benchmedia.co.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><div><div style="font-size:14px;font-family:Calibri,sans-serif">Hola El Rug</div><div style="font-size:14px;font-family:Calibri,sans-serif"><br></div><div style="font-size:14px;font-family:Calibri,sans-serif">
I've been tasked with creating a user activity tracking engine, in order to build more complete profiles on our users, their preferences and common activities. </div><div style="font-size:14px;font-family:Calibri,sans-serif">
<br></div><div style="font-size:14px;font-family:Calibri,sans-serif">But I'm not really sure in which direction to go, I'm pretty sure that it's got to be a separate app, with a fast write database, something like MongoDB perhaps. Along with an exposed REST API to post and retrieve data.</div>
<div style="font-size:14px;font-family:Calibri,sans-serif"><br></div><div style="font-size:14px;font-family:Calibri,sans-serif">I also want the tracking engine to send calls to various other external services, such as the Facebook Graph API.</div>
<div style="font-size:14px;font-family:Calibri,sans-serif"><br></div><div style="font-size:14px;font-family:Calibri,sans-serif">It's something I've never tackled before so any advice or articles you would recommend would be great!</div>
<div style="font-size:14px;font-family:Calibri,sans-serif"><br></div><div style="font-size:14px;font-family:Calibri,sans-serif">Thanks</div><div style="font-size:14px;font-family:Calibri,sans-serif">Adam</div><div><div style="font-size:14px;font-family:Calibri,sans-serif">
<br></div><font face="Calibri,Verdana,Helvetica,Arial" style="font-size:14px;font-family:Calibri,sans-serif"><span style="font-size:11pt">-</span></font><div><font><font><font><div style="font-size:14px;font-family:Calibri,sans-serif">
<font color="#7F7F7F"><font size="1"><font face="Arial"><span style="font-size:9pt"><b>Twitter</b> @frozenproduce</span></font></font></font></div><div><font color="#7f7f7f" face="Arial"><span style="font-size:12px"><b><br>
</b></span></font></div></font></font></font></div></div></div></div></div>
<br>_______________________________________________<br>
Chat mailing list<br>
<a href="mailto:Chat@lists.lrug.org">Chat@lists.lrug.org</a><br>
<a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
<br></blockquote></div><br></div>