Hi Chris + Ronny,<div><br></div><div>Just another data point, but I've had a somewhat different experience to Ronny working with dedicated QA. At the one company where this was done we had one dedicated tester per every five or six developers. They had two basic functions for a given user story:</div>
<div><br></div><div>1. Make sure that developers had implemented the acceptance criteria.</div><div>2. Break the shit out of everything.</div><div><br></div><div>While we really should be doing the 1 ourselves (or preferably automating it), having staff whose sole purpose is to break your web app in new and creative ways made a big difference to the quality of the finished software. Examples might be:</div>
<div><br></div><div>* If I get to the order confirmation screen, then edit my order in a new tab, then switch to the US store and complete my order, should I be paying in GBP?</div><div>* Localization in Simplified Chinese breaks the design on the FAQ page.</div>
<div>* If my address_line_1 is <script>alert('gimme da cookies!');</script> then the code runs on the order confirmation page.</div><div><br></div><div>These are typically scenarios that your automated acceptance tests won't pick up.</div>
<div><br></div><div>They were also responsible for regression testing the most important flows though for the most part they automated this.</div><div><br></div><div>Note that this is a different thing entirely to UX testing, which is for finding out "what to build/change" rather than "how what I've already built is hilariously broken".</div>
<div><br></div><div>Afraid I don't have any recommendations, hope this helps!</div><div><br></div><div>-Ali<br><div><br><div class="gmail_quote">On Wed, Oct 31, 2012 at 7:16 AM, Ronny Ager-Wick <span dir="ltr"><<a href="mailto:ronny@ager-wick.com" target="_blank">ronny@ager-wick.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hi, Chris.<br>
<br>
I have some experience with this, as I had a dedicated tester
while developing one of my bigger projects. To be honest, I wasn't
too impressed with the result. My tester was a remote worker, but
I worked with her in the office for a while too, and it made no
difference. <br>
The issues I discovered was that the tester wasn't a user, and
thus had little knowledge and interest in the work the users are
performing. This makes it very hard for this person to use it like
a user, and, having more than enough just understanding what the
real users are actually using the system for, somewhat challenging
to try something provoking it that may make the system fall over.
Of course one person is hardly representative. I might just have
gotten the wrong person for the job, but it did wake me up to the
fact that testing is actually not easy to find good people for
this task. Standard testing, as in "we've just made this, and it
differs from the old functionality the following way, can you
test?" was fine, but frankly, that's nearly useless testing. If
you can describe exactly what to test, then you've practically
tested it already... I found myself to be the one finding most of
the obscure or complicated bugs that the developers didn't find
themselves. I believe it helps a lot to be a developer. We kind of
instinctively know what can make things blow up... probably
because we've done these types of mistakes before. And being a
software architects I guess helps as well, as we're used to
understanding use cases. I think most software architects find it
much easier than the general population to put themselves in
someone else's situation, imagining how they would use the system.
That should be a minimum requirement for any tester, but I believe
that ability is quite rare.<br>
A tester who can write tests - that would also be interesting! But
that practically means the tester must also be a developer.<br>
<br>
So my advice is then, hire an experienced software architect an/or
developer to do the testing - if you can find one willing of
course... But even if you can, that may be a bit expensive.<br>
On the other hand, I've found actual users to be of great value
for testing. Two-three real users can find an amazing number of
issues which we developers have overlooked, regardless of how good
we are at looking at it from their point of view. Preferably pick
someone that seem interested in the development - typically the
ones with lots of ideas for improvement (or lots of complaints),
and try to get users that do different jobs (if the system does
more than one thing of course), so you get a representative
selection of them. However users (non-developers) are often not
very good at anticipating future issues, possibly because of lack
of technical knowledge. They use the software, and if it works for
them there and then, it's fine with them. But I've experienced a
few times that I have had a nagging feeling that a certain thing
is bound to create issues in the future, which users non-devs
wouldn't even think of, or even argue will *not* create issues.
Most of the time, these turn out to be issues in the end.
Architect/Developer's intuition maybe? Perhaps a combination of
users and developers would be ideal for testing?<br>
<br>
Hope this was at least a bit useful :)<br>
Cheers!<br>
<br>
Ronny.<div><div class="h5"><br>
<br>
<br>
On 30/10/12 07:13, Chris Adams wrote:<br>
</div></div></div>
<blockquote type="cite"><div><div class="h5">
<div> Hi guys, </div>
<div><br>
</div>
<div>We're looking into hiring a tester to work with us in on a
project we've been building for the last 6 weeks or so, to help
catch bugs and issues before they make it to production on a
Rails app we're working on at AMEE.</div>
<div><br>
</div>
<div>We're working on an app that's fairly well protected by
tests, but has a few complex ajax interactions that keep
catching us off guard as we develop, so we're looking for
someone who is particularly good at ferreting out these kinds of
issues.</div>
<div><br>
</div>
<div>The app is pretty small, so I assume we'd be looking for
someone who might be available on a freelance basis. </div>
<div><br>
</div>
<div>However, I haven't really worked with dedicated testers
before, so this is fairly new territory for me - does anyone on
list have any recommendations of people you've worked with, or
any advice on working with testers like above?</div>
<div><br>
</div>
<div>Thanks,</div>
<div><br>
</div>
<div>Chris</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div><br>
</div>
-- <br>
Chris Adams<br>
mobile: 07074 368 229
<div>skype: chris.d.adams</div>
<div>twitter: mrchrisadams</div>
<div>web: <a href="http://chrisadams.me.uk" target="_blank">http://chrisadams.me.uk</a><br>
<div><br>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div></div><pre>_______________________________________________
Chat mailing list
<a href="mailto:Chat@lists.lrug.org" target="_blank">Chat@lists.lrug.org</a>
<a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a>
</pre>
</blockquote>
<br>
</div>
<br>_______________________________________________<br>
Chat mailing list<br>
<a href="mailto:Chat@lists.lrug.org">Chat@lists.lrug.org</a><br>
<a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
<br></blockquote></div><br></div></div>