[LRUG] Looking for recommendations for a tester on a Rails project

[Najaf Ali]

Hi Chris + Ronny,

Just another data point, but I've had a somewhat different experience to
Ronny working with dedicated QA. At the one company where this was done we
had one dedicated tester per every five or six developers. They had two
basic functions for a given user story:

1. Make sure that developers had implemented the acceptance criteria.
2. Break the shit out of everything.

While we really should be doing the 1 ourselves (or preferably automating
it), having staff whose sole purpose is to break your web app in new and
creative ways made a big difference to the quality of the finished
software. Examples might be:

* If I get to the order confirmation screen, then edit my order in a new
tab, then switch to the US store and complete my order, should I be paying
in GBP?
* Localization in Simplified Chinese breaks the design on the FAQ page.
* If my address_line_1 is <script>alert('gimme da cookies!');</script> then
the code runs on the order confirmation page.

These are typically scenarios that your automated acceptance tests won't
pick up.

They were also responsible for regression testing the most important flows
though for the most part they automated this.

Note that this is a different thing entirely to UX testing, which is for
finding out "what to build/change" rather than "how what I've already built
is hilariously broken".

Afraid I don't have any recommendations, hope this helps!

-Ali

On Wed, Oct 31, 2012 at 7:16 AM, Ronny Ager-Wick <ronny at ager-wick.com>wrote:

>  Hi, Chris.
>
> I have some experience with this, as I had a dedicated tester while
> developing one of my bigger projects. To be honest, I wasn't too impressed
> with the result. My tester was a remote worker, but I worked with her in
> the office for a while too, and it made no difference.
> The issues I discovered was that the tester wasn't a user, and thus had
> little knowledge and interest in the work the users are performing. This
> makes it very hard for this person to use it like a user, and, having more
> than enough just understanding what the real users are actually using the
> system for, somewhat challenging to try something provoking it that may
> make the system fall over. Of course one person is hardly representative. I
> might just have gotten the wrong person for the job, but it did wake me up
> to the fact that testing is actually not easy to find good people for this
> task. Standard testing, as in "we've just made this, and it differs from
> the old functionality the following way, can you test?" was fine, but
> frankly, that's nearly useless testing. If you can describe exactly what to
> test, then you've practically tested it already... I found myself to be the
> one finding most of the obscure or complicated bugs that the developers
> didn't find themselves. I believe it helps a lot to be a developer. We kind
> of instinctively know what can make things blow up... probably because
> we've done these types of mistakes before. And being a software architects
> I guess helps as well, as we're used to understanding use cases. I think
> most software architects find it much easier than the general population to
> put themselves in someone else's situation, imagining how they would use
> the system. That should be a minimum requirement for any tester, but I
> believe that ability is quite rare.
> A tester who can write tests - that would also be interesting! But that
> practically means the tester must also be a developer.
>
> So my advice is then, hire an experienced software architect an/or
> developer to do the testing - if you can find one willing of course... But
> even if you can, that may be a bit expensive.
> On the other hand, I've found actual users to be of great value for
> testing. Two-three real users can find an amazing number of issues which we
> developers have overlooked, regardless of how good we are at looking at it
> from their point of view. Preferably pick someone that seem interested in
> the development - typically the ones with lots of ideas for improvement (or
> lots of complaints), and try to get users that do different jobs (if the
> system does more than one thing of course), so you get a representative
> selection of them. However users (non-developers) are often not very good
> at anticipating future issues, possibly because of lack of technical
> knowledge. They use the software, and if it works for them there and then,
> it's fine with them. But I've experienced a few times that I have had a
> nagging feeling that a certain thing is bound to create issues in the
> future, which users non-devs wouldn't even think of, or even argue will
> *not* create issues. Most of the time, these turn out to be issues in the
> end. Architect/Developer's intuition maybe? Perhaps a combination of users
> and developers would be ideal for testing?
>
> Hope this was at least a bit useful :)
> Cheers!
>
> Ronny.
>
>
>
> On 30/10/12 07:13, Chris Adams wrote:
>
>  Hi guys,
>
>  We're looking into hiring a tester to work with us in on a project we've
> been building for the last 6 weeks or so,  to help catch bugs and issues
> before they make it to production on a Rails app we're working on at AMEE.
>
>  We're working on an app that's fairly well protected by tests, but has a
> few complex ajax interactions that keep catching us off guard as we
> develop, so we're looking for someone who is particularly good at ferreting
> out these kinds of issues.
>
>  The app is pretty small, so I assume we'd be looking for someone who
> might be available on a freelance basis.
>
>  However, I haven't really worked with dedicated testers before, so this
> is fairly new territory for me - does anyone on list have any
> recommendations of people you've worked with, or any advice on working with
> testers like above?
>
>  Thanks,
>
>  Chris
>
>
>
>  --
> Chris Adams
> mobile: 07074 368 229
> skype: chris.d.adams
> twitter: mrchrisadams
> web: http://chrisadams.me.uk
>
>
>
> _______________________________________________
> Chat mailing listChat at lists.lrug.orghttp://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
>
>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20121031/cb6f7dac/attachment-0003.html>