[LRUG] Looking for recommendations for a tester on a Rails project

Wed Oct 31 02:56:44 PDT 2012

On 31 Oct 2012, at 07:51, Najaf Ali wrote:

> Hi Chris + Ronny,
> 
> Just another data point, but I've had a somewhat different experience to Ronny working with dedicated QA. At the one company where this was done we had one dedicated tester per every five or six developers. They had two basic functions for a given user story:
> 
> 1. Make sure that developers had implemented the acceptance criteria.
> 2. Break the shit out of everything.
> 
> While we really should be doing the 1 ourselves (or preferably automating it), having staff whose sole purpose is to break your web app in new and creative ways made a big difference to the quality of the finished software. Examples might be:
> 
> * If I get to the order confirmation screen, then edit my order in a new tab, then switch to the US store and complete my order, should I be paying in GBP?
> * Localization in Simplified Chinese breaks the design on the FAQ page.
> * If my address_line_1 is <script>alert('gimme da cookies!');</script> then the code runs on the order confirmation page.

There's a name for that: Exploratory Testing. It's good stuff.

The danger of slapping the QA label on one member of the team is that the rest of the team stop caring about quality so much. Better to make sure that person is viewed as a 'quality evangelist' or 'quality coach'. That way their job is to help the rest of the team up their game, rather than being the person who does all the testing chores nobody else has time to do because they're too busy writing bugs.

> 
> These are typically scenarios that your automated acceptance tests won't pick up.
> 
> They were also responsible for regression testing the most important flows though for the most part they automated this.
> 
> Note that this is a different thing entirely to UX testing, which is for finding out "what to build/change" rather than "how what I've already built is hilariously broken".
> 
> Afraid I don't have any recommendations, hope this helps!
> 
> -Ali
> 
> On Wed, Oct 31, 2012 at 7:16 AM, Ronny Ager-Wick <ronny at ager-wick.com> wrote:
> Hi, Chris.
> 
> I have some experience with this, as I had a dedicated tester while developing one of my bigger projects. To be honest, I wasn't too impressed with the result. My tester was a remote worker, but I worked with her in the office for a while too, and it made no difference. 
> The issues I discovered was that the tester wasn't a user, and thus had little knowledge and interest in the work the users are performing. This makes it very hard for this person to use it like a user, and, having more than enough just understanding what the real users are actually using the system for, somewhat challenging to try something provoking it that may make the system fall over. Of course one person is hardly representative. I might just have       gotten the wrong person for the job, but it did wake me up to the fact that testing is actually not easy to find good people for this task. Standard testing, as in "we've just made this, and it differs from the old functionality the following way, can you test?" was fine, but frankly, that's nearly useless testing. If you can describe exactly what to test, then you've practically tested it already... I found myself to be the one finding most of the obscure or complicated bugs that the developers didn't find themselves. I believe it helps a lot to be a developer. We kind of instinctively know what can make things blow up... probably because we've done these types of mistakes before. And being a software architects I guess helps as well, as we're used to understanding use cases. I think most software architects find it much easier than the general population to put themselves in someone else's situation, imagining how they would use the system. That should be a minimum requirement for any tester, but I believe that ability is quite rare.
> A tester who can write tests - that would also be interesting! But that practically means the tester must also be a developer.
> 
> So my advice is then, hire an experienced software architect an/or developer to do the testing - if you can find one willing of course... But even if you can, that may be a bit expensive.
> On the other hand, I've found actual users to be of great value for testing. Two-three real users can find an amazing number of issues which we developers have overlooked, regardless of how good we are at looking at it from their point of view. Preferably pick someone that seem interested in the development - typically the ones with lots of ideas for improvement (or lots of complaints), and try to get users that do different jobs (if the system does more than one thing of course), so you get a representative selection of them. However users (non-developers) are often not very good at anticipating future issues, possibly because of lack of technical knowledge. They use the software, and if it works for them there and then, it's fine with them. But I've experienced a few times that I have had a nagging feeling that a certain thing is bound to create issues in the future, which users non-devs wouldn't even think of, or even argue will *not* create issues. Most of the time, these turn out to be issues in the end. Architect/Developer's intuition maybe? Perhaps a combination of users and developers would be ideal for testing?
> 
> Hope this was at least a bit useful :)
> Cheers!
> 
> Ronny.
> 
> 
> 
> On 30/10/12 07:13, Chris Adams wrote:
>> Hi guys,
>> 
>> We're looking into hiring a tester to work with us in on a project we've been building for the last 6 weeks or so,  to help catch bugs and issues before they make it to production on a Rails app we're working on at AMEE.
>> 
>> We're working on an app that's fairly well protected by tests, but has a few complex ajax interactions that keep catching us off guard as we develop, so we're looking for someone who is particularly good at ferreting out these kinds of issues.
>> 
>> The app is pretty small, so I assume we'd be looking for someone who might be available on a freelance basis. 
>> 
>> However, I haven't really worked with dedicated testers before, so this is fairly new territory for me - does anyone on list have any recommendations of people you've worked with, or any advice on working with testers like above?
>> 
>> Thanks,
>> 
>> Chris
>> 
>> 
>> 
>> -- 
>> Chris Adams
>> mobile: 07074 368 229
>> skype: chris.d.adams
>> twitter: mrchrisadams
>> web: http://chrisadams.me.uk
>> 
>> 
>> 
>> _______________________________________________
>> Chat mailing list
>> Chat at lists.lrug.org
>> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> 
> 
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
> 
> 
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org

cheers,
Matt

--
Freelance programmer & coach
Author, http://pragprog.com/book/hwcuc/the-cucumber-book
Teacher, http://bddkickstart.com
Founder, http://www.relishapp.com/
Twitter, https://twitter.com/mattwynne

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20121031/1a6cec0f/attachment-0003.html>