[LRUG] Looking for recommendations for a tester on a Rails project

Wed Oct 31 03:09:43 PDT 2012

> The danger of slapping the QA label on one member of the team is that the
rest of the team stop caring about quality so much.

My experience differs from this significantly. In practice we found we
cared about quality *more* with dedicated QA (or whatever you'd like to
call them) staff on board to bring to light issues us well-meaning
developers tend to miss. They only have time to dig deep for
the weirder issues if us devs are pretty stringent about handing over good
quality software in the first place.

After having had that experience, I tend to find the occasional security
bugs that other developers sometimes miss, but my 'quality evangelism'
pales in comparison to getting a dedicated QA to beat the crap out any new
features before they rollout to prod.

> being the person who does all the testing chores nobody else has time to
do because they're too busy writing bugs

They're not chores if they're your fulltime job! Just because we don't
enjoy doing that work doesn't mean that someone else isn't perfectly happy
doing it, doing it annoyingly well and improving the actual quality of the
software in the process.

- Ali

On Wed, Oct 31, 2012 at 9:56 AM, Matt Wynne <matt at mattwynne.net> wrote:

>
> On 31 Oct 2012, at 07:51, Najaf Ali wrote:
>
> Hi Chris + Ronny,
>
> Just another data point, but I've had a somewhat different experience to
> Ronny working with dedicated QA. At the one company where this was done we
> had one dedicated tester per every five or six developers. They had two
> basic functions for a given user story:
>
> 1. Make sure that developers had implemented the acceptance criteria.
> 2. Break the shit out of everything.
>
> While we really should be doing the 1 ourselves (or preferably automating
> it), having staff whose sole purpose is to break your web app in new and
> creative ways made a big difference to the quality of the finished
> software. Examples might be:
>
> * If I get to the order confirmation screen, then edit my order in a new
> tab, then switch to the US store and complete my order, should I be paying
> in GBP?
> * Localization in Simplified Chinese breaks the design on the FAQ page.
> * If my address_line_1 is <script>alert('gimme da cookies!');</script>
> then the code runs on the order confirmation page.
>
>
> There's a name for that: Exploratory Testing. It's good stuff.
>
> The danger of slapping the QA label on one member of the team is that the
> rest of the team stop caring about quality so much. Better to make sure
> that person is viewed as a 'quality evangelist' or 'quality coach'. That
> way their job is to help the rest of the team up their game, rather than
> being the person who does all the testing chores nobody else has time to do
> because they're too busy writing bugs.
>
>
> These are typically scenarios that your automated acceptance tests won't
> pick up.
>
> They were also responsible for regression testing the most important flows
> though for the most part they automated this.
>
> Note that this is a different thing entirely to UX testing, which is for
> finding out "what to build/change" rather than "how what I've already built
> is hilariously broken".
>
> Afraid I don't have any recommendations, hope this helps!
>
> -Ali
>
> On Wed, Oct 31, 2012 at 7:16 AM, Ronny Ager-Wick <ronny at ager-wick.com>wrote:
>
>>  Hi, Chris.
>>
>> I have some experience with this, as I had a dedicated tester while
>> developing one of my bigger projects. To be honest, I wasn't too impressed
>> with the result. My tester was a remote worker, but I worked with her in
>> the office for a while too, and it made no difference.
>> The issues I discovered was that the tester wasn't a user, and thus had
>> little knowledge and interest in the work the users are performing. This
>> makes it very hard for this person to use it like a user, and, having more
>> than enough just understanding what the real users are actually using the
>> system for, somewhat challenging to try something provoking it that may
>> make the system fall over. Of course one person is hardly representative. I
>> might just have gotten the wrong person for the job, but it did wake me up
>> to the fact that testing is actually not easy to find good people for this
>> task. Standard testing, as in "we've just made this, and it differs from
>> the old functionality the following way, can you test?" was fine, but
>> frankly, that's nearly useless testing. If you can describe exactly what to
>> test, then you've practically tested it already... I found myself to be the
>> one finding most of the obscure or complicated bugs that the developers
>> didn't find themselves. I believe it helps a lot to be a developer. We kind
>> of instinctively know what can make things blow up... probably because
>> we've done these types of mistakes before. And being a software architects
>> I guess helps as well, as we're used to understanding use cases. I think
>> most software architects find it much easier than the general population to
>> put themselves in someone else's situation, imagining how they would use
>> the system. That should be a minimum requirement for any tester, but I
>> believe that ability is quite rare.
>> A tester who can write tests - that would also be interesting! But that
>> practically means the tester must also be a developer.
>>
>> So my advice is then, hire an experienced software architect an/or
>> developer to do the testing - if you can find one willing of course... But
>> even if you can, that may be a bit expensive.
>> On the other hand, I've found actual users to be of great value for
>> testing. Two-three real users can find an amazing number of issues which we
>> developers have overlooked, regardless of how good we are at looking at it
>> from their point of view. Preferably pick someone that seem interested in
>> the development - typically the ones with lots of ideas for improvement (or
>> lots of complaints), and try to get users that do different jobs (if the
>> system does more than one thing of course), so you get a representative
>> selection of them. However users (non-developers) are often not very good
>> at anticipating future issues, possibly because of lack of technical
>> knowledge. They use the software, and if it works for them there and then,
>> it's fine with them. But I've experienced a few times that I have had a
>> nagging feeling that a certain thing is bound to create issues in the
>> future, which users non-devs wouldn't even think of, or even argue will
>> *not* create issues. Most of the time, these turn out to be issues in the
>> end. Architect/Developer's intuition maybe? Perhaps a combination of users
>> and developers would be ideal for testing?
>>
>> Hope this was at least a bit useful :)
>> Cheers!
>>
>> Ronny.
>>
>>
>>
>> On 30/10/12 07:13, Chris Adams wrote:
>>
>>  Hi guys,
>>
>>  We're looking into hiring a tester to work with us in on a project
>> we've been building for the last 6 weeks or so,  to help catch bugs and
>> issues before they make it to production on a Rails app we're working on at
>> AMEE.
>>
>>  We're working on an app that's fairly well protected by tests, but has
>> a few complex ajax interactions that keep catching us off guard as we
>> develop, so we're looking for someone who is particularly good at ferreting
>> out these kinds of issues.
>>
>>  The app is pretty small, so I assume we'd be looking for someone who
>> might be available on a freelance basis.
>>
>>  However, I haven't really worked with dedicated testers before, so this
>> is fairly new territory for me - does anyone on list have any
>> recommendations of people you've worked with, or any advice on working with
>> testers like above?
>>
>>  Thanks,
>>
>>  Chris
>>
>>
>>
>>  --
>> Chris Adams
>> mobile: 07074 368 229
>> skype: chris.d.adams
>> twitter: mrchrisadams
>> web: http://chrisadams.me.uk
>>
>>
>>
>> _______________________________________________
>> Chat mailing listChat at lists.lrug.orghttp://lists.lrug.org/listinfo.cgi/chat-lrug.org
>>
>>
>>
>> _______________________________________________
>> Chat mailing list
>> Chat at lists.lrug.org
>> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>>
>>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
>
>     cheers,
> Matt
>
> --
> Freelance programmer & coach
> Author, http://pragprog.com/book/hwcuc/the-cucumber-book
> Teacher, http://bddkickstart.com
> Founder, http://www.relishapp.com/
> Twitter, https://twitter.com/mattwynne
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20121031/a3f46ddc/attachment-0003.html>