[LRUG] Testing email invitations with Cucumber

[Stephen Strudwick]

Hi Ian,

One of the things that jumps out about that feature is that you are
testing something outside your system and your control, as soon as the
email is sent your app is done with that scenario until the user
clicks the accept invitation link and hits your system again.

So I would focus testing the emails are sent correctly and invitations
are accepted correctly as different scenarios

Also with this kind of feature I would focus on real world testing and
not try to automate the whole thing. A few things I would do/think
about:

- Record stats on emails sent and invitations accepted, produce a
report on the conversion ratio, if only 50% of invitations are
accepted there might be an issue
- Do manual testing of the the whole scenario (end to end) using
different combinations of email clients and systems, some email
clients might garble the links or display horribly
- Do A/B testing on the email you sent out, see what drives conversions
- Make sure your emails are not being spammed because of something
silly in the headers or body (again you can only test this manually)

So I guess I'm saying, take your initial scenario, break it down into
the bits your app is responsible for, then do manual (end to end )
testing on the complete scenario

also never lose sight of the value you get from the testing, the value
here is making sure invitations are converted into sign ups. Also
there is still a place for old fashioned manual testing in certain
circumstances.

Stephen







On Tue, Jan 29, 2013 at 6:35 PM, Ian Kynnersley <iankynnersley at gmail.com> wrote:
> Hi,
>
> We have a Rails app which has the concept of invitations. I.e. I can invite
> you to join me in using this app.
>
> We have ended up with some pretty ugly cucumber scenarios that I am trying
> (and failing) to refactor.
>
> We are using email_spec (https://github.com/bmabey/email-spec) to do the
> email interaction stuff and it seems to work well but the scenarios end up
> pretty tortuous.
>
> Things like:
>
> Given I am logged in as a user
> When I invite someone else
> Then they should receive an email
> When they open the email
> Then they should see the subject "Invitation"
> And the contents "You have been invited"
> When I sign out
> And the user clicks the first link in the email
> Then they should be on the "accept invitation" page
>
>
> I'm paraphrasing but this is the sort of thing I'm talking about. The step
> "When I sign out" feels totally wrong here but is required otherwise
> clicking the email link will react as me, not the recipient. I'm trying to
> break the test down into discrete units but I'm struggling.
>
> My thought was to finish the first test at "Then they should receive an
> email" and started a new one that started:
>
> Given I have been invited
> Then I should have an email
> ...
>
> However, the emails are sent in the controller so there is no email.
>
> Does this indicate the email should be sent from somewhere else (in the
> model or maybe a creator class) so that my "Given I have been invited" step
> will automatically generate the email.
>
> Or should I explicitly send an email in the "Given I have been invited"
> step. This doesn't test that I am actually sending the right email in the
> controller but that could be dealt with elsewhere.
>
> Has anyone done this? It is pretty standard behaviour. Does anyone have any
> recommendations?
>
> Thanks
> Ian
>
>
> --
>
> Ian Kynnersley
> http://iankynnersley.co.uk | +44 (0) 7973 420 829 |
> http://twitter.com/kpopper
>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>