An Exercise from Matt Heusser

 The Exercise

  Matt Heusser, asked on twitter if anyone was up for a "test thinking exercise" and I accepted. He asked me to read this Softengi white paper on Testing in Scrum and asked, "What do you think of their advice? Is it GOOD?" Good in this case is a bit subjective, and there is nothing saying what about it is supposed to be good (i.e. their approach to testing is good, the spelling, punctuation and grammar is good etc.) So my approach was to read it and pick out where my experience differs. It is presented as an experience report, not a One True Way and should be taken as such. This means I will not be answering the first question, since I don't see this paper as giving advice, as such.

  One last point to make before we start, I know nothing about the company, the processes or the people that work at Softengi. All my impressions, thoughts and assumptions come from my own experiences and what I've read within this paper.

Thoughts on the paper 

  The first thing that stood out to me was the first line:
"It is a commonly held belief that testing is useless"
Commonly held by whom? It isn't a common conception from people in the places I have worked, either developers, project managers or senior managers. It is a fairly sweeping statement to make without a citation.

  On a similar note, there was the line:
"a short-cut today raises the probability of a low-quality, error-filled solution tomorrow"
which got a 'hell yeah' from me. But I realise I have the same problem I've just criticized the writer for. I have no independent study I can point to that supports that statement. Whereas the previous line goes against my experience, therefore grates, this one bolsters my own view points. If I wasn't doing this exercise, I might not have noticed I was falling into the same trap.

  Whilst we are on the subject of lines that stand out, in reference to demoing to the customer:
"In the classic Scrum approach, the QA team does it." In the classic approach I know, the team does it.

  There are a couple of places I took exception to the wording that was used within the paper. For example, where Software Development is used in such a way that doesn't include testing. That is not development in my mind, that is just programming. Similarly, it talks about testing finished parts of the software. I'm sorry but in my mind, if it hasn't been tested, it is not finished.

  It exemplifies just how difficult it is to talk about estimation without referring to time, since it has the line "Our team measures tasks not in hours, but in Story Points", then goes on to talk about how many hours particular task sizes are. Then "the team members provide the customer with real timings for completing each task", which is basically a time estimation.

  Before I get onto my biggest issue with the paper, let me reiterate that this is an experience report, so is a description of what Softengi are doing and presumably is working for them. However, just because it works for them does not mean I would recommend this approach to others, and the reasons why are summarized by Pic 1.


  Everything about this screams that the process is not how I would describe an agile approach, but is in fact a mini-waterfall development process. It is telling that they state "at least 40% of sprint hours should be dedicated to functional testing and stabilization". Note this is specifically the end of the sprint (Bug fixing in last week of sprint. "~40%, if we have a three weeks[sic] Sprint"). And there is a hardening sprint, which to me is a strong indicator that your in sprint testing is lacking something, and is more mini-waterfall. I get the impression they took how they tested and forced that into scrum, rather than adapting how they tested to best fit scrum. I'm not saying it is what happened, but it is the impression I get from this paper.

Conclusion

  So is it good? Were someone talking to me about how their company was transitioning to using scrum, and they were having trouble testing within that framework, I would not point them to this paper. I think that sums up my view.