Archive for the ‘Uncategorized’ Category

Rapid Software Testing with James Bach – First Two Days

20 March, 2013

My employer, Allies Computing Ltd, has hosted a condensed version of this post:


Rapid Software Testing is a course I have wanted to do for a few years now having heard really good things about how it has improved various people’s testing and the way they think. We are two days in now and I am really excited about the things I have learned.

Thinking like a tester

This is a particularly challenging area for me. I tend to build ideas then narrow them down by critically analysing them however they can be quite haphazard and this could lead to an accusation that I’m not testing properly. I am going to have a go at categorising what I do in a similar way to James does with the Heuristic Test Strategy Model – where he has headings such as:

  • Structure
  • Function
  • Data
  • Interface
  • Platform
  • Operations
  • Time

I will likely have different headings (some may be the same) but by organising my thoughts and ideas in this way will give me confidence that I’m considering the right things and a much more professional image.

The other way I think this will help me is that it will channel my thoughts and make me less inclined to get stressed about what I am doing and whether I am doing it right. Critical analysis is good; it is what makes a good tester but sometimes striking a middle ground is a good thing. We spoke a lot about System 1 and System 2 thinking – with System 1 thinking we tend to be much more off-the-cuff and emotive and this is great for generating ideas quickly but the more measured, time-consuming approach is System 2.

I find this very useful because I often don’t recognise when I need to switch mode – and maybe even when the way I am thinking is counter-productive to the situation I am in.


Another big thing for me was the importance of testability and seeking things that will help me test very well. This was brought home to me during one particular exercise involving a sphere. James played a customer in a top secret problem domain and was unable to tell us things because we were not allowed to know these things! For me this is all part of honing our problem solving abilities.

A Bit of Magic

We had magic tricks to help us see the importance of having a broad mind and giving us the imagination to conceive of our ideas. It showed me the importance of what is lurking in my blind spots of which I have many. It is really important to recognise our limitations and the problems these limitations bring to us as testers.


It is really important to recognise who, what and where our oracles are as it is they which will provide some help answering the question about whether or not we have a problem. I realised that I have oracles hidden away in all sorts of surprising places!

I am really looking forward to tomorrow when we will be looking at Exploratory Testing and I am excited to play the famous dice game!


SIGiST Conference 13/03/2013

14 March, 2013

Good session at the Special Interest Group in Software Testing (SIGiST) conference yesterday run, as usual, by the BCS (British Computer Society) with a good representation of testers from across the different project lifecycles.

Matt Robson from Mastek gave an opening keynote under the title “Be Agile or Do Agile” and gave some salutary warnings on the dangers of testing becoming an ‘ology’. It is very easy to become set in our ways and dogmatic about our approaches to testing and that is harmful. To be ‘agile’ does not just mean that we adopt the Agile Manifesto ( and follow an agile approach to project management; it means we think and act in a way that embraces change and adapts to the situations we are in.

Very often we forget the ‘people’ side of software development and the example was given of a company where senior management turned round one day and said ‘we’re going to go agile and this is how you’re going to work in future’ but didn’t get the staff on board with them. The consequences on staff morale were horrendous and as a result software quality dipped.

One of the ways we can do this is to think in terms of business goals and outcomes because that has meaning for people. For instance instead of saying ‘the registration widget looks broken; I advise against going live’ approach it more as ‘we have found instances where sales staff might not be able to register new customers on our system; I advise against going live’.

What was particularly good was the talk was done with no PowerPoint slides so it concentrated the mind far more. I think this is an area that testers really have to get good in but it is also an area that can easily go horribly wrong.

Next we had a talk from George Wallace on systems challenges going from an R & D product straight into production. The project was for a very large and complex system and it was being developed in a very traditional manner with testing entering the fray late on in the product’s lifecycle. Suffice to say that testing was supposed to take 3 months and they are already 9.5 months in and still going!

Sakis Ladopoulos from Intrasoft International talked to us about what he termed project agnostic test metrics for an independent test team. Essentially this was an attempt to measure the performance of testers working on projects but do so independently of how the whole project is performing. The way this was being approached was to normalise scores for whatever was being counted across all the projects the team was involved with. The ‘best tester’ was the one who found the highest number of bugs very often as a percentage of the time they had taken.

I was quite uncomfortable with the idea of detaching testing from the rest of the project team but I am just not used to working like that.

After lunch we had a talk on website testing from Balaji Iyer from Mindtree talking about the challenges faced by modern websites. In particular there was discussion around scripting challenges and how performance can be impacted by technologies such as Ajax (used extensively by Google), Flash (for instance You Tube) and JavaScript which is often used for making sites look ‘pretty’.

Mindtree have a module currently in development that works with JMeter (a popular open source load and performance testing tool operated under the Apache banner) to help testers parameterize their requests and correlate them.

Chris Comey and Davidson Devadoss from TSG (Testing Solutions Group) then gave a fascinating talk on test strategies and how we can write a strategy which looks great on paper but does not work at all in practice because we have written our strategy looking in on testing and have not thought about dependencies on other parts of the business and how to deal with the issues that arise as a result.

It was a great talk because both test strategies that were used as examples were good strategies; they just weren’t the right ones for the job in hand. There is little point in doing a Post Project Review either if, as in the case of one of the projects, you are just going to type it up then stick it on a shelf somewhere and not learn the lessons. All the failings in one of the projects looked at had been raised in a previous ‘lessons learned’ document. Perhaps it would have been better to have called it a ‘Lessons NOT learned’ document!

The closing keynote from Martin Mudge from was great and was talking about crowd sourcing services to get testing done quicker and perhaps with greater coverage. With some audience participation from 3 people all had different paths that they would take through a functionality diagram.

The testers come in from all over the world via registration and are selected for projects based on their skills, experience and ability. Defects that are raised are all re-tested to verify that they are repeatable as recorded and genuine issues. Testers receive training materials if there are problems with their testing.

This seems to be a particularly good way for small teams wanting to quickly catch user interface issues but it would certainly be dangerous to rely on crowd-sourcing for deeper level testing (and I can just see some companies getting the impression that is the way to go)!

Overall a good conference and plenty to take away and think about.

Of Marathons and Testers

19 March, 2012

A friend of mine, Simon Alexander, has recently started training to run in the London Marathon in just over a month’s time.

He has started a fund-raising page at VirginMoney and any donations would be much appreciated by – not just by Simon and I – but the staff and children at the hospices too.

It made me think about the relationship with my own chosen craft though and the comparisons that can be drawn:


As testers it is important that we are prepared for the testing we are to carry out. We have to keep practicing and honing our skills so that when we test we do so efficiently. It is important that we understand the problem domain that we are testing (subject to the usual time constraints of course) so we have to do our research. In this regard I love James Bach’s story, in Secrets of a Buchaneer Scholar, talks about how he applied himself to learning about patents in order to win a court case.

Similarly, Simon must adequately prepare himself for the task he has set himself. It is no good expecting to be able to turn up and ‘just’ run the marathon; he has to ensure he understands the messages his body is giving him about whether he can run faster, needs to slow down, is going at a steady pace; how does he cope going up and down inclines; what should ‘steady going’ feel like. To monitor Simon’s progress see ReachforEACH.

The Marathon Itself

A marathon is a long hard slog. Just like testing persistence and dedication are key. Once we are on the trail of a bug or something in the System under Test does not seem right we need to be dogged in our determination to get to the bottom of what is going on. We need, as much as possible, to ensure we know enough to be able to help our colleagues who have to make a go/don’t go decision make that decision.

When we are testing we need to be organised in what we are doing. There are various approaches for this: we can have a mental model that we are working through; a detailed Gantt chart showing exactly how long to spend on each component of the System under Test; a mind map such as those created by Darren McMillan on a regular basis; a prose-based plan.

We also have to keep our concentration levels up. It is important that we keep our minds focussed on the task in hand otherwise we risk missing serious defects

If Simon does not apply his training and not keep going on the day he will fail in his mission to complete the marathon. He has various strategies at his disposal for completing the run for example he could gradually build up speed across the distance; run steadily for the first 25 miles and then speed up; run at a steady speed throughout; etc.

He also has to maintain high concentration levels to ensure he keeps up the pace and keeps up with his fellow runners.


Once testing has finished on a project it is important to reflect and learn lessons from the exercise. Make sure any lessons are learned that mean next time you test you do so more efficiently.

Simon needs to keep his fitness levels up so that he does not go back to square one in his trainng.

If you can please give generously and may I take this opportunity to wish Simon the very best of luck. Good luck Simon!

“Define structure”: my thoughts on James Bach’s challenge

10 October, 2010

Yesterday evening (in the UK anyway) James Bach set a challenge for Rob Lambert:

jamesmarcusbach:  @Rob_Lambert Quick tester challenge for Rob Lambert: Define “structure.” You have 10 minutes. #softwaretesting

It got me thinking too and my brain went into overdrive so I thought I would set out my own thoughts on the matter in a blog post.

In his response Rob highlighted several different types of structure and brought out that structures are in essence a ‘system’.  I like to think of structures as providing a framework or a set boundaries within which people, or things, should operate.

For example:

  • The laws of the land – a set of constraints governing how people in the country are to behave;
  • Buildings – the walls of the building define the space available to its occupants;
  • Skeletons – defines the shape and provides the basis for growth of the person or animal it belongs to;
  • DNA – defines the characteristics of the living organism containing that DNA sequence;
  • Roads – show us where we should and should not drive.

Some of these structures are more flexible than others; they are easier to change than others.  For example there are some fish and animals which, because of their DNA, can change their shape or their colouring to exactly match their surroundings and thus evade predators.  This change can happen in an instant.  Buildings can be extended but someone has to do something to make that happen and it requires hard work.  The impact of structural change can be quite dramatic.  If a road is re-routed the impact on the natural environment can be huge and get people quite upset.

Some structures are clearly vital for us: what we know of as the laws of physics attempt to document the way the universe works.  It would be horrendous if what we know as gravity stopped behaving in a constant fashion; if the earth’s rotation round the sun stopped we would have massive problems.

In computing we rely on certain structures.  For example networking: could you imagine trying to test a network application if there was no defined way of communicating between two computers and every manufacturer did something completely different?

There seemed to be a lot of emphasis at Agile Testing Days this year on education.  Our education provides us with a structure and I believe we are each responsible to ensure we keep ourselves up-to-date, that we self-education and strive to be as flexible and adaptable to our environments as possible.  We will not do this by sticking rigidly to the requirements of a particular certification body, we have to go out there and do what is right for us and for our clients.  We need to be like those creatures that can change in an instant to blend into their surroundings.

Your thoughts and comments are, as always, welcome…

European Weekend Testing 24 July 2010

25 July, 2010

I participated in my second European Weekend Testing session yesterday (Saturday 24 July 2010, EWT 28) where we were looking at teamwork – in particular working in a paired testing scenario – using the Fantastic Contraption game ( as the System Under Test.

For me the session highlighted how useful it is to be able to discuss strategies, techniques and approaches with others and is something I will take into work with me tomorrow morning.  During the discussion afterwards – the most valuable part for me at least – it became apparent that even though many of us do not pair formally with other testers and/or developers, we do on an informal basis and still derive much benefit from the exercise.

Although I did not get very far with the actual game, I still found the experience of working with other testers enjoyable.  I would encourage others – especially if you are the sole ‘tester’ in your organisation – to consider getting involved with Weekend Testing.  If nothing else it helps us learn from and encourage other craftspeople whilst sharing knowledge and experience for the good of the community.

Many thanks to Markus and Anna for facilitating the session and to the other testers who joined us from around the world for a great learning experience.  Commitments permitting, I hope to join you next week.