I attended the SIGiST yesterday entitled “A Testing Toolbox” and found it to be, as usual, an excellent conference with lots of thought-provoking talks.
The Irrational Tester
The opening keynote from James Lyndsay was focussing on the biases we all have built into us and need to avoid to be effective testers.
James used the headings:
Confirmation bias – where we find what we expect to find so don’t look for situations where we might find unexpected behaviour;
The “Endowment effect” – where people will often demand much more to give up something they have acquired than they got it for;
A “failure to commit” – if work is broken up into small chunks with deadlines set for each we are more likely to make progress on our projects than if there was a single deadline set for the end of the project;
Illusion of control – where we fool ourselves into thinking that we have found the only cause of a problem and don’t think about whether there may be anything else that might cause the same defect; and
Broken windows – where acceptance of minor bugs might lead to an acceptance of other much more serious bugs in the system.
Talks where human psychology is discussed – especially how it affects groups of people and how they interact and behave – are really interesting to me and I thoroughly enjoyed James’ talk.
Application Security Awareness
The second talk was by Martin Knobloch from OWASP.org entitled “Application Security Awareness”. OWASP.org is a great starting point for getting information on security testing; it contains extensive documentation, code projects, conference details and is made up of over 100 Chapters worldwide (and still growing).
The main thrust of the talk was encouraging people to identify and thoroughly understand the weakest link in their systems. Very often this is not a technical weakness: it can easily be a process or ‘people’ weakness that leads to systems being exploited.
We need to beware of the dangers of creating the illusion of security but not actually doing anything to really make our applications secure.
All applications have the same issues – the techniques discussed on the OWASP website can be applied equally to ‘normal’ applications as to websites and web applications.
Delight Your Organisation, Increase Your Job Satisfaction and Maximise Your Potential
The next talk was entitled “Delight your organisation, increase your job satisfaction and maximise your potential” and was given by John Isgrove. This talk focussed on what characterises an Agile project and what does not and then discussed a methodology called DSDM Atern. I had previously heard of DSDM but I had not encountered the ‘Atern’ variation on the theme.
DSDM Atern provides a framework for the management and delivery of an entire project with guidance for managers. Scrum provides a one-size-fits-all process but contains little guidance for managers – yes, there are the Scrum Masters, but are they always the decision makers?
There seem to be a lot of benefits for organisations adopting the approach and I intend to study it a bit more and find out what other people within my organisation know about it and whether any of its principles can be adopted by us.
What I found interesting was the way the Features, Quality, Time, Cost triangle is turned on its head. In a traditional environment Features and, to a certain extent, Quality are fixed and the Time and Cost elements are flexible. With DSDM Atern, Time, Cost and Quality are all fixed and the Features to be implemented are flexible.
I found myself agreeing with James Windle’s comment at the end that it was one of the best talks I had heard on agile methodologies and the difficulties that must be overcome. I am just sorry that there is neither the time nor the space to put a lot of detail on the talk in this blog post.
The excellent SIGIST lunch followed this talk and, as usual, it was great to network with other testers and see the tools and services exhibition.
Lessons From Data Warehouse Testing
The Sharepoint after lunch was interesting. Peter Morgan shared his experiences of testing data warehouse applications
The New Role of The Tester: Becoming Agile
Stuart Taylor was next up with an inspirational experience report of how his organisation made move to an agile process.
Wholesale changes to the working environment (even moving from curved desks to straight desks arranged so paired working was easier and people could talk across the table) were made, testers were involved throughout the design, development and delivery processes, automating as much as possible in Java using test driven development techniques which allowed the dedicated testers to get on with manual Exploratory Testing (the stuff we all love to do).
As a result of moving to an agile process they have seen improvements to the quality of their software, their response to changes in business need and there is much more negotiation with schedules.
How to Suspend Testing and Still Succeed – A True Story
Graham Thomas gave an account of his experiences when testing had to be suspended. Testing was suspended on this project because there was no way anything was going to be delivered with the way things were working at the time. The biggest problems were with the systems integration risk which was accepted at an earlier stage in the development process and the automation infrastructure.
Initially there was progress made: Systems Integration testing was successfully completed (or at least it was as scoped in the 50-page test strategy) but testing was held up by slippage in code delivery from development, issues with the test automation infrastructure and a qualified exit on non-functional proving of the infrastructure (it also took 250% more time than it should have done).
8 weeks into a 12 week schedule is was estimated that at the current rate of progress, UAT was going to take over a year to complete and many of the issues being found were to do with the automation infrastructure and product configuration – i.e. the systems integration risk had matured.
They held a series of workshops with all stakeholders to get an idea of what was wrong and plan a resolution that would allow a resumption of testing. Graham pointed out that it is very difficult to set effective Resumption Requirements without knowing the criteria by which testing was suspended. It was also difficult to set Suspension Criteria without knowing what was going wrong. This is at variance with IEEE 829 but, when you think about it, it is rather obvious!
So the remainder of the project was re-planned – bearing in mind that the go-live date was non-negotiable due to regulatory constraints – and amazingly the Resumption Requirements were met on time at the end of 4 weeks.
A daily war room meeting was set up at 13:00 at which attendance was mandatory for all the decision-makers and those actually doing the work. Only directly grinding out the work to achieve the project’s aims and decision-makers were permitted at these meetings.
Graham made it all sound very easy but it was clear that it was a very painful process which caused a lot of heartache and irretrievable breakdowns in the professional relationships between people.
Graham’s talk was fascinating and gave a real insight into Suspension Criteria and Resumption Requirements and the effects that suspension can have on a project.
UAT: A Game for Three Players
The final keynote talk of the afternoon was “Acceptance Testing: A Game for Three Players” by James Windle. This was another excellent talk in which James gave us a run-down on how he approaches UAT going right back to the definition of the Acceptance Test Criteria. Whilst there was nothing really ‘new’ about James’ talk it served as a very helpful reminder of this critical part of testing.
The day ended, as usual, at the Volunteer on Baker Street enjoying further networking with testers.
A big thank you to the SIGIST committee for organising the event once again and congratulations to Graham Thomas and Mohinder Khosla on their respective appointments as Programme Secretary and Secretary of SIGIST.