Archive for the ‘Usability’ Category

User Experience Testing: Communicating Through the User Interface

20 December, 2010

One of my many interests is how the individual parts of systems – whether they be software-driven or not – communicate with each other.  A lot of time and energy is spent on making sure that the software components work together in different situations but how much time do we devote to making sure that the systems all work together cohesively to form an entire process?  How much time is spent making sure that the process itself can work correctly?

One of the things that I think we need to plan more time for is testing the way humans interact with systems.  I know that this is not easy because time is short and there is a lot of pressure to make sure the computer software side of the system is working correctly – the rest can be handled with training so the argument goes – but I think as testers we should keep bringing the human side of systems to the table in meetings and discussions about the projects we are involved in.

The human side of systems is something that ‘just happens’ when everything is going well but when it all goes wrong the results can be spectacular.  My favourite example of this is the London Heathrow Terminal 5 opening debacle.  A lack of familiarity by staff and passengers about car park locations led to baggage build up because the people were not in place at the right times to move bags around the baggage system.  This in turn caused a heavy load on the baggage belts leading to a failure of the automated baggage delivery system and so on…  Testers, as the eyes and ears of a project, should be vigilant for situations that no-one else has thought of and raise them.  Of course it is possible that the testers on this project had asked these questions and nothing was done about mitigating the risks, but everybody did seem to be taken by surprise at the turn of events on T5’s opening day…

Let’s move on now to another aspect of human-computer interaction: messages and warnings.  I am sure we have all been bemused by the sight of an error message that just says: “An error occurred.” However, put yourself in a user’s shoes for a moment and think about how you would react to seeing the following (I have pulled this from my Application Event Log but the text is pretty much as I  remember it appearing on screen in the form of an error message):

Faulting application name: OUTLOOK.EXE, version: 12.0.6539.5000, time stamp: 0x4c12486d

Faulting module name: olmapi32.dll, version: 12.0.6538.5000, time stamp: 0x4bfc6ad9

Exception code: 0xc0000005

Fault offset: 0x00051c7c

Faulting process id: 0x1a18

Faulting application start time: 0x01cb9c9e50018c07

Faulting application path: C:\Program Files\Microsoft Office\Office12\OUTLOOK.EXE

Faulting module path: c:\progra~1\micros~2\office12\olmapi32.dll

Report Id: 0c733b39-0896-11e0-b5bf-00197ed8b39d

The practice of delivering such ‘techie’ messages to end users is common-place but in my opinion it is a bad approach.  Receiving messages like this is completely bewildering for novice computer users who are likely to panic and do something that really messes things up.  In my case I knew that it was an add-in that I had installed which was incompatible with Outlook 2007 and did not panic – I understood what I had to do and I got on with it but it made me think of my less experienced friends and colleagues and how disconcerting such a message would be for them.

Let me give you another example from my Application Event Log:

The application (Microsoft SQL Server 2005, from vendor Microsoft) has the following problem: After SQL Server Setup completes, you must apply SQL Server 2005 Service Pack 3 (SP3) or a later service pack before you run SQL Server 2005 on this version of Windows.

I would like to encourage you all to think carefully about the wording of error messages and warning that are displayed to users.  The above is not a ‘problem’ at all; I need to do something else before I can run SQL Server 2005 and there is no cause for alarm.  It might be argued that someone seeking to use SQL Server 2005 is bound to be a competent computer user and therefore does not need much help but I beg to differ.  I might have been given a task to do for which I am completely out of my depth and I do not need to be panicked further.

There is a fine balance to be reached between being able to give enough information so support professionals and developers can debug and understand how to fix a problem (which Microsoft may have done with their message from Outlook above – assuming they are all well versed in hexadecimal) and being informative to users.

If we get the user experience right we stand a much better chance of designing and implementing a system which really does work efficiently because people will not be wasting countless hours trying to understand cryptic messages coming back from the system; they will be less frustrated; and everyone will have a better perception of the system and the organisation that is using it.

I was in a supermarket a few weeks ago and I heard the remark from a fellow customer that “there are always problems at the tills here – nobody seems able to work them”.  Standing in the queue I could see where that perception would come from: two till operators and a supervisor were needed to make sense of a message that had come up on the screen.  Testers should be making more noise about human-computer interaction and user experience problems that they can foresee for the future good of our craft.

This is an area that I am striving to get better at and I hope there are other testers out there who give serious consideration to the user experience they are giving in their systems.


Usability and Process Control

23 April, 2010

A blog post by James Christie ( has prompted me to post a few thoughts about recent experiences I have had where illogical (at least to the user) processes have apparently gone unchallenged.

The Utility Company

I received a letter one morning from my electricity and gas supplier to express their sorrow that I was leaving them – but I was not intending to leave them at all!  I contacted the company and explained the situation.  The operator looked up my account details and said that there was no record of me leaving them.

A couple of months later I attempted to log on to my online account management system but I was unable to do so because I no longer had an account with the company.  I contacted the helpdesk again and was told that another utility company was supplying my electricity (not my gas – that was still with them).

What followed was a lengthy process to get my account reinstated.  I became very interested in establishing what went wrong when the same thing happened a second time.  This is what I found out:

  • A representative of another company had mistyped the meter number of a property.
  • At no point was a check carried out to make sure the installation and billing addresses matched.
  • My energy company had no means of verifying that the details matched because insufficient information was sent to enable them to do so.
  • Energy companies are reliant on people noticing that they have been erroneously transferred and informing them.
  • Having erroneously transferred my account to another supplier it was impossible for my original dual fuel account number and information to be reinstated because only the one fuel type had been switched.  This is despite a regulatory requirement for them to do so.

I would be very interested to hear which testing techniques were employed by the suppliers and which test types they used.  I suspect there were a lot of automation and manual scripts with very little in the way of exploratory testing carried out.

To me this emphasises the importance of elements of process auditing to testing.  As an ISO 9001 auditor I have to examine the business processes my employers have in place.  Testers need to think about the wider processes of which the software they are testing is just a part.

The Mobile Phone Company

I have experienced the opposite effect to James with my present mobile phone operator.  Far from being able to have an account with multiple telephone numbers and/or services, I have to set up a separate on-line account for each number/service.

Worse still the website for the company concerned is only fully functional in Internet Explorer.  There is a help forum which does not work at all in Firefox; one of the most commonly used web browsers.

Registering numbers/services on your account involves:

  • Entering your details on the website
  • Submitting the form
  • Waiting for a text message to come through to the telephone
  • Entering the code into the website – whilst hoping the session and/or code has not timed out in the meantime.

Once this has been done, the number/service can be administered online.

The other day my father purchased a new telephone from this company and tried to go through the registration process.  He was unfamiliar with the handset and is not adept at text messaging.  This meant he found the whole process time-consuming and frustrating and in the end called on me for assistance.

I can understand that the company wants to make sure the telephone really does belong to the account holder but there must be better ways of achieving the same effect.  Also, does it really matter that much to them?

I am sure this whole process works as designed – it sounds like a wonderfully technical solution to the desire to have users register their telephones but, again, a bit of process auditing logic could have been applied to say “this is giving a bad user experience”.