Information Technology Dark Side

Struggles of a Self-Taught Coder

Information Technology Dark Side header image 2

10 Seconds… Go: Exploratory Testing in a Nut Shell

June 17th, 2010 · 4 Comments

The Doit

Step 1: Take a tour of the feature
A tour is an exploratory testing exercise that helps you create a mental model you can use for planning your testing where you simply clicky around the feature and get a sense of what it does. A typical tour is aimed at answering questions like the following:

  • What does the feature do?
  • How does the feature vary by user?
  • What other features are related to this feature or impact the way it operates?
  • What capabilities are included in this feature?
  • What setup activities are required to make the feature operational or to customize it?

It’s important to take notes during your tour as you find the answers to these questions. These notes will provide the information you need for the next step: writing session charters.

Step 2: Write some session charters
Writing session charters is easy – all you really have to do is capture the mission of the charter. A mission is what you are trying to accomplish in a 30-90 minute testing session. Examples of missions include:

  • Try to find bugs in the happy path of adding a new person
  • Look for XSS bugs in the event data entry views
  • Look for bugs creating forms using IE7

Use your notes from step 1 to create these charters. You can record them in a spreadsheet, a wiki page, a notebook, whatever. Just leave some room for notes.

Step 3: Test away
Doit! Grab a charter and start testing. As you test, try to think of different variations of what you are doing that might matter and try those too. If you think of something else that should be tested that isn’t captured in the charters you’ve already written, pause for a minute and write a new mission down. Make notes of any bugs you’ve found, but don’t go submit a bug report yet.

If you encounter something related to the mission you’re on that looks strange or interests you for whatever reason, focus on it for a while until something else distracts you (that is consistent with your mission of course). Following your instincts in this way helps keep you mentally engaged and avoid “brain-dead testing.”

Step 4: Isolate and document bugs
After you’ve done a charter or two, go back over your notes and highlight any bugs you’ve found. Take each one and try to isolate them. An isolated bug is one for which the simplest path to repeating it reliably has been identified. Once you get there, then document it in whatever way your team reports bugs.

There are good bug reports and there are bad bug reports. Good bug reports:

  • Include information for reproducing the bug
  • Clearly identify the part of the system in which they occur
  • Clearly identify the outcome of the bad behavior (system crash, user confusion, etc)
  • Include the reason you think it is a bug (see oracles below)
  • Include supporting information such as screenshots and stack traces

Step 5: Take a nap

Testing this way requires energy. Most testers can only manage 3-4 charters per day before their brains start to melt and their productivity dips. It’s important to take breaks between charters and mix in meetings (the few we have) as well as other work. For instance, if you are planning on doing 3 charters, an expense report, a call with your boss, lunch, and some development, don’t plan on doing all 3 charters in a row after lunch. Instead, mix them in throughout the day to give your brain some time to recover.

Tips

Stay out of ruts

Use your domain knowledge
In TriSano, this means knowing things like

  • which diseases occur most frequently
  • which forms are longest
  • what reports are used for
  • how tech-savvy the users are

This could be different for your app, but what matters is that you have some sensible understanding of the problem your solving and how the solution is used in the real world. When you test, take this knowledge into account and put it to use by modeling real scenarios in your testing.

Pretend to be someone else
Here are some people you can pretend to be:

  • A slow reader
  • An angry user
  • A malicious user
  • A new user
  • A distracted or bored user
  • A user with a disability

Use oracles to find bugs

An oracle is a theory of error that tells you something is wrong. Here are the oracles I use most frequently, captured by the mnemonic “HICCUPPS”.

A bug could anything that is inconsistent with one of the following:

  • History of the product
  • Image of the product
  • Claims made about the product
  • Comparable products
  • User expectations
  • Purpose of the product
  • Product itself (i.e. similar things work differently elsewhere)
  • Statues or internal standards

For more information on HICCUPS, check this out: http://www.testingreflections.com/node/view/2635

If you enjoyed this post, make sure you subscribe to my RSS feed!
Stumble it!

Tags: Uncategorized

4 responses so far ↓

Leave a Comment