Risk-Driven Testing

	In this lesson you'll learn:
				How to find the most important bugs first
				How the concept of Risk can make this happen
				What components to use for assigning Risk

Whenever there's too much to do and not enough time to do it, we have to prioritize so that at least the most important things get done. In testing, there's never enough time or resources. In addition, the consequences of skipping something important are severe. So prioritization has received a lot of attention. The approach is called Risk Driven Testing, where Risk has very specific meaning. Here's how you do it: Take the pieces of your system, whatever you use - modules, functions, section of the requirements - and rate each piece on two variables, Impact and Likelihood.  Want more detail about what are the pieces of your system? Impact is what would happen if this piece somehow malfunctioned. Would it destroy the customer database? Or would it just mean that the column headings in a report didn't quite line up? Likelihood is an estimate of how probable it is that this piece would fail. Together, Impact and Likelihood determine Risk for the piece. The easiest way to see this is with a picture. Let's go to the whiteboard.

		Here we set up the components of Risk in graphical form.

		When each piece of the system is assigned values, we get a plot that looks something like this.

Which quadrant's pieces need the most testing? Pretty easy.

		The pieces in the upper right hand quadrant, which have both high Impact and high Likelihood, need the most testing.

After those pieces, we need to spend time on the whole top edge. While A and B don't have high Likelihood of being wrong, if they are wrong, seriously bad things will happen. So we have to give them some attention, especially since Likelihood is an estimate.

		Each piece with high Impact needs attention, even if the Likehood value is low.

Next we test the high Likelihood pieces - everyone agrees that they probably don't work right.

		Here are the high Likelihood pieces.

Finally, of course, we've got to spend some time making sure that we've covered everything in the feature set to at least some level. For example, suppose we're testing new Mercedes as they come off the production line. We only have two hours to test each car. We could easily justify spending all that time testing the brakes. After all, if the brakes don't work right, there'll be a really large Impact - literally, in this case. Therefore, while it's important to see that the car runs, it's even more important to be sure that if it does run, it will stop.

Just as in testing software, we say that something "works" if it works every time, under all combinations of conditions. So we could soak the brakes with water, freeze them, overheat them, and so on - all the standard engineering "shake and bake" things. We could spend the whole two hours doing this. But we also have to make sure that the car starts, steers, check that the lights, horn and wipers work, etc. Eventually, if there's time, we'd like to check that the FM Seek button on the radio snags the stations. Unfortunately, in the real world there's never enough time to do all the testing we'd like to do. So, as in the car example, we have to prioritize testing by Risk. That makes sure that we're doing the most important things first. But we also need to allow some time to get some coverage of the less risky features and usage scenarios.

	Summary:
				Testers with different backgrounds sometimes disagree, but experienced testers all agree on this: There's never enough time, so you need to prioritize testing by something, and that something is Risk. Risk drives good testing.
				Risk has two components: Impact and Likelihood.
				Impact is how bad it would be if a piece malfunctioned.
				Likelihood is how shaky we think a piece is.
				After testing the high Risk pieces, we need to allow some time to get some coverage on all the pieces.

Check your understanding with this exercise.

Questions about this lesson? Just . . . Ask the instructor

In the next lesson we'll see how to assign formal Risk numbers to the pieces of a system, and how to apply that to the system you're working on.

Copyright © 1998 - 2005 by The Testing Center. All rights reserved. Images digitally watermarked.

   ------------------------------------------------------------------------------------------------

Supplemental Material:

 

More detail about the "pieces" of your system: What you'll use as the pieces of your system for the purpose of Risk depends on how you view your system. If you're testing based on requirements, the testable pieces would be sections of the requirements. For example, 3.4.1.2 Update the Time of Last Access would be a piece. If you work closer to the code, your pieces might be code modules. For example, a module that controls routing of output to printers would be a piece. If you work with an object-oriented system, your testable pieces are usually the methods of objects. Don't worry about this if you're not sure what pieces to use for your system. If it isn't clear by the end of the lesson, just send me email in the space provided there. I'll be glad to talk to you about it.

 

  End of Supplemental Material   

------------------------------------------------------------------------------------------------

Copyright © 1998 - 2005 by The Testing Center. All rights reserved. Images digitally watermarked.