Risk Driven Testing
|
|
 What are some factors that you could use in estimating the Likelihood of a piece? For example, one would be interactivity. Does it interact with a lot of other pieces, especially in real time? Or does it just kind of sit there and do its thing when called upon occasionally? Obviously, there are a lot more things that can go wrong with a piece that's highly interactive, especially if it depends on uncontrollable outside factors like loading or network response time. How would size affect your estimate of Likelihood? How about past history, if any, for the piece in previous versions? List as many things as you can.
Previous Question Next Question
How could you use your old Bug Reports to get a better handle on Risk? . Previous Question Next Question
Suppose you only had one more hour left to do testing. There are two different pieces that you could test, but you only have time for one of them. The both have the same Risk, but one is high Impact while the other is high Likelihood. Which one would you use the time on?
Previous Question Next Question |
|
|
|
Some factors that people have found useful for estimating Likelihood include: |
|
|
|
|
|
|
|
|
|
|
|
Size Bigger pieces tend to be less reliable than smaller ones. Hence they have higher Likelihood estimates. |
|
|
|
|
|
|
|
|
|
|
|
Complexity Is the logic complex? (A tricky tax calculation, say, or an update of multiple databases?) If so, it's going to need more testing than some other pieces. |
|
|
|
|
|
|
|
|
|
|
|
Technology Does this piece use technology such as Java, that's relatively new? Or is it new to the developers? Are you, heaven forbid, a beta site for something called, for example, D++? |
|
|
|
|
|
|
|
|
|
|
|
Interactivity Discussed previously. The more interfaces there are, especially if things happen in real time and events are generated by things not under the piece's control, the more testing the piece is going to need. |
|
|
|
|
|
|
|
|
|
|
|
History If this piece existed in a previous version of the system, or is used in other systems, what's its track record? Was it stable and trouble-free, or was it a problem? |
|
|
|
|
|
|
|
|
|
|
|
Who wrote it? Did this piece come from experienced developers that you have a lot of confidence in? Or is it a product of mostly trainees? Or an outside vendor you know little about? These are some ideas to help you get started; we'll deal with Likelihood again in the next lesson. |
|
|
|
|
|
|
|
|
|
|
|
|
|
If you save your old Bug Reports (Trouble Reports, Problem Reports, or whatever you call them), you can use them the next time you have to test major work on the system. Likelihood numbers, unlike Impact, are not static in time. If we're nervous about a particular piece of the system we'll initially give it a high Likelihood number, raising Risk. But if it works well for two years, the next time around we'll lower the number accordingly. Similarly, if a piece that we thought we could trust generates a ton of Bug Reports, its Likelihood of causing problems in the future will go up. So save those old Bug Reports. At a minimum they should be linked to the piece of the system where the bug was found, and weighted by the severity of the bug. (By the way, people get some very creative names for bugs - "glitches", "action items", and so on. One place I visited was so nice that they called them "Suggestions". If you've heard an interesting euphemism for a bug, please send it along in your next email.)
Most people would choose the high Impact piece. You could make a case that you're more likely to find a bug with the high Likelihood piece, but it's usually more important to prevent high Impact disasters, even if it's less probable.
|
|
|
Images digitally watermarked. |