In this lesson you'll learn:
				Why we need to focus on one output at a time
				How to find the dependencies for that output
				How to automatically verify that you've got the dependencies right

The Place To Start The most important thing when designing a test is to select a single output to test and construct the tests for that one output. The output could be data, such as a dollar amount, or it could be an action, such as "save the image" or "close the liquid sodium valve"". As you've probably seen, it doesn't work to try to design a single test that verifies that a whole report, for example, is correct. Something that comes out of a computer is only correct if it's always correct, under any circumstances. There are too many possible combinations of input values to draw this conclusion from a single test for a whole report. And even for just one output, we'll need a number of tests to verify that it always works. The Pacific Ocean Problem showed us that there are too many tests to perform for any real-world piece of software. Soon we'll see how to prioritize out testing so that the most important things get tested first. For now let's look at setting up a test.

Again, we're off to the whiteboard.

		Some inputs and outputs to a piece of code.

In Board 1 we see a piece of code with inputs a, b, and c. It produces the outputs x, y, and z. This code could be a small, "unit test" module or a complete large system. It doesn't matter. We only care about what goes in and what comes out. The outputs could be either data or actions, but for now we'll assume they're data. Suppose output y is the amount on a payroll check, and that it's more critical than either output x or z. So we focus testing on it first.

		We select y as our output to test.

Here's a critical step: nailing down the dependencies. For our chosen output, we need to know which inputs it depends on. Suppose output y depends on inputs a and c, but does not depend on input b, as shown on Board 3 below. If we know that, we can see how to construct the test cases for output y. We need to vary the inputs a and c through the right combinations of their possible values, but we don't have to generate test cases for different values of b.

		Crucial information: the inputs that the selected output depends on.

	Several important questions about dependencies:
				What good are they?
				Where do we get the dependency information?
				How do we know we've got it right?

Let's take these one at a time. 1. What good are they? First, it's important to know the dependencies of the outputs on the inputs because then we can set up tests that vary just the inputs that count, and do that in a systematic way. This means that we won't have extra, redundant test cases and ensures that we won't miss any, either. Note that we don't need to understand how an output depends on its inputs - just what the inputs are. This is much simpler to determine. In fact, analyzing exactly how the outputs depend on their inputs (in a complete, Boolean sense) leads to an impractical mess known as Cause and Effect Graphing. If you're curious, here's more than you probably wanted to know about it. We're not going down that road. In the next lesson, however, we will cheat just a bit and look at the dependencies in a little more detail, because we can substantially decrease the number of test cases this way.   2. Who knows the dependencies? The best source of dependency information is the users. Just ask them "Which inputs could affect the value of this output?" They'll know the answer: "Well, it could depend on this and on this and, on days when it's raining, it can sometimes depend in a funny way on ... etc." The only problem here is that they generally know so much that they'll want to tell you exactly how it depends. Be polite but feel free to stop writing at that point. If you're lucky enough to have specs, they also contain the dependency information, but it's usually difficult to dig it out. Finally, one place we don't want to get it from, if we can possibly help it, is by reading the code - that's what we're testing! For each output you'll be testing, write down the inputs that it depends on, and put this information in a file or little database. It's one of the key items we'll use to construct the test cases systematically. 3. How do we know the dependencies are right? After all, if they're wrong, our test cases may be flawed. In older, poorly documented systems, there are probably several important outputs that we're nervous about.

	That's where the trick of "twinkling" comes in: To check the dependencies of an output on a set of inputs have a little program or script written to do the following:
				Run the code and save the answer.
				Set things up to vary the "shouldn't matter" inputs through random values within the valid ranges. (This is the "twinkling" part.)
				Have the computer compare the answer each time to the stored one. If these varying inputs truly don't matter, it should always get the same answer.
				If the answer is different, log the fact that the output does actually depend on one of the "shouldn't matter" variables.
				Go to lunch while it keeps running. Depending on how long a lunch you take, you can be certain to any desired level of confidence that the output indeed depends only on the variables you thought it did. And if it does actually depend on something else you'll know that, too.
				Using this method you'll be able to check your dependencies to any desired confidence level.

Now, having focused our attention on the most important out, having found what it depends on, and having verified that dependence, we're ready to start designing the test cases - and only the test cases - that we'll need.

	Summary:
				Pick a single output to test.
				Find out what inputs can affect that output.
				If you're not sure you've got all the dependencies, use twinkling to double check.

Check your understanding with this quick exercise.

Questions about this lesson? Just . . . Ask the instructor

Now that we know the inputs for our chosen output, we're going to see how to exploit the fact that not all inputs are alike to reduce the number of test cases.

 

Copyright © 1998 - 2005 by The Testing Center. All rights reserved. Images digitally watermarked.

------------------------------------------------------------------------------------------------

Supplemental Material: (The Basement)

Cause and Effect Graphing Briefly, Cause and Effect Graphing is a technique for creating test cases by looking at all the possible ways that inputs can combine to produce outputs. The upside is that it's easy to automate this, and there are several packages on the market that do that. The downside is that it generates zillions of "test cases" (because of the Pacific Ocean Problem.) There's no easy way to tell what any of these test cases represent in terms of real-world things, since they're just the result of the computer mindlessly generating combinations. Since there's no way to tell what they are, and there's too many to do them all, this method is essentially useless. Worse, it's very error prone, because it involves translating the requirements into some variant of Martian so that the computer can process them. Any errors in this translation are automatically propagated into unknown numbers of incorrect "test cases". Enough said.

 

End of Supplemental Material   

------------------------------------------------------------------------------------------------

Copyright © 1998 - 2005 by The Testing Center. All rights reserved. Images digitally watermarked.