What Is Regression Testing?

	In this lesson you'll learn:
				What regression testing is all about
				What regression testing can do for you
				The two different kinds of changes to a system

		Regression Testing The idea behind regression testing is simple. We know that changes to an existing system often cause other things to break. To detect when this happens, we keep a set of tests (the Regression Test Set, or the RTS for short) containing tests of important things that used to work okay. We run the RTS after each set of changes to make sure that these things still work. That's it. The problem is to do regression testing correctly, so that it solves more problems than it creates.

It turns out that if you do regression testing almost right, but not quite right, it doesn't work. But it does require a lot of effort in this losing cause that could better be spent elsewhere in testing. (Where there's always far too much to do and never enough time to do it.) So we've got to do it completely right. It reminds me of the old Woody Allen piece where a friend has taken a speed reading course and proudly announces that he's read War and Peace in five minutes. Woody: "Five minutes?? How was the comprehension?" Friend: "Good, it was good. The book was . . . something about Russia." And that's the problem with regression testing. It's not "something about Russia", and you have to do it a certain way to make it work. There are a lot of similar things that sound almost like the right way, but they're not - and they're going to cause problems. This course will show you the right way to do it. Benefits of Regression Testing There are lots of benefits to doing regression testing. (1) It increases our chances of detecting bugs caused by changes to an application - either enhancements or bug fixes. Note that we don't guarantee that there are no side effects. We'll talk later about what you need to guarantee that you've detected any side effects. (2) It can also detect undesirable side effects caused by changing the operating environment. For example, hardware changes, or upgrades to system software such as the operating system or the database management system. (3) The Regression Test Set is also useful for a new way of doing integration testing. This new method is much faster and less confusing than the old way of doing integration testing - but you need a Regression Test Set to do it. We'll show you how. Why It's Called Regression Testing No one knows. You'll hear all kinds of interesting folklore about the term being borrowed from statistics, or the system somehow "regressing" from somewhere to somewhere else, but none of that makes much sense - and it adds to the confusion. We'd all be better off it they'd called it something descriptive like "Side Effects of Changing Things" testing, which even has a pronounceable acronym, SECT. But it's too late for that now. We're stuck with it . Great concept, weird name. The Big Questions After A Change To The System: 1. Did the change work? 2. Did it break anything else? 3. Is the RTS still okay? The first question is almost always easy. We simply test whatever the change is supposed to do, such as "don't write that type of policy in Oregon or Massachusetts anymore" or "all new engineering notes will be displayed in red". The answer to the second question, "Did It Break Anything Else?" is :"Probably". The mysteries are (a) what's broken, and (b) how would we know that? Answer: run the Regression Test Set (RTS). The third question, "Is the RTS still okay?", is often overlooked, but it's one of the things that can get you in trouble. The RTS is about our system, and if the system changes (which is the name of the game here), then if the RTS doesn't also change to reflect the changes to the system, it will eventually drift out of step with the current system and will eventually become useless- or worse, misleading. It's exactly like documentation that way. So we need to keep it up to date, and we'll see ways to do that. In a perfect world, development produces three things at a minimum:

The code you always get. (What condition it's in is another story!) The documentation, sometimes. The Regression Test Set, almost never. If I were going to be maintaining a system and I had to pick just one of either the documentation or the Regression Test Set, I'd take the Regression Test Set. With a good Regression Test Set, you can make sure that the side effects of changes aren't happening, and that's one of your highest priorities in maintenance. So one thing you'll want to do is to build an awareness among developers of the high value of saving tests used when the system was being constructed for use in a Regression Test Set. But of course it's not all roses. We also have to cover: What Can Go Wrong In the next lesson, we'll be going over this in detail. Briefly, regression testing can become too time consuming, it can be done incorrectly so that the side effects are not detected, or it can become a nightmare to organize and manage. But, as we'll see, none of this needs to happen.

	Summary:
				Regression testing means rerunning tests of things that used to work to make sure that a change didn't break something else.
				The set of tests used is called the Regression Test Set, or RTS for short.
				It's enormously helpful when you change an application, change the environment, and during integration of pieces.
				Regression testing is a simple concept, but it needs to be done just right to work in the real world.

Check your understanding with this quick exercise.

Questions about this lesson? Just . . . Ask the instructor

In the next lesson, we'll look at the six biggest problems to avoid during regression testing.

Copyright © 1998 - 2005 by The Testing Center. All rights reserved. Images digitally watermarked.

-----------------------------------------------------------------------------------------------

Supplemental Material:

Preload:        

(No supplemental material for this lesson) 

End of Supplemental Material   

-----------------------------------------------------------------------------------------------

Copyright © 1998 - 2005 by The Testing Center. All rights reserved. Images digitally watermarked.