How Client/Server Is Different

How Testing Client/Server Applications Is Different

In this lesson you'll learn:

What new kinds of things can go wrong

A concrete example of a major problem area

A preview of how to expand our testing techniques

What's Different About Testing
Client/Server Applications?
Testing client/server applications requires some additional techniques to handle the new effects introduced by the client/server architecture.
Testing client/server systems is definitely different - but it's not "from another planet" type different.
We're still testing software, so the picture looks like this:

This class includes all the core testing techniques that you'll need to test any system, including systems that have a client/server design, plus the special techniques needed for client/server.
Even if you're an experienced tester we think you'll find some interesting and useful new ideas for testing all types of systems.
So what's different about client/server?
Testing client/server applications is more challenging than testing traditional systems because:
1. New kinds of things can go wrong.
(Example: Data and messages can get lost in the network)
2. It’s harder to set up, execute, and check the test cases.
(Example: Testing for proper responses to timeouts)
3. Regression testing is harder to automate.
(Example: It’s not easy to create an automated ‘server is busy’ condition)
4. Predicting performance and scalability become critical.
(Example: It seems to work fine with 10 users. But what about with 1,000 users? 10,000 users?)
Obviously some new techniques are needed to test client/server systems. Most of these apply to distributed systems of all kinds, not just the special case of a client/server architecture. So as you encounter various design permutations, “three-tiered” and so on, you’ll be able to put together an effective test plan for them.
The key to understanding how to test these systems is to understand exactly how and why each type of potential problem arises. With this insight, the testing solution will usually be obvious. So we’ll take a look at how things work, and from that we’ll develop the new testing techniques we need.

First, let’s define “client/server”.
We mean it very broadly: one program requests a service from another program. Often the service is ‘providing data’, but we won’t make any assumptions about what the service is.
Here’s an example of the new kinds of things that can go wrong:
Suppose Program A, (a client program, because it makes a request) asks Program B, a server program, to update some fields in a database.
Program B is on another computer.
Program A expects Program B to report that either:
(1) the operation was successfully completed,
or
(2) the operation was unsuccessful (for example, because the requested record was locked.)

However, time passes and A hears nothing.
What should A do?
Depending on the speed and reliability of the network connecting A and B, there comes a time when A must conclude that something has probably gone wrong. But what?
Some possibilities are:
1. The request got lost and it never reached B.
2. B received the request, but is too busy to respond yet.
3. B got the request, but crashed before it could begin processing it.
B may or may not have been able to store the request before it crashed.
If B did store the request, it might try to service it upon awakening.
4. B started to process the request, but crashed while processing it.
5. B finished processing the request (successfully or not), but crashed before it could report the result.
6. B reported the result, but the result got lost and was never received by A.

There are more possibilities, but you get the idea.
So what should A do?

The problem: A can’t tell the difference between any of the above cases (without taking some further action).
Dealing with this problem involves complex schemes such as the Two Phase Commit.
When we test the client program A, we need to see whether it’s robust enough to at least do something intelligent for each of the above scenarios. Otherwise we can’t say that A “works right”.
This example illustrates one new type of testing that we have to do for client/server systems – testing the client program for correct behavior in the face of uncertainty. To do that we're going to have to create or simulate various conditions, such as "no response from server".
Later, we’ll look at the other new type of testing, this time on the server side: performance testing and scalability issues.
For now let's look at some major reasons why client/server systems cause new effects:

(1) Most of these systems are event-driven.
Basically, this means:
"Nothing Happens Until Something Happens"
Most program action is triggered by an event, such as the user hitting a key, some I/O being completed, or a clock timer expiring.
The event is intercepted by an "event handler" or "interrupt handler" piece of code, which generates an internal message (or lots of messages) about what it detected.
This means that it's harder to set up test cases than it is, say, to define a test case for a traditional system that prints a check.
To set up a test case you need to create events, or to simulate them. That's not always easy, especially because you have to generate these events when the system is in the proper state - but there are ways to do it.
We have a whole lesson coming up on States and Events, and how to use them for testing.
(2) The systems never stop.
Many client server/systems are set up to never stop running unless something goes really wrong.
It's true for the servers, and in many cases, it's true for the client machines.
Traditional systems complete an action, such as printing a report, and then turn in for the night. When you restart the program it's a whole fresh new day for it.
In systems that don't stop (on purpose), things are different.
Errors accumulate.
As someone put it, "Sludge builds up on the walls of the operating system."
So things like memory leaks that are wrong, but that probably wouldn't affect most traditional systems, will eventually bring non-stop systems down if they aren't detected and corrected.
One good way to minimize these effects is to use something called SOW and HILT variables in testing, which we'll cover in detail in a few lessons.
(3) The system contains multiple computers and processors which can act (and fail) independently.
Worse, they communicate with each other over less than perfectly reliable communication lines.
These two factors are the root cause of the problem detailed above, where Program A makes a request of Program B, but gets no response.
We'll cover several techniques for testing the robustness of programs in the kind of circumstances that client/server systems can generate.
First, we want to look at several concepts that form the bedrock for all of testing.

Summary:

Client/server systems can create conditions that require new testing techniques.

Despite the added complexity, there are methods available to deal with most of the testing problems.

Client/server systems aren't that different from traditional systems. We're going to need to use basic testing techniques as well as more exotic ones to get the job done.

Questions about this lesson? Just . . . Ask the instructor

Looking Ahead

In the next lesson, we'll look at the big problem that underlies all of testing - including client/server testing - and move on to an array of solutions.

------------------------------------------------------------------------------------------------

Supplemental Material:

The Two Phase Commit
The Two Phase Commit is a procedure designed to prevent partial updates in a distributed system.
For example, suppose that a new employee is hired. This will require updates to databases in the employee records area, in payroll, and in the manpower/project area, among others.
What we don't want is for some of these updates to be done, but not all of them.
One way to have a partial update occur is for each of these databases to live on different machine. When the update request to "add an employee" is sent out, some of these machines may be down, or their databases may not be available.
In the old days, when everything was on one huge mainframe, there was elaborate system software that coordinated all of this. It knew the status of everything and it backed out transactions in an orderly manner when necessary.
With multiple machines, even if some are mainframes, there is no Big Guy who knows what's going on everywhere.
So the Two Phase Commit acts like this:
Phase 1: The requesting program asks all the programs who will be doing updates, "Is everybody ready?" If anyone says they're not ready, or fails to respond, the update is canceled, perhaps to be retried later.
Phase 2: If the programs all reply that they are ready, willing, and able, then update orders are issued to all of them.
They are expected to report whether they succeeded in completing their update.
If any program reports a failure, or does not reply, the update has not succeeded totally and must be rolled back.
"Roll back" requests are then issued to everyone who reported that they had completed the update, and the update process is either tried again immediately or scheduled to be retried at a later time.

While this Two Phase Commit process often works well, there are a number of possible problems.
(1) We don't know whether the non-reporting programs did the update or not.
(2) A request for a rollback may or may not be successful, and we may or may not be notified how it went.
So there's still a possibility that we've left things in an inconsistent and unknown state.
If you're doing lots of transactions, the number left in doubt may be considerable.
Because of these problems, some installations with complex configurations have abandoned the Two Phase Commit in favor of even more elaborate schemes "because it seems like someone always isn't ready, or didn't do it right, or we can't tell what happened."
As you can guess, the more complicated schemes also have problems, because they're also a victim of the basic uncertainty. When you have multiple, independent processors, subject to independent failure, communicating over less than perfectly reliable communication lines, you can't know everything for sure.
So we have to test that we do something reasonable anyway.

Memory Leaks
A memory leak is a bug in an application's code. It slowly decreases the amount of memory available.
The first time it runs (or maybe the hundredth time it runs), the application appears to work fine. But eventually the application will crash or freeze. And depending on what operating system you're running ("separate memory spaces" or not), it can cause the whole system to crash or freeze.
Usually a memory leak is caused by an application that requests, say, 5000 bytes of memory to hold something, but doesn't give it back (or all of it back) when it's finished with it.
There are two obvious causes of memory leaks:
(1) The program (read "programmer") is just plain sloppy and forgets to return the memory that was taken.
(2) The program terminates abnormally before it gets to return the memory that it used. In good operating systems "garbage collection" fixes this, but in bad ones it doesn't.
There are also more subtle mechanisms. For example, a program can prematurely release a block of memory that contains the pointers to the other memory blocks that it has requested. When this happens, those blocks can never be returned by the system because it forgot it had them.
There are lots of software tools that detect memory leaks, usually by acting as a Big Brother overseeing memory requests and releases.
Memory leaks are rarely a problem in "do it and stop" systems, but in non-stop systems such as server code, it's a good idea to use one of the tools described at the Web sites referenced on the Resources page.
Use the Navigate icon at the top and bottom of each lesson page to get to the Resources page quickly.

End of Supplemental Material

------------------------------------------------------------------------------------------------

	In this lesson you'll learn:
				What new kinds of things can go wrong
				A concrete example of a major problem area
				A preview of how to expand our testing techniques

	Summary:
				Client/server systems can create conditions that require new testing techniques.

				Despite the added complexity, there are methods available to deal with most of the testing problems.

				Client/server systems aren't that different from traditional systems. We're going to need to use basic testing techniques as well as more exotic ones to get the job done.