 |
|
 |
 |
 |
 |
 |
|
|
|
|
| |
Testing Your Program
Introduction
Black-Box Testing
Boundary Values
Absurdity Testing
Extension Work
Software testing is the process of running a software system to determine
whether it matches its specification and executes properly within its intended environment. In other words, does it work properly? Does it meet the original specification?
So, lets suppose you have spent a lot of time and effort designing and implementing some smart new piece of software for your client. Would you feel confident in handing it over as a finished product the minute you have finished coding? Or would you instinctively run it under various conditions to check that it worked without error. I suspect the latter.
For your own peace of mind you will want to be confident your software is robust and works as expected. Your client will also want to feel confident the software meets the original specification and is error free since it is costing money to buy the software and may very well cost the client's company even more if it did not work as intended.
So, software testing has two main objectives:
- To uncover errors in the software product, so they can be fixed before distribution
- To give confidence that the product is working well.
Of course the irony is that, although a product may be thoroughly tested, there is still no guarantee the software is one hundred percent perfect. There may still be undiscovered bugs or conditions where errors may arise because that condition was just not tested for; perhaps the testing was not very good or the error condition so extremely obscure and unlikely to occur it was just not imagined. In effect, testing just improves confidence in the produce.
Now, although you may think that testing your software during the development process, while you are coding, is good enough, it isn't. Testing has to be carried out systematically, it has to be thorough and cover a range of conditions, and it has to be documented using test plans.
The question is, what do you test? Here are a few ideas:-
User input validation testing
Suppose a program requires that a user enters a name into a text box, then you may need to check that the user has done so. Or a command-line program requires the user to type in a username and password, then these inputs will need to be checked for validity. Or perhaps a user has to enter bank account details or an age, in which case you need to check for a valid number within the required range.
User interface testing
Your program may have a graphical user interface (GUI) or may run on the command-line or be menu-driven. Today, the most prominent interface is the is the GUI but older
interfaces like the command line and
menu-driven interface are still in use.
Possible input methods to consider are
mouse clicks, keyboard events, and input
from other devices. These all need to be tested to be sure they work as expected.
Other
You may need to test the software in different environments. Perhaps your software needs to run on different platform, in which case it needs to be tested under each different one.
Does your program interface with anything else? For example, perhaps it need to read and write to and from files. What happens if these files go missing or become corrupt. Perhaps the program interfaces with a database. Is the database connection fast or slow; what happens if a user tries to read from a database while another user is updating it?
Does you program rely on network connectivity. Can it handle it if the network connection fails. Can it recover gracefully if is in the process of reading or writing data and the connection is suddenly lost. etc.
Imagine you have been given a program executable to test. You can run the program and you can test it's user interface, give it inputs and see what the outputs are but you cannot see inside the program to see how it works.
Since the program code is unavailable, how the program carries out each action is a mystery to you. Like a black box, you cannot see inside it and so the program logic is unknowable. But hey! As long as the program works properly is doesn't matter if you can see inside it, wouldn't you agree.
Given the situation where you do not have access to the program code and you have to test the program , the procedure is to give it inputs, record the outputs, then compare expected outputs with the actual outputs. This is called black-box testing.
Black-box tests are tests carried out on a software component where the internal logic/code is unknown
Example: A program requires the user to input an age and we have to produce a test plan that covers this input. What input is valid for the user to type in and what input is invalid? In other words...
What do we test for?
Must the input be a number? Must it be a whole number? Must it be positive. Are there a limit on the maximum number of characters allowed. What happens if the user types in letters, etc. In many cases, the program specification is quite clear on the inputs it is expecting. Suppose the program requires the age to be a positive whole number, with a maximum of 4 characters entered. With this information we can prepare a test plan as follows...
Valid Data
Case |
Test Data |
Expected Result |
Actual Result |
Notes |
1 |
0 |
accepted (positive whole number at the boundary) |
|
|
2 |
9999 |
accepted (positive whole number at the boundary) |
|
|
3 |
88 |
accepted (positive whole number within range) |
|
|
Invalid Data
Case |
Test Data |
Expected Result |
Actual Result |
Notes |
4 |
-1 |
rejected (negative number, outside the range) |
|
|
5 |
10.5 |
rejected (decimal number) |
|
|
6 |
10000 |
rejected (> than 4 characters) |
|
|
7 |
nothing entered |
rejected (must enter something) |
|
|
8 |
abc#?> |
rejected (not numeric) |
|
|
Having produced a test plan, the next step is to carry out the testing on the program and fill in the Actual Result column. In the Notes column I might comment on the outcome, such as whether the code needs fixing because an actual result did not match an expected result.
A common question asked is - what data do I use in the tests?
Looking at the age input example above, you might ask, how did I decide on those values to test. Why did I choose 0 and 9999 to test in the first valid data table? I chose these because they are boundary values. The program specification specifies that the maximum number of characters entered is 4. This means the largest number that can be entered is 9999. Conversely, the smallest number that can be entered and still be a positive whole number is 0. So I chose those values to test, since they are at the boundaries between valid and invalid data.
In the second invalid data table I chose to test the program using the value 10000 because it is larger than the boundary value of 9999. The same is true of the value -1. It is smaller than the boundary value of 0.
Testing the minimum, maximum, and zero values is common practice. Any values between boundaries are treated as the same number. In other words, whether I tested using the value of 10000 or 11000 should be irrelevant in that if the value of 10000 is rejected then it is highly likely the value of 11000 also will be, since both are outside the boundary value of 9999.
The possible input combinations that thousands of users can make across a given software interface are simply too numerous for testers to apply them all. Testers must make tough decisions about which inputs to test. Choosing test data by partitioning numbers into sets where some are within valid boundaries and other sets outside makes testing more systematic and thorough.
A program that accepts user input must consider all the ways a user could enter invalid input by mistake. Moreover, a good program must not only deal with simple errors, it must be able to deal with conditions that could only be caused by a user who's escaped from an insane asylum.
User is prompted: Please enter your annual gross income: £____________
User enters: 100000000000000000000000000000000000000000000000000000000000000000000000
If the program was for Bill Gates or the Sultan of Brunei then this input would be acceptable. But you must accept that some user's cannot resist the temptation to play with a program, to input absurd values, to impress friends or out of sheer boredom. So your program must expect and be prepared for absurdities.
A good way of doing carrying out testing, taking into account absurdities is is to take any system limitation and test it to an extreme. This differs from error testing since error testing simply tests expected error conditions, such as entering alphabetic characters where numeric input is expected. It also differs from boundary testing insofar as boundary testing generally consists of inputs at or closely exceeding tolerance.
The cat test - recipe
Take a cat and...
- sit it on a keyboard,
- encourage it to pad the keyboard,
- let it take a stroll across the keyboard,
- encourage it to play with a dangling mouse
See how your program stands up to that absurdity test. Can it cope gracefully, outputting messages such as "$%^%&^*& is not valid input" .
If your program can withstand the formal testing using your own test plans (and of course the cat test) then you can feel a certain confidence that your software works as intended.
Carry out the following exercise:-
- Start step12 of the practice assignment
That's it!
|
|
| |
|
|
