THE TESTING OF COMPUTER SOFTWARE
by
A.I. Llewelyn and R.F. Wickens
Contents
1. Introduction
2. The aims and problems of acceptance testing software
3. The requirements of a software testing procedure
4. Present software testing procedures
5. Possible test strategies
6. A proposed test procedure
7. Application of the proposed procedure
8. Criteria of acceptance
9. Effort and cost estimates for the approval scheme
10. Consequences of approval
11. Conclusions and discussion
1. Introduction
Computer software has changed rapidly from being merely an adjunct to the hardware to becoming an extremely important part of an installation and one that profoundly affects the overall performance. It is now widely accepted that computer hardware should be given some form of acceptance test by the customer and such tests have formed part of the United Kingdom government’s computer contracts for about eight years. However, the weak area of most installations is now that of software.
Users complain that the software they received was badly designed and produced, was not delivered on time and when finally delivered, contained serious errors and restrictions and was also inefficient. Unfortunately, these criticisms are often only too true although, happily, it is unusual for all the criticisms to apply simultaneously. The consequences of inadequate software can be disastrous, both in financial terms as a result of delays and wasted effort in getting a system operational and also in terms of the morale of an installation’s staff.
It is my opinion that software should now be subjected to a testing discipline just like any other product. The user should be provided with a specification of the software he is to receive and be able to have confidence that the software he gets will perform in accordance with this specification. By specification is meant a document that describes the product’s operation and performance in some detail. Certainly, this specification should be more detailed than the typical user manuals provided to-day. To obtain the user’s confidence, the software will have to be rigorously tested for conformity with the specification. This testing has two aspects: that carried out by the producer of the software to satisfy himself that it is fit for release to his customers and that carried out on behalf of the customer in order to give him sufficient confidence in the software to justify his paying for it. This paper is primarily concerned with the second aspect of testing. However, both aspects are very closely related and, of course, if the in house testing of the supplier is adequately carried out over a long enough period of time to gain customer confidence, the second aspect becomes largely unnecessary. It is certainly true that what might be described as the acceptance testing of software must not become a substitute for proper in house quality assurance.