Work tends to proceed a lot better if everyone is speaking the same language. The British Computer Society has come up with a standard (BCS SIGIST Standard Glossary of Testing (BS 7925-1)) which is used in its ISEB examinations.
| Defect | The departure of a quality characteristic from its specified value that results in a product or service not satisfying its normal usage requirements. |
| Error | An error is a human action that produces an incorrect result. |
| Fault | A fault is a manifestation of an error in software (also known as a defect or bug). |
| Failure | A fault, if encountered, may cause a failure, which is a deviation of the software from its expected delivery or service |
| Quality | The totality of the characteristics of an entity that bear on its ability to satisfy stated or implied needs. Testing determines the quality of the system. |
| Quality Assurance | All those planned actions used to fulfil the requirements for quality. |
| Reliability | Reliability is the probability that software will not cause the failure of a system for a specified time under specified conditions. |
| Testing | Testing is the process of exercising software to verify that it satisfies specified requirements, and to detect errors. |
If a system is released without testing, there is a real risk that it will have bugs in it. This could cause loss of customers. For instance, people may click away from a buggy web site and never come back. A faulty system could also cause catastrophic failure (e.g. a Venus probe crashed because testing had not revealed that one part of the system was using metric units, and another part imperial units). Faults can also cause loss of valuable data. In other words, failure to detect bugs can cost a lot of money. Faults in some systems, for instance avionics or military, can also cost lives. Testing can also discover systems which do not meet the original requirements, do not meet legal requirements, or are difficult or impossible to use.
Errors occur because we are not perfect and, even if we were, we are working under constraints such as delivery deadlines, so we are likely to make mistakes or omissions. Other causes of errors include poor training and communication, poor definition of requirements or changes to requirements. A major source of errors is assumption.
A single error can cost nothing or a lot. Errors can occur at any stage of the cycle, including during specification. Testing is carried out throughout the lifecycle, including after implementation.
Exhaustive testing would in most cases take an enormous amount of resource and is therefore usually impractical.The amount of testing performed depends on the risks involved. Risk must be used as the basis for allocating the test time that is available and for selecting what to test and where to place emphasis. It is difficult to determine how much testing is enough.
Testing identifies faults, whose removal increases the software quality by increasing the software's potential reliability. Testing is the measurement of software quality. We measure how closely we have achieved quality by testing the relevant factors such as correctness, reliability, usability, maintainability, reusability, testability, etc.
Other factos that may determine the testing performed may be contractual requirements, or legal requirements, normally defined in industry-specific standards, or based on agreed best practice (or more realistically non-negligent practice).
The necessity for testing has been highlighted in major exercises such as year 2000 and the introduction of the euro, in safety-critical systems such as avionics, and in highly-visible, fast-moving applications such as e-commerce.
This page is very heavily based on the ISEB Foundation Certificate Syllabus.
This is unavoidable, as to pass the exam, you have to use their terminology.
Back to top Next Home Serious Stuff
© 2002. Copyright Sue Nethercott.