- Recall reasons that both specification-based (black-box) and structure-based (white-box) approaches to test case design are useful and list the common techniques for each. (K1)
- Explain the characteristics and differences between specification-based testing, structure-based testing and experience-based testing. (K2)
In this section we will look at the different types of test design technique, how they are used and how they differ. The three types or categories are distinguished by their primary source: a specification, the structure of the system or component, or a person’s experience. All categories are useful and the three are complementary.
In this section, look for the definitions of the glossary terms: black-box test design techniques, experience-based test design techniques, specification-based test design techniques, structure-based test design techniques and white-box test design techniques.
4.2.1 Introduction
There are many different types of software testing technique, each with its own strengths and weaknesses. Each individual technique is good at finding particular types of defects and relatively poor at finding other types. For example, a technique that explores the upper and lower limits of a single input range is more likely to find boundary value defects than defects associated with combinations of inputs. Similarly, testing performed at different stages in the software development life cycle will find different types of defects; component testing is more likely to find coding logic defects than system design defects.
Each testing technique falls into one of a number of different categories. Broadly speaking there are two main categories, static and dynamic. Static techniques were discussed in Chapter 3. Dynamic techniques are subdivided into three more categories: specification-based (black-box, also known as behavioral techniques), structure-based (white-box or structural techniques) and experience-based. Specification-based techniques include both functional and nonfunctional techniques (i.e., quality characteristics). The techniques covered in the syllabus are summarized in Figure 4.1.
4.2.2 Static testing techniques
As we saw in Chapter 3, static testing techniques do not execute the code being examined and are generally used before any tests are executed on the software.
They could be called non-execution techniques. Most static testing techniques can be used to ‘test’ any form of document including source code, design documents and models, functional specifications and requirement specifications.
However, “static analysis” is a tool-supported type of static testing that concentrates on testing formal languages and so is most often used to statically test source code
4.2.3 Specification-based (black-box) testing techniques
The first of the dynamic testing techniques we will look at are the specification-based testing techniques. These are also known as “black-box” or input/output driven testing techniques because they view the software as a black-box with inputs and outputs, but they have no knowledge of how the system or component is structured inside the box. In essence, the tester is concentrating on what the software does, not how it does it.
Notice that the definition mentions both functional and non-functional testing. Functional testing is concerned with what the system does, its features or functions. Non-functional testing is concerned with examining how well the system does something, rather than what it does. Non-functional aspects (also known as quality characteristics or quality attributes) include performance, usability, portability, maintainability, etc. Techniques to test these non-functional aspects are less procedural and less formalized than those of other categories as the actual tests are more dependent on the type of system, what it does and the resources available for the tests.
Non-functional testing is part of the Syllabus and is also covered in Chapter 2. There are techniques for deriving non-functional tests [Gilb, 1988], [Testing Standards], but they are not covered at the Foundation level.
Categorizing tests into black and white-box is mentioned in a number of testing books, including [Beizer, 1990] and [Copeland, 2003].
4.2.4 Structure-based (white-box) testing techniques
Structure-based testing techniques (which are also dynamic rather than static) use the internal structure of the software to derive test cases. They are commonly called “white-box” or “glass-box” techniques (implying you can see into the system) since they require knowledge of how the software is implemented, that is, how it works. For example, a structural technique may be concerned with exercising loops in the software. Different test cases may be derived to exercise the loop once, twice, and many times. This may be done regardless of the functionality of the software.
4.2.5 Experience-based testing techniques
In experience-based techniques, people’s knowledge, skills and background are a prime contributor to the test conditions and test cases. The experience of both technical and businesspeople is important, as they bring different perspectives to the test analysis and design process. Due to previous experience with similar systems, they may have insights into what could go wrong, which is very useful for testing.
4.2.6 Where to apply the different categories of techniques
Specification-based techniques are appropriate at all levels of testing (component testing through to acceptance testing) where a specification exists. When performing system or acceptance testing, the requirements specification or functional specification may form the basis of the tests. When performing component or integration testing, a design document or low-level specification forms the basis of the tests.
Structure-based techniques can also be used at all levels of testing.
Developers use structure-based techniques in component testing and component integration testing, especially where there is good tool support for code coverage. Structure-based techniques are also used in system and acceptance testing, but the structures are different. For example, the coverage of menu options or major business transactions could be the structural element in system or acceptance testing.
Experience-based techniques are used to complement specification-based and structure-based techniques, and are also used when there is no specification, or if the specification is inadequate or out of date. This may be the only type of technique used for low-risk systems, but this approach may be particularly useful under extreme time pressure – in fact this is one of the factors leading to exploratory testing.
