CHAPTER 1 – Fundamentals of testing – Part 1/4

In this chapter, we will introduce you to the fundamentals of testing: why testing is needed; its limitations, objectives and purpose; the principles behind testing; the process that testers follow; and some of the psychological factors that testers must consider in their work. By reading this chapter you’ll gain an understanding of the fundamentals of testing and be able to describe those fundamentals.

1.1 WHY IS TESTING NECESSARY?

1. Describe, with examples, the way in which a defect in software can cause harm to a person, to the environment or to a company. (K2)
2. Distinguish between the root cause of a defect and its effects. (K2)
3. Give reasons why testing is necessary by giving examples. (K2)
4. Describe why testing is part of quality assurance and give examples of how testing contributes to higher quality. (K2)
5. Recall the terms “mistake”, “defect”, “fault”, “failure” and the corresponding terms “error” and “bug”. (K1)
6. Explain the fundamental principles in testing. (K2)

1.1.1 Introduction

In this section, we’re going to kick off the book with a discussion on why testing matters. We’ll describe and illustrate how software defects or bugs can cause problems for people, the environment or a company. We’ll draw important distinctions between defects, their root causes and their effects. We’ll explain why testing is necessary to find these defects, how testing promotes quality, and how testing fits into quality assurance. In this section, we will also introduce some fundamental principles of testing.

As we go through this section, watch for the Syllabus terms bug, defect, error, failure, fault, mistake, quality, risk, software, testing and exhaustive testing. You’ll find these terms defined in the glossary.

You may be asking “what is testing?” and we’ll look more closely at the definition of testing in Section 1.2. For the moment, let’s adopt a simple everyday life usage: “when we are testing something we are checking whether it is OK”.

We’ll need to refine that when we define software testing later on. Let’s start by considering why testing is needed. Testing is necessary because we all make mistakes. Some of those mistakes are unimportant, but some of them are expensive or dangerous. We need to check everything and anything we produce because
things can always go wrong – humans make mistakes all the time – it is what we do best!

Because we should assume our work contains mistakes, we all need to check our own work. However, some mistakes come from bad assumptions and blind spots, so we might make the same mistakes when we check our own work as we made when we did it. So we may not notice the flaws in what we have done.

Ideally, we should get someone else to check our work – another person is more likely to spot the flaws. In this book, we’ll explore the implications of these two simple paragraphs again and again. Does it matter if there are mistakes in what we do? Does it matter if we don’t find some of those flaws? We know that in ordinary life, some of our mistakes do not matter, and some are very important. It is the same with software systems. We need to know whether a particular error is likely to cause problems. To help us think about this, we need to consider the context within which we use different software systems.

1.1.2 Software systems context

Testing Principle – Testing is context dependent

Testing is done differently in different contexts. For example, safety-critical software is tested differently from an e-commerce site

These days, almost everyone is aware of software systems. We encounter them in our homes, at work, while shopping, and because of mass-communication systems. More and more, they are part of our lives. We use software in day-today business applications such as banking and in consumer products such as cars and washing machines. However, most people have had an experience with software that did not work as expected: an error on a bill, a delay when waiting for a credit card to process and a website that did not load correctly are common examples of problems that may happen because of software problems.

Not all software systems carry the same level of risk and not all problems have the same impact when they occur. A risk is something that has not happened yet and it may never happen; it is a potential problem. We are concerned about these potential problems because, if one of them did happen, we’d feel a negative impact. When we discuss risks, we need to consider how likely it is that the problem would occur and the impact if it happens. For example, whenever we cross the road, there is some risk that we’ll be injured by a car. The likelihood depends on factors such as how much traffic is on the road, whether there is a safe crossing place, how well we can see, and how fast we can cross. The impact depends on how fast the car is going, whether we are wearing protective gear, our age and our health. The risk for a particular person can be worked out and therefore the best road-crossing strategy.

Some of the problems we encounter when using software are quite trivial, but others can be costly and damaging – with loss of money, time or business reputation – and even may result in injury or death. For example, suppose a user interface has typographical defects. Does this matter? It may be trivial, but it could have a significant effect, depending on the website and the defect:

• If my personal family-tree website has my maternal grandmother’s maiden name spelt wrong, my mother gets annoyed and I have to put up with some family teasing, but I can fix it easily and only the family see it (probably).
• If the company website has some spelling mistakes in the text, potential customers may be put off the company as it looks unprofessional.
• If a software program miscalculates pesticide application quantities, the effect could be very significant: suppose a decimal point is wrongly placed so that the application rate is 10 times too large. The farmer or gardener uses more pesticide than needed, which raises his costs, has environmental impacts on wildlife and water supplies and has health and safety impact for the farmer, gardener, family and workforce, livestock and pets. There may also be consequent loss of trust in and business for the company and possible legal costs and fines for causing the environmental and health problems.