Testing is a complex activity. Testing is often a distinct sub-project within the larger software development, maintenance, or integration project. Testing usually accounts for a substantial proportion of the overall project budget. Therefore, we must understand how we should manage the testing we do.
In this chapter, we cover essential topics for test management in six sections. The first relates to how to organize the testers and the testing. The second concerns the estimation, planning and strategizing of the test effort. The third addresses test progress monitoring, test reporting and test control. The fourth explains configuration management and its relationship to testing. The fifth covers the central topic of risk and how testing affects and is affected by product and project risks. The sixth and final section discusses the management of incidents, both product defects and other events that require further investigation.
5.1 TEST ORGANIZATION
- Recognize the importance of independent testing. (K1)
- List the benefits and drawbacks of independent testing within an organization. (K2)
- Recognize the different team members to be considered for the creation of a test team. (K1)
- Recall the tasks of typical test leaders and testers. (K1)
In this section, let’s talk about organizing a test effort within a project. We’ll look at the value of independent testing, and discuss the potential benefits and risks associated with independent testing. We will examine the various types of different test team members we might want on a test team. And we’ll familiarize ourselves with the typical tasks performed by test leaders and testers.
As we go through this section, keep your eyes open for the glossary terms tester, test leader and test manager.
5.1.1 Independent and integrated testing
In Chapter 1 we talked about independent testing from the perspective of individual tester psychology. In this chapter, we’ll look at the organizational and managerial implications of independence.
The approaches to organizing a test team vary, as do the places in the organization structure where the test team fits. Since testing is an assessment of quality, and since that assessment is not always positive, many organizations strive to create an organizational climate where testers can deliver an independent, objective assessment of quality.
When thinking about how independent the test team is, recognize that independence is not an either/or condition, but a continuum. At one end of the continuum lies the absence of independence, where the programmer performs testing within the programming team.
Moving toward independence, you find an integrated tester or group of testers working alongside the programmers, but still within and reporting to the development manager. You might find a team of testers who are independent and outside the development team but reporting to project management.
Near the other end of the continuum lies complete independence. You might see a separate test team reporting into the organization at a point equal to the development or project team. You might find specialists in the business domain (such as users of the system), specialists in technology (such as database experts), and specialists in testing (such as security testers, certification testers, or test automation experts) in a separate test team, as part of a larger independent test team, or as part of a contract, outsourced test team. Let’s examine the potential benefits and risks of independence, starting with the benefits.
An independent tester can often see more, other, and different defects than a tester working within a programming team – or a tester who is by profession a programmer. While business analysts, marketing staff, designers, and programmers bring their own assumptions to the specification and implementation of the item under test, an independent tester brings a different set of assumptions to testing and to reviews, which often helps expose hidden defects and problems related to the group’s way of thinking, as we discussed in Chapter 3. An independent tester brings a skeptical attitude of professional pessimism, a sense that, if there’s any doubt about the observed behavior, they should ask: Is this a defect?
At the team level, an independent test team reporting to a senior or executive manager may enjoy (once they earn it) more credibility in the organization than a test leader or tester who is part of the programming team. An independent tester who reports to senior management can report his results honestly and without concern for reprisals that might result from pointing out problems in coworkers’ or, worse yet, the manager’s work. An independent test team often has a separate budget, which helps ensure the proper level of money is spent on tester training, testing tools, test equipment, and so forth. In addition, in some organizations, testers in an independent test team may find it easier to have a career path that leads up into more senior roles in testing.
Independent test teams are not risk-free. It’s possible for the testers and the test team to become isolated. This can take the form of interpersonal isolation from the programmers, the designers, and the project team itself, or it can take the form of isolation from the broader view of quality and the business objectives (e.g., obsessive focus on defects, often accompanied by a refusal to accept business prioritization of defects). This leads to communication problems, feelings of alienation and antipathy, a lack of identification with and support for the project goals, spontaneous blame festivals and political backstabbing.
Even well-integrated test teams can suffer problems. Other project stakeholders might come to see the independent test team – rightly or wrongly – as a bottleneck and a source of delay. Some programmers abdicate their responsibility for quality, saying, “Well, we have this test team now, so why do I need to unit test my code?”
Due to a desire for the benefits of an independent test team, companies sometimes establish them, only to break them up again later. Why does that happen? A common cause is the failure of the test manager to effectively manage the risks of independence listed above. Some test teams succumb to the temptation to adopt a “no can do” attitude, coming up with reasons why the project should bend to their needs rather than each side being flexible so as to enable project success. Testers take to acting as enforcers of process or as auditors without a proper management mandate and support.
Resentments and pressures build, until at last the organization decides that the independent test team causes more problems than it solves. It’s especially important for testers and test managers to understand the mission they serve and the reasons why the organization wants an independent test team. Often, the entire test team must realize that, whether they are part of the project team or independent, they exist to provide a service to the project team.
There is no one right approach to organizing testing. For each project, you must consider whether to use an independent test team, based on the project, the application domain, and the levels of risk, among other factors. As the size, complexity, and criticality of the project increases, it is important to have independence in later levels of testing (like integration test, system test and acceptance test), though some testing is often best done by other people such as project managers, quality managers, developers, business and domain experts or infrastructure or IT operations experts.
