10.45.1 Purpose
A survey or questionnaire is used to elicit business analysis information – including information about customers, products, work practices, and attitudes – from a group of people in a structured way and in a relatively short period of time.
10.45.2 Description
A survey or questionnaire presents a set of questions to stakeholders and subject matter experts (SMEs), whose responses are then collected and analyzed in order to formulate knowledge about the subject matter of interest. The questions can be submitted in written form or can be administered in person, over the telephone, or using technology that can record responses.
There are two types of questions used in a survey or questionnaire:
- Close-ended: the respondent is asked to select from a list of predefined responses, such as a Yes/No response, a multiple-choice selection, a rank/ order decision, or a statement requiring a level of agreement. This is useful when the anticipated range of user responses is fairly well defined and understood. The responses to close-ended questions are easier to analyze than those gained from open-ended questions because they can be tied to numerical coefficients.
- Open-ended: the respondent is asked to answer questions in a free form without having to select an answer from a list of predefined responses. Open-ended questions are useful when the issues are known and the range of user responses is not. Open-ended questions may result in more detail and a wider range of responses than closed-ended questions. The responses to open-ended questions are more difficult and time-consuming to categorize, quantify, and summarize as they are unstructured and often include subjective language with incomplete or superfluous content.
Questions should be asked in a way that does not influence the response data. They should be expressed in neutral language and should not be structured or sequenced to condition the respondent to provide perceived desirable answers.
10.45.3 Elements
.1 Prepare
An effective survey or questionnaire requires detailed planning in order to ensure that the needed information is obtained in an efficient manner.
When preparing for a survey or questionnaire, business analysts do the following:
- Define the objective: a clear and specific objective establishes a defined purpose of the survey or questionnaire. Questions are formulated with the intent of meeting the objective.
- Define the target survey group: identifying the group to be surveyed in terms of population size and any perceived variations (for example, culture, language, or location) helps identify factors that can impact survey design.
- Choose the appropriate survey or questionnaire type: the objective of the survey or questionnaire determines the appropriate combination of close-ended questions and open-ended questions to elicit the information required.
- Select the sample group: consider both the survey or questionnaire type and the number of people in the identified user group in order to determine if it is necessary and feasible to survey the entire group. It may be important to survey all members – even of a large group – if their demographics indicate a wide variance due to geographic distribution, regulatory differences, or lack of standardization in job function or business process. If the population is large and the survey type is open-ended, it may be necessary to identify a subset of users to engage in the questionnaire process. Using a statistical sampling method will help ensure that the sample selected is representative of the population so that the survey results can be reliably generalized.
- Select the distribution and collection methods: determine the appropriate communication mode for each sample group.
- Set the target level and timeline for response: determine what response rate is acceptable and when it should be closed or considered complete. If the actual response rate is lower than the acceptable threshold, the use of the survey results may be limited.
- Determine if the survey or questionnaire should be supported with individual interviews: as a survey or questionnaire does not provide the depth of data that can be obtained from individual interviews, consider either pre- or post-survey or questionnaire interviews.
- Write the survey questions: ensure that all the questions support the stated objectives.
- Test the survey or questionnaire: a usability test on the survey identifies errors and opportunities for improvement.
.2 Distribute the Survey or Questionnaire
When distributing the survey or questionnaire it is important to communicate the survey’s objectives, how its results will be used, as well as any arrangements for confidentiality or anonymity that have been made.
When deciding on a method of distribution (for example, in-person, e-mail, or survey tool), business analysts consider:
- the urgency of obtaining the results,
- the level of security required, and
- the geographic distribution of the respondents.
.3 Document the Results
When documenting the results of the survey or questionnaire, business analysts:
- collate the responses,
- summarize the results,
- evaluate the details and identify any emerging themes,
- formulate categories for encoding the data, and
- break down the data into measurable increments.
10.45.4 Usage Considerations
.1 Strengths
- Quick and relatively inexpensive to administer.
- Easier to collect information from a larger audience than other techniques such as interviews.
- Does not typically require significant time from the respondents.
- Effective and efficient when stakeholders are geographically dispersed.
- When using closed-ended questions, surveys can be effective for obtaining quantitative data for use in statistical analysis.
- When using open-ended questions, survey results may yield insights and opinions not easily obtained through other elicitation techniques.
.2 Limitations
- To achieve unbiased results, specialized skills in statistical sampling methods are needed when surveying a subset of potential respondents.
- The response rates may be too low for statistical significance.
- Use of open-ended questions requires more analysis.
- Ambiguous questions may be left unanswered or answered incorrectly.
- May require follow-up questions or more survey iterations depending on the answers provided.
