The following steps will help you develop a good survey – one that results in useful data and respects the time and effort of your respondents.
1. Clarify the purpose of the survey. What question(s) are you interested in using the survey to answer? Invest some time in describing what it is you want to know from the survey including the question of interest. Be as specific as possible.
2. Evaluate existing surveys. Determine if all or part of an existing survey will help you answer your question(s). For example, if you are interested in the experiences of new students, conduct a web search to determine what sorts of questions are being asked of new students at other universities. If you find a survey, request permission to borrow from it or use it.
3. Continue to clarify what you really want to know. To help clarify your thinking, conduct exploratory in depth interviews and/or casually observe the behavior/issue you want to learn about. As relevant consider reviewing the relevant literature or publications.
4. Identify the question format(s) appropriate for your survey. There are two general types - open ended and close ended.
a. Open-ended questions allow respondents to express ideas in their own words. Think question followed by an open text-box; no answer categories are provided.
Example: How might the tutoring program be improved?
Uses: Open-ended questions are often used in the question development phase because they can generate the input to create a range of responses for close-ended questions.
Analysis: It can take time to analzye the data because coding is needed to identify themes.
Considerations: Use judiciously. It is easier for respondents to answer close-ended questions quickly.
b. Close-ended questions provide a range of answers for respondents to choose.
Example: How might the tutoring program be improved: (a) more tutors, (b) more hours, (c) more subjects, (d) other.
Analysis: This process can also take time to code data, but typically the data will be easy to analyze and report.
Considerations: Close-ended questions are easy for respondents to answer quickly, however, close-ended questions lack the depth of understanding responses that open-ended questions can give.
Additional information about these two types of questions is available here. For further information about using surveys for assessment, click here.
5. Draft questions. Review the following resources to avoid errors or mistakes that can compromise your data collection.
- The 10 commandments for writing outstanding survey questions. (Qualtrics)
- Survey questions 101: Do you make any of these 7 question writing mistakes? (Qualtrics)
- Writing good survey questions tips and advice. (UCSD)
- Tips for writing questionnaire items. (University of Washington)
- Surveying Students at UC Merced. (UCM)
6. Seek feedback. Have a knowledgeable person provide feedback on the items and revise as needed.
7. Pre-test or pilot the survey. Invest some time to
- check for ambiguous terms and/or questions,
- estimate completion time, and
- consider survey appearance/presentation.
You'll be happy you did when the data are useful! Toward that end consider the following:
- Pilot tests and focus groups (Pew Research Center)
- Pretesting (Pew Research Center)
8. Estimate the time it will take your respondents to complete your survey. In general, the longer the survey the lower the response rate. Though long surveys can also be carefully constructed to achieve high response rates.
7. Format the survey to maximize data collection. When conducting web surveys, try to minimize scrolling by, as possible, making each survey page take up only a single screen of space. Respondent answers are recorded only when a submit button is pressed. If your pages are too long, and the respondent doesn't complete a page, answers for that page will not be captured.
8. Provide respondents with the information they need and you are obligated to provide. Ensure respondents have the information they need to successfully complete the survey. Toward that end, determine a means to:
- Provide clear instructions to the person distributing the survey (if not you) and to the respondents filling out the survey.
- Inform respondents
- how their information will be used,
- that responding to the survey is voluntary, and
- whom to contact with questions.
- Thank the respondents for their response.
- Inform respondents about the availability of the results, if applicable.
8. Revise and fine tune. Revise questions as needed. If new items are added, be sure to pre-test them.
