What is Survey Research?

The Director needs information about a new program that she’s considering implementing.  She asks you to survey the right population, analyze the data, and write a report before next month’s meeting with the President. Where do you start? Do data already exist? If not, how do you write a survey?

Survey research uses surveys to collect data. Those data are reported and used by decision makers to improve the student experience.  As its name implies, survey research uses a survey as a data collection tool.  All surveys start with one common mission: collect data to help answer an important question that available data can’t address.   For example, an analyst in enrollment management might survey non-returning students to determine why they didn’t re-enroll; the director of academic advising might survey first-year students to understand the impact of advising on student success; the director of residence life might survey sophomores for feedback on ways to increase satisfaction; senior leaders might survey to gather information to support or understand policy decisions; and academic units might survey to understand teaching effectiveness or as evidence for accreditation.  

When to survey?

When should you conduct a survey?  Before conducting a survey, answer these questions:

• Can you use pre-existing reports or already-collected data to address the issue? 
• Can you collect data using other methods besides a survey (e.g., focus group, data/information in the institution’s student information system)?
• Is action unlikely (e.g., researching the need for more campus parking but there are no plans to build new lots)?

If you answered “yes” to any of these questions, then reconsider conducting a survey.  But, if you can’t address the issue using existing data, reports, or other data collection methods and you’re confident that action will be taken based on the survey results, then survey research is appropriate. 

In other words, survey research is the last resort in data collection; not the first because it utilizes institutional resources (i.e., time, effort, energy) and can contribute to survey fatigue of participants.  

Things to think about before surveying

Who is the audience for this work? We conduct survey research to answer important questions.  There must be an impetus that starts this work.  Who asked for this research to be conducted?  Besides that person(s), who else could use this information to improve the student experience? Once you have identified your audience, keep them in your mind; ultimately our work must answer their questions.

Is there a published/vendor instrument I could use?  Sometimes there may be a published or vendor instrument you could use for a fee.  Survey authoring isn’t easy and can take many days to draft; it might be cost effective to use a vendor survey.  Before drafting a survey, check out some vendors first.

Do you need IRB approval? Some institutions may consider all surveys subject to the state and federal laws for Human Research Protection Program (HRPP) and may require them to be reviewed by an Institutional Review Board (IRB) while other institutions may regard some surveys as not meeting the standards for IRB review.  Check with your institution’s IRB to understand their guidelines for survey research.

What online survey software will you use? Commercial software that makes it possible to design your own online survey, collect data and produce results is ubiquitous (e.g., SurveyMonkey, Zoomerang, eSurveysPro, Qualtrics).  Before exploring these products, ask your institutional research/effectiveness office if your institution has a contract established with a vendor that allows researchers to use their software as part of the existing institution package. 

How will you ensure motivated responses? The data you collect are only as good as the honest responses your respondents provide. Inspiring motivated responses is a challenge for every researcher. Some simple things to consider:

• Meaningful topic: If the survey is meaningful to the respondent or they understand why it’s important to the program, they are more likely to respond and answer honestly;
• Concise and focused survey: Keep the survey brief and focused on the topic.  Surveys that are very long with disjointed topics will confuse and irritate the respondent and they’re more likely to not respond.
• Acknowledge sensitive questions:  Sensitive questions are those that, if the information were made public, would could embarrass the respondent (e.g., sexual orientation, religious/political affiliation, alcohol/drug use).  But, acknowledging the sensitive question and reaffirming the student’s confidentiality will yield higher response rates and truthful responses.

How will your survey research project impact other units?  None of us work in a bubble but sometimes it might feel like that.  Our work will always directly, or indirectly, impact other units and being aware of that impact will be important to your survey research work.  Offices like institutional research (IR), institutional effectiveness (IE), and/or institutional assessment often conduct surveys.  You’ll need to communicate and coordinate with these offices to ensure that your survey isn’t launched at the same time as theirs.

Are there institutional data that can be merged with the survey data?  Ideally your survey won’t ask questions in which those data are already known.  Consider working with your IR/IE office or the registrar to identify data that can be merged with your survey.  Data that is commonly stored and might be useful in survey research includes: 

• Pre-enrollment data (e.g., demographics, high school GPA, parents’ experience with college, test scores)
• Academic data (e.g., course attendance, mid-term grades, GPA)
• Campus life data (e.g., housing assignments, participation in student organizations)
• Use of support services (e.g., advising center, counseling center, health center, tutoring)
• Swipe card data (e.g., access to library, dining hall, recreation center)

When requesting institutional data, ask for a unique student identifier (e.g. a student ID, student email) that can be used to link those data with your survey.

AIR and Survey Research

Why is someone from the Association for Institutional Research (AIR) writing a blog post for NASPA about survey research? AIR is a membership organization for higher education professionals around the world who are involved in institutional research, assessment, effectiveness, and planning; nearly one-third of members are assessment professionals!

To improve higher education’s survey research skills, AIR offers several resources:

• AIR’s annual conference: The 2018 conference had nearly 30 concurrent sessions on survey research;
• Webinars:  AIR hosted two webinars in April 2018 discussing survey research techniques where are available to view
• Online Course:  And, we area wrapping up our new online course devoted exclusively to survey research focused on student affairs.

If you’re interested in learning more about AIR or accessing our resources, please visit our website, www.airweb.org.