Navigating Assessment Politics
Assessment, Evaluation, and Research
April 27, 2023
Assessment is inherently a political activity, as it involves the allocation of resources, decision making, and power.
Gavin Henning and Darby Roberts make this point introducing a chapter on politics in their 2016 book Student Affairs Assessment: Theory to Practice. This intersection of assessment and politics is recognized in the ACPA/NASPA professional competenciesin the assessment, evaluation, and research area as “values/ethics/politics” (2016). A foundational-level outcome in this competency worthy of further discussion is that effective student affairs professionals “handle data with appropriate confidentiality and deference to organizational hierarchies”.
The first element of this outcome mentions confidentiality, which involves deciding when to report and when not to reportassessment results with various stakeholders. Confidentiality is not anonymity. Unlike anonymity, someone has identifying information or other details about an assessment that exceeds what is legally, ethically, or politically savvy to share with a wider audience. One best practice is to only share the minimum information necessary to complete a business need. For example, if a recipient can conduct their business with an anonymous dataset, they should not receive identifiable data. If a summary report is sufficient, they should not receive a dataset at all.
The second element of this outcome mentions organizational hierarchies, which are made up of stakeholders. Stakeholders are anyone with an interest in the topic being assessed and/or the data involved. They may be classified into binary categories of internal to a unit versus external to a unit. More often, stakeholders are at least on a continuum: the most internal parties are directly involved in a project; less internal parties include a direct supervisor, participants, and key collaborators; university administrators might be more towards the external end of the spectrum, and fully external stakeholders include outside entities with interest but little to no authority.
Deciding whether to share information varies by stakeholder but follows a sequential decision tree. The first factor is whether it is legal to disclose or not disclose. If it is legal, the second factor is whether it is ethical to disclose or not disclose. If it is ethical, the final factor is whether it is politically savvy to disclose.
There are somelegal elements that require nondisclosure, others that require disclosure. For example, FERPA-protected data (such as GPA tied to student identity) generally cannot be disclosed to anyone without a business purpose, which involves a university faculty/staff person needing the information as part of conducting work detailed in their job description. On the other hand, Title IX has elements of mandatory reporting, such as certain university staff being required to notify Campus Police if a student alleges sex or gender discrimination.There is also the consideration of public/open records laws, which allow public university records to be shared upon request unless limited by other laws.
If information can legally be shared, the next factor is ethics. Some ethical considerations are formalized at institutions, such as research protocols overseen by institutional review boards. For example, protection of human subjects includes the requirement to inform participants about the costs and benefits of a study, and how their data will be shared,before freely choosing whether to consent. An increasing number of institutions also have formal data governance structures, including protocols for sharing data.Examples include having a minimum cell size in tables to avoid identifying individuals,while also disaggregating to report any differential results of subpopulations.
If information can ethically be shared, the final factor is political impacts, starting with whether information must be or can’t be shared. Some stakeholders, such as a Board of Trustees, Board of Governors, and accrediting agencies, can mandate the release of information. Some stakeholders can also ask or require information to not be disclosed, say out of concern for negative implications.
After navigating this decision tree, through legal, ethical, and political factors, there may remain the question of whether information should be shared. This is somewhat interconnected with how to share information. Here are several strategies to manage the politics of sharing information:
-
When planning an assessment project, consider upfront the anticipated use of results, including when those results are opposite expectation. This will help prepare for possible outcomes as well as decide whether the assessment is necessary.
-
Collaborate with internal and external stakeholders throughout the project to improve buy-in and applicability of results.
-
Identify and address alternative interpretations of information, which are especially possible in the briefer reporting formats.
-
Stage a rollout of results, starting with the most internal stakeholders, so they can contribute to interpretation before sharing context with a wider audience.
-
Share information in a format and level of detail fitting the needs of the recipient. Preferences include succinct talking points, executive summaries, presentations, or raw datasets.
-
Especially when sharing results counter to preconceived ideas, e.g., “negative findings”, proactively note any weaknesses evidenced in the project, and pair results with action plans, so results stay constructive.
-
Map assessment to strategic plans, so the purpose of the assessment is linked to intentional advancement of shared objectives.
-
Tell the story of how results were used, including telling who gave information, such as student participants, so stakeholders see a return in the investment of their time.
These strategies address a small facet of the intersection between assessment and politics. This intersection becomes more constructive not just by navigating the politics of each project individually, but also by holistically building a culture of assessment, which is valuing the use of evidence to continuously improve.Among other qualities, institutions with a culture of assessment use a variety of methods, have assessment embedded in everyday practice, share results transparently, celebrate assessment efforts regardless of outcome, and as a rule act on results.
This is a primer about navigating assessment politics. Continue learning and applying these concepts by reading further, such as Henning and Roberts (2016), talking through considerations with professional colleagues such as those in the NASPA Assessment, Evaluation & Research Knowledge community, and otherwise thoughtfully starting each assessment project with the end in mind, using result to make decisions that improve our service to students.
References
Henning, G. W., & Roberts, D. (2016). Student affairs assessment: Theory to practice. Stylus.