Useful Evidence of Student Learning

Determining the extent to which students have achieved intended program learning outcomes requires collecting evidence that

  • demonstrates what students know and are able to do with that knowledges, together with complementary information about 

  • students perceptions of their learning, and of the learning environment that supported that learning. 

This page describes the different forms of evidence, as well as example of how to use them in a complementary fashion to generate actionable information about student learning. 

Forms of Evidence of Student Learning

Evidence of student learning takes two general, and generally complementary forms: direct and indirect. 

Direct Evidence

Direct evidence of student learning is the actual work that students produce - the materials they generate in the process of learning and to demonstrate what they are able to do. Students can also directly demonstrate behaviors (e.g. team work), attitudes (inquiry), and kinesthetic skills. 

Direct evidence, then, is the tangible, visible demonstration of student abilities as revealed through diverse kinds of assignments, including papers, portfolios, presentations, exams, group work, board work, homework, etc. 

Direct evidence is important because it provides allows faculty to tangibly determine:

  • what students know and can do, as well as,
  • how well they can do it in relation to the intended learning outcomes and established expectations for performance

Examples of Direct Evidence for Undergraduate Programs

  • Capstone projects
  • Portfolios
  • Exams
  • Papers
  • Presentations
  • Think alouds
  • Student reflections on values, attitudes and beliefs
  • Employer or supervisor evaluations of student abilities
  • Student responses to survey questions that ask them to demmostrate knowledge

Examples of Direct Evidence for Graduate Programs

  • Qualifying/Candidacy Exam
  • Dissertation Proposal
  • Thesis/Dissertation
  • Thesis/Dissertation Defense
  • Annual Student Progress Evaluation (can be both formative and summative)

For program assessment, assignments that are intentionally designed to engage students in a project or problem that 

  • is connected to "real world" (authentic; including the world of academia)
  • is meaningful to them
  • engages more than one program learning outcome, concurrently, and in an integrative fashion
  • provides students with the time and incentives to do their best work1

are the best sources of evidence and learning. These kinds of assignments can be "signature work"..... 

1For example, an effective assignment might provide students with the time to plan and revise their work, and have a point value commensurate with the effort expected of students. 

Indirect Evidence

Indirect Evidence provides insight into:

  • Why students are able to do what they can do, and
  • How they have learned what they have learned to the degree they have

Forms of indirect evidence reveal characteristics associated with learning but only imply that learning has occurred. They also include students' self-perceptions of their learning and factors that influence student learning outcomes, like the student experience of the learning environment.

Gathering indirect evidence is done through surveys, group interviews, and focus groups. 

Examples of Indirect Evidence for Undergraduate Programs:

  • Surveys of student attitudes
  • Curriculum maps and course articulations
  • Counts of types of assignments
  • Counts of types of learning experiences in and out of the classroom
  • Interviews with teaching assistants about their perceptions of student learning opportunities
  • Descriptive data- demographics, etc.

Examples of Indirect Evidence for Graduate Progams

  • Annual student progress evaluations (can be both a form of direct and indirect evidence)
  • Graduate student focus groups
  • Number of publications and professional presentations
  • Course grades
  • Curriculum maps
  • Exit interviews/surveys
  • Alumni surveys

Complementary Evidence

Combining forms of evidence is about more than choosing a line of direct evidence and a line of indirect evidence. The goal is for the direct and indirect evidence to inform each other to provide information on what students know and can do and how and why they know and can do it.

For example, faculty might choose to assess the Capstone project students turn in during their senior year. In addition to a rubric-based assessment of the quality of the project, faculty might also collect the following indirect evidence:
 
1) A written reflection on how the paper illustrates achievement of one or more PLOs, what they are doing well, and what needs more practice
 
2) The number of papers, of this type, that a student writes prior to this assignment
 
These pieces of indirect evidence provide insight into the students' thinking skills and areas of knowledge that may need attention, as well as the environment in which the learning took place. Viewed holistically, faculty will be able to determine potentional changes in the curriculum that will enable improve student learning outcomes on the Capstone project.
 
Campus Resources for Collecting Indirect Evidence
 
SATAL Program

The Students Assessing Teaching and Learning (SATAL) Program offered by the Center for Research on Teaching Excellence (CRTE) will support FAOs with  data collection and reporting by conducting focus groups, surveys, class interviews, among other options, with no charge to its users. 

​Institutional Surveys & Statistics

Institutional Research & Decision Support (IRDS) provides information on student enrollment and demographics. They also partner with units across campus to administer surveys to students, faculty, staff, and alumni. 

 
  • gathering evidence that yields actionable insights into student learning achievement and the student experience in relation to the expected program learning outcomes. The most effective strategies involve complementary lines of direct and indirect evidence designed to represent the cumulative impact of the program's curriculum on student learning and success at a given point in the degree.
  • using broadly shared, programmatic criteria and standards to evaluate the evidence. Usually elaborated in the form of a rubric, the criteria and standards describe what students are able to do at the time of graduation if they have achieved a given program learning outcome.
  • discussing results as a faculty and other stakeholders as appropriate, including graduate student instructors in the program.
  • identifying and implementing actions to improve student learning and/or the student experience, as appropriate, recognizing that the action of making no change is also possible.

Tips for identifying useful evidence: Imagine the results, and how you would respond with action....examples. 

Actionable Information

In order to generate "actionable" information , i.e. information that suggests or points to one or more courses of action to improve student learning, it's important to collect quality and complementary forms of evidence along the degree path (i.e. the formative level) and at the time of graduation (i.e. the summative level). For instance, student grades might tell us how well students performed in a given class. However, if the grades are lower than expected, we don't know why they were lower or how the course might be improved to raise them. We need other forms of evidence to explain the lower grades and determine a course of action. 

Additionally, combining different forms of complementary evidence has the benefit of providing insight into what students have learned by also explaining how  they have learned it. This information together can produce a clear picture of what steps to take to improve instruction and/or curriculum delivery.

Alignment - as a factor in chosing evidence