Student Success by DesignA practitioner’s guide to theory, data, and AI in student success

6.5 · Theory–Technology Bridge

Evaluation beyond retention

Retention and graduation are lagging outcomes. Theory points to the processes that come before them, and those can be measured too.

Short shows the summary, takeaway, and questions.
Belonging
Validated sense-of-belonging scales administered before and after an intervention, disaggregated by student group.
Equity-gap closure
Whether gaps between groups narrow. A tool can lift the average while widening a gap.
Involvement and engagement
Participation in advising, tutoring, and co-curricular activity, read alongside student-reported effort.
Validation
Whether students report that faculty and staff reached out, believed in them, and treated their experience as relevant.
Thriving
Academic, interpersonal, and psychological well-being beyond simply enrolling next term.
Qualitative evidence
Interviews, focus groups, and open responses on how students experienced the tool.
Institutional capacity
Staff time per student, response time to alerts, and whether alerts lead to real contact.

Disaggregation in practice

  • Report every outcome by race and ethnicity, Pell status, first-generation status, disability, age, and attendance pattern where numbers allow.
  • Check model accuracy and outcomes for each group.
  • State in advance what result would cause you to pause or change the tool.