Student Success by DesignA practitioner’s guide to theory, data, and AI in student success

8.2 · Equity and Student Experience

Algorithmic bias and differential impact

Predictive models can be accurate on average and still wrong in patterned ways for particular groups.

Short shows the summary, takeaway, and questions.

Research on college success prediction has found that common models under-predict success for Black and Hispanic students and over-predict it for White and Asian students. In practice, this can mean more Black and Hispanic students are flagged as at risk when they would have succeeded, and some students who need help are missed.

Removing race from a model does not remove the problem, because other variables, such as high school, ZIP code, and financial aid status, carry the same patterns.

What to ask of any model

  • Is accuracy reported separately for each student group?
  • What happens to a flagged student: support, or restriction such as advising holds or steering away from majors?
  • Who reviewed the model for bias, and when?
  • Can a student see why they were flagged and challenge it?