Student Success by DesignA practitioner’s guide to theory, data, and AI in student success

3.7 · Student Success Frameworks

What the evidence supports

Causal evidence is strongest for comprehensive programs and specific teaching practices, not for frameworks as a whole.

Short shows the summary, takeaway, and questions.
Interactive chart · Strength of causal evidence

Select a row for the details.

Comprehensive support (CUNY ASAP and replications)Strong

Evidence type: Randomized. Randomized trials. CUNY: three-year graduation about 40% vs. 22%. ASAP Ohio: 35% vs. 19% at three years, 46% vs. 31% at eight, with year-eight earnings $3,337 higher. CUNY fourteen-year follow-up: 57.7% vs. 50.0% degree attainment, with no detectable earnings effect in New York State data. The finding is for the full package.

Active learning in STEMStrong

Evidence type: Meta-analysis of experiments. Freeman et al. (2014): exam performance up about 0.47 SD; lecture students about 1.5 times as likely to fail. Theobald et al. (2020): gaps for underrepresented students narrowed 33% on exams and 45% in pass rates.

Need-based grant aidStrong, small

Evidence type: Meta-analysis of causal studies. Grants raise persistence and attainment by about 2–3 percentage points; roughly 1.2 points per additional $1,000.

Individualized coachingStrong, specific

Evidence type: Randomized. Bettinger and Baker (2014): 3.4-point persistence gain after 24 months for the coaching model tested.

Brief belonging interventionsModerate

Evidence type: Randomized. Walton et al. (2023), 26,911 students at 22 institutions: gains for some groups where real opportunities to belong existed. A 2026 commentary notes null results for a prespecified hypothesis.

Faculty–student relationshipsAssociational

Evidence type: Cross-sectional and longitudinal (meta-analysis). Associated with academic outcomes (r ≈ .18) and persistence (r ≈ .33). These are associations, not effects of assigning mentors.

Guided PathwaysMixed

Evidence type: Longitudinal observational. About 100 colleges studied. Positive associations in Tennessee; no detected improvement after statewide initiatives in Ohio or Washington. Not causal.

Early-alert systemsMixed

Evidence type: Observational and randomized (mixed). An Australian study (16,091 students) found links to retention; a German randomized evaluation of warning emails found no average gain. Prediction is not intervention.

Key claims, classified
Evidence typePopulation and contextSampleOutcomeMain limits
ASAP OhioRandomized trialLow-income students at three Ohio community collegesAbout 1,500 studentsDegree attainment at 3 and 8 yearsFull-time students only; package cannot be separated
Active learning (Freeman et al., 2014)Meta-analysis of experimental and quasi-experimental studiesUndergraduate STEM courses225 studiesExam scores, failure ratesSTEM only; possible publication bias
Active learning and gaps (Theobald et al., 2020)Meta-analysisUndergraduate STEM courses with disaggregated dataStudies reporting results by groupExam scores, pass rates for underrepresented studentsEffect depends on intensity of active learning
Grant aid (Nguyen et al., 2019)Meta-analysis of causal studiesU.S. postsecondary students43 studiesPersistence, degree completionEffects vary by program design
Coaching (Bettinger and Baker, 2014)Randomized trialStudents at several institutions using one coaching providerAbout 13,500 studentsPersistenceOne provider’s model
Belonging intervention (Walton et al., 2023)Randomized trialFirst-year students at 22 institutions26,911 studentsFirst-year full-time completionGains only where real opportunities to belong existed
Guided PathwaysLongitudinal observationalCommunity colleges in several statesAbout 100 collegesEarly momentum, completionNot causal; results differ by state
Early alertObservational and one randomized evaluationAn Australian university; a German university16,091 students (Australia)RetentionObservational links; randomized warning emails showed no average gain