A tool that cannot answer the first question is optimizing for a metric, not a mechanism. A tool that cannot answer the second cannot be improved. A tool that has not considered the third will find out in production, on students.
Three questions to ask of any tool
- Which theoretical mechanism is this tool intended to support?
- How will we know if it is working?
- What could go wrong?
- Student success theoriesThe mechanism
- Design questionsThe bridge
- Technology, data & AIThe application
Start every project in the left column. If a tool can’t be traced back to a mechanism, it is optimizing a metric.