A Harvard AI tutor more than doubled what an active-learning classroom taught. A chatbot handed to high schoolers left them seventeen percent behind kids who never had it. Same class of tool. The difference was what each design made the human do before the answer arrived.
This episode returns to chapter six of AI Empowered, the collaboration demands more of you, not less, and pays off the half of the argument that “The Deskilling Trap” (July) left standing alone. That episode used Bastani et al.’s PNAS field experiment (about 1,000 Turkish math students) to show what unscaffolded AI costs when the tool goes quiet. This one adds Kestin et al.’s crossover RCT in Scientific Reports (194 Harvard introductory-physics students), where a GPT-4 tutor built on the course’s own pedagogy, and told to guide rather than answer, produced more than double the median learning gains of the in-class version, in less time, with higher reported engagement. Set side by side, the two studies isolate design, not model capability, as the variable.
Aaron carries both studies’ boundaries in the script: Kestin measured immediate gains only and says nothing about retention or deskilling; it compared two scaffolded designs, not AI against human, and no plain chatbot arm was tested; the at-home, self-paced delivery is confounded with the tutor; the material sat at the understand-apply-analyze level, not synthesis or professional judgment; and both samples are students, so the extension to knowledge work is the book’s inference and is labeled as such. The work anchor is a design rule in the spec for the AIRAA (pronounced “era”) app’s coaching feature: ask what the user thinks first, ask for evidence, refuse to write the work product, end every exchange with a decision the user makes. Dependency on the system is treated as a failure mode. The turn to the listener is one audit question for one workflow: what does it make the human do before the answer arrives?
Studies:
Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning. Scientific Reports, 15, 17458.
Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning. PNAS, 122(26).
Earlier episode: The Deskilling Trap (July 2026).
—
This show extends the book AI Empowered: The Psychology of Extraordinary Human-AI Collaboration. Read the first chapter free at https://www.aiempoweredbook.com, or get the book on Amazon.
Aaron Douglas runs Auspicious, a fractional marketing practice.
Blog: https://auspicious.llc/insights/
X: https://x.com/aaron_douglas
LinkedIn: https://www.linkedin.com/in/aaronddouglas/
