OpenAI and Bocconi find ChatGPT and reasoning training aid different skills

OpenAI’s economic-research team and researchers at Bocconi University in Milan published results from a randomised experiment testing how access to ChatGPT and training in causal reasoning affected student work, separately and in combination. More than 1,000 first-year undergraduates worked on a real business case — developing marketing recommendations for the university’s own merchandise store — after being randomly assigned by class to one of four conditions: access to ChatGPT running on GPT-4o, a short course in causal reasoning, both, or neither.

Access to ChatGPT alone improved the coherence and overall quality of students’ written recommendations; one account of the results put the improvement at nearly a full point on a five-point rubric. The causal-reasoning training alone did not raise that score, but led students to produce a wider range of distinct ideas. Students who received both interventions improved on both measures.

The reasoning course itself involved no AI at all — a game, worked examples, questions and feedback aimed at teaching students to link cause and effect — which let the researchers treat coherence and originality as separable effects, one produced by a tool and one by teaching, that combined for a larger gain than either alone. As with other industry-run education research, the study was designed and published by a company with a direct commercial interest in demonstrating that its product improves learning outcomes, working alongside independent academic co-authors at Bocconi.