Would You Know Either Way?

Nearly a thousand students took part in a randomized study on AI and learning. The ones whose scores were quietly getting worse did not notice it happening. The ones whose scores hadn't actually improved were convinced they'd gotten sharper. Both groups were wrong, in opposite directions, and neither one could tell.

The experiment

Researchers built two versions of a GPT-4 tutor for math classes at a high school in Turkey, nearly a thousand students across three grades. One was a plain chat interface, close to using a general AI chatbot directly. The other used the same underlying model with teacher-written prompts designed to protect learning, guiding students toward an answer instead of handing it over. Students practiced with one version across several sessions, then took a test with no AI, no notes, no help.

The plain version helped a lot during practice: scores rose 48% compared to a group with no AI at all. But on the unassisted test, those same students scored 17% worse than the no-AI group. The guided version helped even more during practice, a 127% jump, and caused no measurable harm on the test afterward. Same underlying model, same students, opposite result on the one measure that actually matters: what happens when the tool isn't there. The researchers published it under a title that states the finding outright: “Generative AI without guardrails can harm learning.”

The part that should worry you more than the number

The researchers also asked students how they thought they'd done. The ones who'd used the plain version and scored worse did not think they'd performed worse or learned less. The ones who'd used the guided version and hadn't actually improved were convinced they had.

That's the finding that matters most if you run a business with people in it. Self-report doesn't just under-perform here, it's disconnected from reality entirely. If you ask your team whether AI is helping them, the answer won't track what's actually happening to their skills, because the people in this study couldn't track it about themselves either.

It isn't one lucky result

A separate study recruited 1,222 adults, not students, for a series of randomized experiments on math and reading tasks. People given unrestricted access to an AI assistant performed worse and gave up more often once it was taken away, compared to people who'd had no access at all, after as little as ten to fifteen minutes of use. When the researchers asked people afterward how they'd actually used it, the ones who said they'd leaned on it for direct answers showed the largest drop. The ones who said they'd used it mainly for hints showed a smaller drop, but a real one, not zero. Even the gentler way of using AI here still left a mark. The researchers describe the overall finding as the first large-scale causal evidence of this effect. Earlier work was either correlational or built on much smaller samples. (For a different angle on where trust in AI breaks down, see our earlier post Should I Trust AI?)

And here's the part we're not going to leave out

A third study randomized 1,174 adults between 25 and 45 into a workplace-style business problem-solving task, done online for real incentives, with a plain, unmodified AI assistant. It found something different: no average drop in performance once the AI was removed. If anything, AI narrowed the performance gap between higher- and lower-education participants by roughly three-quarters while it was in use.

We're not going to pretend that result doesn't exist because it complicates the story. Three studies, three outcomes: harm from an unguarded assistant in one, harm specifically tied to direct-answer use in another, no measurable harm in a third. What holds across all three is that the setup decides the outcome, not the model. What doesn't hold is the flat claim that plain AI access always causes harm. In at least one adult workplace setting, it didn't.

It shows up outside the classroom too

Experienced endoscopists across four medical centers in Poland adopted AI-assisted screening. In the months afterward, their detection rate on ordinary, non-AI-assisted exams dropped, from 28.4% to 22.4%. This wasn't a randomized study, so it can't prove AI caused the drop by itself, and the researchers are careful to say it only “might” have. But the skill in question was spotting the growths that can become colon cancer, and the pattern matches everything else in this post: the tool did the work, and the ability to do it alone quietly slipped.

What to actually do about it

You can't ask your team whether AI is making them worse at their jobs and trust the answer. Build a check instead. Every so often, have someone do the task without the tool and see what happens. Not because you assume the worst, but because feeling fine isn't information here, in either direction.

That's the same discipline behind a Map of Gaps: the number has to come from your own operation, not from an average, and not from how anyone feels about it.