Education news

An AI Made Students Think Harder? Check Who Did the Scoring.

A Utah district says its AI chatbot raised students' critical thinking by 28 percent. But the vendor wrote and scored the study on its own product, with no control group. Read past the headline and the company's own data points somewhere far more useful than the tool itself.

Laura Lurns Last updated September 15, 2026 · 9 min read

An AI Made Students Think Harder? Check Who Did the Scoring.

If your child’s school switched on an AI chatbot this year, you have almost certainly heard the pitch that arrives with it. The kids are engaged. Usage is climbing. Conversations are everywhere. This month a large Utah district put a number on that pitch, reporting that two years with an AI tool lifted students’ critical thinking by 28 percent. The figure is traveling fast, and it deserves a slower read, because the people who produced it also happen to sell the product it praises.

TL;DR

Jordan School District in Utah, serving about 55,000 students across 68 schools, partnered with the education-technology company SchoolAI in 2023 and built a moderated student-to-AI chat feature.
A SchoolAI case study reported a 28 percent rise in a critical-thinking score and more than a doubling of conversations reaching higher-order reasoning between October 2023 and October 2025, across 82 teachers and nearly 14,000 conversations.
The study was written and scored by SchoolAI itself, used no control group, and carries an ESSA Tier 3 rating following external review by Instructure, a fellow education-technology company that makes the Canvas platform.
By the company's own promotional materials, the significant gains tracked teachers who built six or more deliberately designed “Spaces,” not teachers who simply had access to the tool.
District administrator Jared Covili said the tool is built to push students past recall and stressed keeping teachers in the loop: “You don't want to take the human out of the loop.”

Context the coverage leaves out

What the coverage gets wrong

The story traveled as a clean win: a district deploys AI, and critical thinking climbs 28 percent. To their credit, the fuller reports named the source as a SchoolAI case study and quoted the district's guardrails. What most coverage skipped is what that source actually is. It is a company grading its own product, with no control group, no independent evaluator, and an evidence rating that sits near the bottom of the federal scale. Treating that as proof that AI raised student thinking is the oldest move in education technology, counting activity and calling it learning. The genuinely useful finding is buried inside the vendor's own materials: the gains tracked how deliberately teachers designed the work, not whether students had access to a bot. That is the story, and it is close to the opposite of “the AI did it.”

AI is arriving in classrooms fast, and parents are left sorting the hype from the help. Here are the questions parents are actually asking.

Common questions

Does my child using an AI chatbot at school mean they are learning more?

Not on its own. Usage numbers, conversation counts, and time on a device measure whether a child is busy, not whether they learned anything. What tends to teach is design that makes a child do the effortful thinking, then responds to where that thinking went, rather than a tool that hands over answers. Ask what the tool is built to make your child do.

My child’s school is bringing in AI. What should I ask?

Ask two things. First, what is the tool designed to do, prompt a child’s reasoning or produce the work for them? Second, how are results being measured, by activity like logins and conversations, or by whether children actually got better at something? In the Utah case that made headlines, the gains tracked how carefully teachers designed the lessons, not whether students had access to a bot.

How do I tell whether a learning tool is building skills or filling time?

A tool that builds a skill asks your child to do the hard part, to recall it, reason through it, get it wrong and try again, and it gives feedback on where the reasoning broke down. A tool that fills time rewards clicking, streaks, and time on screen. The first is effortful and sometimes frustrating; the second is smooth. Smooth is often the warning sign.

How do I find out where my child actually needs help?

Start by looking at the whole picture rather than a single subject. The Learning Success learning difficulties analysis asks what you are already noticing across reading, writing, math, and attention, and points you toward where to begin. A screener is a starting point, not a diagnosis. If your child might need formal accommodations such as an IEP or 504 plan, or you suspect a vision, hearing, or medical cause, pursue a professional evaluation too, since that is the only route to those supports.

Share the idea

A Utah district credits an AI chatbot with a 28% thinking gain. But the vendor scored its own study, no control group. What actually moved kids, by its own data: how teachers designed the work.

Post on X Follow

What the Utah district reported

Jordan School District sits in West Jordan, Utah, and serves roughly 55,000 students across 68 schools, one of the state’s largest. Starting in 2023 it partnered with an education-technology company called SchoolAI, and out of that partnership it built a moderated chat that lets a student talk with an AI bot under a teacher’s framework. This month the two-year results drew national coverage.

The headline numbers come from a case study titled “SchoolAI Makes Students Think.” Looking at the same 82 teachers across nearly 14,000 student-to-AI conversations, and comparing October 2023 with October 2025, it reported a 28 percent rise in a critical-thinking score and more than a doubling of conversations that reached the top of a four-level rubric adapted from Bloom’s Taxonomy. Jared Covili, the district’s digital teaching and learning administrator, told K-12 Dive the tool is built to push students past recall. “It goes beyond low-level sharing of facts,” he said. “We’re asking kids to go beyond that and think about the issues involved. It’s more than just having a basic understanding of a concept.”

Here is the part the coverage tends to hurry past. That study was written, published, and scored by SchoolAI itself, about its own product. It had no control group and no matched classrooms learning without the tool. Its “ESSA Tier 3” label is one of the weaker rungs on the federal evidence ladder, a promising correlational rating rather than a controlled-trial finding, and the external review behind it was performed by Instructure, the company that makes the Canvas learning platform, not by an independent research body. Covili himself kept the human at the center of the story. “You don’t want to take the human out of the loop,” he said. “We want teachers connecting with their kids.”

It goes beyond low-level sharing of facts. We’re asking kids to go beyond that and think about the issues involved. It’s more than just having a basic understanding of a concept.
Laura Lurns · Learning Success expert

Engagement is not the same as learning

There is a reflex in how new classroom technology gets reported, and this story shows it in the open. A tool arrives, students use it a lot, conversations pile up, and the sheer volume of activity becomes the evidence that learning happened. That move has a name in the research on educational technology: engagement mistaken for learning. Time in an app, lessons completed, conversations logged, these are engagement metrics. They measure whether a child is busy. They do not measure whether a child got better at anything.

What the cognitive-science research points to is a different question, and it is worth stating plainly even though none of it studied this particular tool. Kulik and Fletcher’s 2016 review of tutoring systems found that effects looked much larger when the test was built to match what the program drilled than when the yardstick was an independent, standardized measure. Line that up against this case. The outcome here is the platform’s own chat transcripts, scored against a rubric the platform chose, reviewed by a fellow technology company. That is close to the textbook example of a program grading itself on its own aligned test, which is exactly where apparent benefits inflate. The mechanism a well-designed AI could genuinely be using is real, and VanLehn’s work points to it: tutoring helps most when it responds to a child’s reasoning step by step rather than only checking the final answer. But that has to be shown, not assumed from how many conversations happened.

And here is the most telling result in the whole report, tucked inside SchoolAI’s own promotional materials rather than the news write-ups. The gains did not spread evenly across every teacher who had the tool. By the company’s own account, teachers who built six or more deliberately designed learning experiences saw the movement, while teachers who offered generic, open-ended access did not. That is the vendor’s own data quietly undercutting the “the AI did it” headline. The medium is not what teaches. The design underneath it is. It is the same instinct a parent brings to screens and digital habits when they stop asking how much time and start asking what the time is building. A points system makes a child feel like they are learning. Real mastery makes them feel capable.

Key takeaways

Engagement is not learning: Conversation counts and time in an app measure whether a child is busy, not whether they learned anything.
The seller graded the exam: A vendor wrote and scored this study on its own product, with no control group and no independent evaluator.
The design did the work: By the company's own data, gains tracked teachers who built structured lessons, not chatbot access alone.

What a parent actually does with this

If your child’s school is rolling out an AI tool this year, the useful thing you hold is a question, not a verdict. Ask what the tool is designed to make your child do. Does it hand over the answer, or does it make your child do the thinking and then respond to where that thinking went wrong? A tool built to withhold the answer and prompt the next step is a different animal from a chatbot that writes the essay. The district’s own line drew that boundary well. “We don’t want to turn over student writing to AI,” Covili said. “We want students creating their own work. We don’t want AI bots grading the work.”

The tradeoff is real, and it is yours to weigh. This is the same lesson hiding inside every debate over mandatory dyslexia screening. Screening a whole population sounds like unambiguous good news, and it helps enormously when the result steers a family toward the right support. It does harm when the result hardens into a label that lowers expectations, or when a support gets bolted on without anyone asking what it is for. AI in the classroom has the same shape. Used with structure, an AI conversation partner frees a teacher to sit with the children who are stuck. Used as open-ended access with no design behind it, it drifts back toward activity for its own sake. A self-scored study with no comparison group does not settle which of those your child is getting. You are the one positioned to watch whether a tool is building the skill or standing in for the effort, and our overview of how the approach works is built around that question.

So hold the number lightly and hold the question tightly. The tool is neither the hero nor the villain of your child’s year. What it is built to make them do, and who is watching while they do it, will decide that far more than any percentage in a press release.

What we saw over two years wasn’t students becoming dependent on AI for answers, but rather students learning to think harder because the AI was designed to expect more of them.
Laura Lurns · Learning Success expert

Your child is capable of real thinking, and you are the person best placed to tell whether a shiny new tool is building that or keeping them busy. The villain here is not AI, and it is not a district working hard to serve its students. It is the old habit of counting logins and conversations and calling it learning, and of letting the company that sells the tool be the one that grades the exam. Learning Success was built the other way around, on the design underneath the activity, with you coaching the practice that makes it pay off. Our All-Access membership opens an assessment that asks about the skills your child’s learning runs on, and a roadmap that names what to build first.

See what All Access gives your child

Is your child struggling in school?

Get your free personalized learning roadmap

You describe what you see at home. We turn it into a plan you start this week.

Answer 5 short questionnaires about what you already notice, 30–45 minutes at your own kitchen table
Your child sits no test and gets no score: nothing to schedule, nothing for them to dread
You do the answering, the AI does the writing, and a person reviews it before it reaches you
Access all 40+ courses instantly: reading, math, focus, processing and more, with new ones added regularly

Why we use AI, plainly: it writes from a knowledge base our team maintains and audits. We work through it line by line and pull anything the evidence stops supporting. The roadmap you get on Tuesday reflects what we corrected on Monday, and a human still reads it before you do.

Your school district must evaluate your child free of charge if you ask in writing, whatever your income and whatever the outcome (US, 34 CFR 300.111 and 300.301(b)). That route takes time and answers a different question than you do. This one starts today, from what you already know.

Get my free assessment & 7-day trial Cancel anytime during your trial.

Your answers stay yours. We do not sell your personal information, and we do not hand identifiable assessment data to outside AI companies to train their models.

A screener is a starting point, not a diagnosis. If your child might need formal accommodations (an IEP or 504 plan), or you suspect a vision, hearing or medical cause, pursue a professional evaluation too. That is the only route to those supports.

References

SchoolAI — SchoolAI Makes Students Think (vendor case study)
K-12 Dive — Utah district reports critical thinking boost amid AI deployment
GlobeNewswire — SchoolAI Earns ESSA Tier 3 Certification, Backed by Research Showing 28% Critical Thinking Gain
K-12 Dive — Week in Review, Sept. 14, 2026
Fox 13 Now (KSTU) — Jordan School District uses AI to help students learn, not cheat

Laura Lurns · Learning Success expert Writes about the learning brain for parents who want plain answers. Every article is grounded in current neuroscience and classroom practice.