Students Overestimate AI Skills by 32 Pts

Students are the most overconfident group in our dataset when it comes to AI skills. Across 104 students who predicted their scores before taking the AISA assessment, the average predicted score was 66.2 — solidly in the Proficient tier. The average actual score was 34.1, barely inside the Developing tier. That 32.1-point prediction gap is the largest of any role we measure, nearly double the overall average gap of 19.4 points.

This is the generation that grew up with ChatGPT, Copilot, and AI-generated everything. They use these tools daily. They assume that familiarity equals fluency. The data says otherwise.

The 32.1-Point Student Prediction Gap

Students overestimate their AI skills by 32.1 points on a 100-point scale — more than any other professional role in the AISA dataset. This isn't a rounding error or a small sample artifact. It's 104 students predicting they'd land in the Proficient tier and actually landing in the Developing tier.

To understand the scale, compare it to other roles:

Engineers overestimate by 15.7 points. Founders by 8.2. Product managers by 9.4. Students blow past all of them at 32.1. And notice something else: students don't just have the biggest gap — they also have the lowest actual score of any role in the prediction cohort. Engineers who overestimate still land at 53.6. Students land at 34.1.

The prediction gap pattern we see across all roles is well-documented — we've written about it in the context of professionals broadly and career changers specifically. But the student gap is in a category of its own.

What a 32.1-Point Gap Actually Means

A 32.1-point gap means students are miscalibrated by an entire tier and a half. They think they're Proficient (60-79). They're actually Developing (28-59) — and barely Developing at that, sitting at 34.1. In confidence calibration terms, this is a severe case of the Dunning-Kruger effect applied to an entire demographic.

How This Compares to Career Changers

Career changers — people transitioning into new fields — score 36.8 on average and show significant overconfidence too. But even career changers, who are explicitly entering unfamiliar territory, don't show a gap as dramatic as students. The student gap is unique because students believe they're already skilled, not that they're becoming skilled.

Student AI Skills by Dimension: Where the Gaps Are Deepest

The composite score of 36.0 (across 181 students in the full dimension dataset) tells one story. The dimension breakdown tells five.

Students score below the overall average in every single dimension. The smallest gap is in prompting (−7.9), which makes sense — prompting is the most visible, most practiced skill. The largest gaps are in workflow (−12.2) and safety (−11.2), which are the dimensions that require real-world context and deliberate practice.

Prompting: Familiar but Shallow

At 36.3, student prompting scores sit in the Developing band. This is surprising if you assume daily ChatGPT use translates to prompting skill. It doesn't. The AISA rubric evaluates whether candidates can articulate why they structure prompts a certain way, whether they use techniques like chain-of-thought prompting deliberately, and whether they can adapt their approach when initial outputs fall short. Most students we assess can describe what they type into ChatGPT. Few can explain the reasoning behind their approach or demonstrate iterative refinement strategies.

Critical Thinking: The Missing Layer

At 34.3, critical thinking is the second-lowest dimension for students. This dimension measures whether candidates verify AI outputs, identify potential hallucinations, cross-reference claims, and maintain appropriate scepticism. A 2024 study published in Nature Human Behaviour by Luo et al. found that participants who used AI assistants showed reduced analytical thinking on subsequent tasks — a pattern the researchers termed "cognitive offloading." Students who've grown up with AI as a default research tool may be particularly susceptible to this pattern. Our data is consistent with that hypothesis: students score 8.0 points below the overall average on critical thinking, suggesting they accept AI outputs with less scrutiny than working professionals.

Technical Understanding: The Weakest Dimension

At 30.6, technical understanding is the lowest-scoring dimension for students. This covers how models work, what tokens are, why context windows matter, and what the practical implications of model architecture are for everyday use. A score of 30.6 places the average student squarely in the Developing band. They use the tools without understanding the machinery — which means they can't diagnose failures, select appropriate models for different tasks, or reason about why an output went wrong.

Why Familiarity Breeds Overconfidence

The core paradox: students use AI more casually and more frequently than most professionals, yet score lower on every measured dimension. This isn't contradictory. It's predictable.

Using ChatGPT to write an essay, summarise a reading, or generate code snippets is consumption. It's the AI equivalent of using Google — you type something in, you get something back, you move on. AI fluency requires something different: understanding why a model produces a given output, knowing when to trust it, being able to steer it toward better results, and recognising where it fails.

The Exposure-Competence Illusion

Psychologists call this the exposure-competence illusion — the tendency to confuse familiarity with a tool for mastery of it. Everyone who drives a car daily feels like a good driver. Most aren't. The same dynamic plays out with AI tools. Students have logged thousands of hours with ChatGPT. Those hours built comfort, not competence.

Anthropics's research supports this framing. Their 2025 AI Fluency Index found that 93% of the markers they identified for AI fluency involve skills beyond basic tool operation — skills like output evaluation, failure diagnosis, and responsible deployment. AISA's rubric covers these same markers, which is why daily ChatGPT use doesn't translate to a high score.

What Students Practice vs. What Gets Measured

Consider what a typical student AI interaction looks like:

1. Open ChatGPT
2. Type a question or paste an assignment prompt
3. Read the output
4. Use it (possibly with light editing)

Now consider what AISA's rubric measures across its five dimensions:

• Prompting: Can you explain your prompting strategy? Do you use constraints, role-setting, or structured formats deliberately?
• Critical Thinking: How do you verify outputs? What's your process for catching hallucinations?
• Technical Understanding: Why might a model produce a different output with the same prompt? What are token limits and why do they matter?
• Workflow: How do you integrate AI into a multi-step process? Where does human judgment enter?
• Safety: What data should you never paste into a public model? How do you think about bias in AI outputs?

The gap between typical student usage and what the rubric measures explains the 32.1-point prediction gap almost entirely. Students assess their skill based on how often they use AI. The rubric assesses skill based on how well they use it.

The Safety Crisis: 29.6 Is a Red Flag

Students score 29.6 on Safety & Responsibility. This is the lowest safety score of any role in the AISA dataset — lower than Engineering (46.9), Product (47.9), Founders (49.6), Data (43.9), and even Design (34.9). It's 11.2 points below the overall average of 40.8.

A score of 29.6 places the average student at the boundary between Novice and Developing on safety. In practical terms, this means most students we assess cannot articulate basic AI data privacy principles, don't have a framework for thinking about bias in AI outputs, and haven't considered the implications of pasting personal or proprietary data into public models.

Why Student Safety Scores Are So Low

Three factors converge:

No stakes exposure. Working professionals encounter data privacy, compliance, and liability concerns in their daily work. A product manager knows not to paste customer data into ChatGPT because their company has a policy — or because they've seen what happens when someone does. Students rarely face these constraints. Their AI use happens in low-stakes academic contexts where the worst outcome is a bad grade, not a data breach.

No formal training. The World Economic Forum's 2024 Future of Jobs Report found that only 39% of surveyed organisations had implemented AI-related training programs. If employers are behind on AI safety training, universities are further behind. Most computer science curricula cover AI ethics as a single lecture or elective, not as an integrated practice. Students arrive at the workforce with no safety muscle memory.

Normalised risk. Students have grown up sharing personal information online. The boundary between "data I should protect" and "data that's fine to share" is blurrier for a generation raised on social media. This normalisation extends to AI tools: pasting personal conversations, academic records, or other sensitive content into ChatGPT feels no different from posting on Instagram.

We've covered the broader safety gap across all roles in our analysis of the 36% who report no safety practices. The student data is the sharpest edge of that broader problem.

What 29.6 Looks Like in Practice

At this score level, a typical student in our assessment:

• Cannot name specific types of data that shouldn't be shared with AI tools
• Has no process for evaluating whether an AI output might be biased
• Hasn't considered intellectual property implications of AI-generated content
• Doesn't distinguish between consumer and enterprise AI tool privacy policies

This isn't a knowledge gap that fixes itself with more ChatGPT use. It requires deliberate instruction.

Implications for Universities and Bootcamps

If students are arriving at the workforce with a composite AI score of 36.0 — Developing tier — and they think they're Proficient, universities have two problems, not one.

Problem 1: The Skills Deficit

A composite of 36.0 means the average student is a Dabbler — someone who uses AI tools casually but lacks the structured knowledge to use them effectively. For context, the Dabbler persona (31.5% of all AISA test-takers, average score 26.0) is characterised by surface-level engagement without depth. Students score slightly above the Dabbler average, but they're in the same neighbourhood.

Universities that want to close this gap need to move beyond "here's how to use ChatGPT" workshops. The data suggests students need structured development in:

• Technical understanding (30.6): How models work, what tokens are, why different models produce different outputs
• Safety practices (29.6): Data privacy, bias awareness, responsible deployment
• Critical thinking (34.3): Output verification, hallucination detection, source triangulation
• Workflow integration (34.9): Multi-step processes, human-AI handoffs, knowing when not to use AI

Problem 2: The Calibration Deficit

The skills deficit is fixable with curriculum. The calibration deficit — students believing they're already skilled — is harder. You can't teach someone who thinks they don't need to learn.

This is where assessment plays a critical role. When students see their actual score against a published rubric with evidence-linked feedback, the prediction gap becomes a learning moment rather than a hidden liability. The AISA assessment provides a detailed breakdown across all five dimensions, giving students (and their institutions) a concrete map of where to focus.

What Bootcamps Should Do Differently

Bootcamps face the same challenge in compressed form. A 12-week program that teaches students to build with AI APIs but never covers output verification, safety practices, or cognitive surrender — the tendency to defer entirely to AI judgment — is producing graduates who can ship code but can't evaluate whether that code is trustworthy.

The Stanford HAI 2024 AI Index Report noted that AI education is expanding rapidly across universities, but most programs focus on technical implementation rather than responsible use. Our data suggests this implementation-first approach produces students who can operate AI tools but can't evaluate their outputs — which is exactly the pattern we see in the dimension scores.

What Students Can Do Right Now

The prediction gap isn't permanent. It's a snapshot of where students are today, and it's addressable with deliberate practice.

Step 1: Get a Baseline

The first step is replacing assumed competence with measured competence. Take the AISA assessment — it's a conversation, not a multiple-choice test, and the free report gives you dimension-level scores with specific evidence from your own responses. Knowing you score 30.6 on technical understanding is more useful than vaguely feeling like you're "pretty good with AI."

Step 2: Focus on the Weakest Dimensions

For most students, that means safety (29.6) and technical understanding (30.6). Start with concrete practices:

• Before pasting anything into an AI tool, ask: "Would I be comfortable if this data were public?"
• After receiving an AI output, spend 60 seconds checking one claim against a primary source
• Learn what a token is, what a context window is, and why your prompt length matters

Step 3: Move from Consumer to Practitioner

The gap between a Dabbler and a Tactician (average score 59.8) isn't about using AI more. It's about using it with intention. That means planning prompts before typing them, evaluating outputs before using them, and building workflows that include human checkpoints.

Role-level data across all professions shows what this progression looks like in practice — see our breakdown by job role for comparison points.

---

Related reading: 5 AI Skills Professionals Overestimate — the prediction gap isn't just a student problem.

Related reading: AI Safety Gap: 36% Report No Safety Practice — the safety deficit across all roles, with students at the extreme.

Related reading: Is AI Making Us Dumber? What the Research Says — the cognitive offloading research behind the familiarity-competence gap.

Frequently Asked Questions

Are students good at AI?

Students score an average composite of 36.0 on the AISA assessment, placing them in the Developing tier (28-59). This is the lowest composite of any role in the dataset — below Engineering (54.2), Product (55.7), Founders (55.3), and Data (49.2). Students use AI tools frequently but score below average on every measured dimension, including prompting, critical thinking, technical understanding, workflow integration, and safety.

Why do students overestimate their AI skills?

Students show a 32.1-point prediction gap — predicting 66.2 but scoring 34.1. This overconfidence stems from the exposure-competence illusion: daily use of ChatGPT and similar tools creates a feeling of mastery that doesn't match measured performance. Students assess their skill based on frequency of use rather than quality of use, and most lack the real-world stakes (data privacy concerns, compliance requirements) that force professionals to develop deeper AI fluency.

What AI skills do students lack?

The weakest dimensions for students are Safety & Responsibility (29.6) and Technical Understanding (30.6), both near the Novice-Developing boundary. Students struggle to articulate data privacy principles, identify bias in AI outputs, explain how models work, or describe why different models produce different results. Workflow & Application (34.9) and Critical Thinking (34.3) are also significantly below the overall averages of 47.1 and 42.3 respectively.

How can universities improve student AI skills?

Universities need to address both the skills deficit and the calibration deficit. Curriculum should cover output verification, AI safety practices, and technical fundamentals — not just tool operation. Equally important is helping students recognise their current skill level through evidence-based assessment rather than self-evaluation. Integrating structured AI fluency measurement into coursework gives students a concrete baseline and a roadmap for improvement across specific dimensions.