Critical Thinking With AI: How Good Are You?
Critical thinking with AI is the second-lowest scoring dimension at 42.4/100. See where you stand and how to improve your AI critical thinking skills.
Most people who use AI regularly believe they're thinking critically about the output. The data says otherwise. Across 2,553 completed assessments on AISA — a conversational AI fluency assessment — critical thinking with AI scores an average of just 42.4 out of 100, making it the second-lowest dimension after technical understanding (39.2). The gap between what people think they can do and what they actually demonstrate is stark: the average person overestimates their overall AI ability by 17.7 points.
What makes this particularly interesting is that critical thinking isn't a new concept. Most professionals have spent years evaluating information, questioning sources, and making evidence-based decisions. But critical thinking in the context of AI output requires a different set of instincts — ones that most people haven't developed yet.
Why Critical Thinking With AI Differs From General Critical Thinking
General critical thinking evaluates arguments, evidence, and logic produced by humans. AI critical thinking requires evaluating output from systems that present information with uniform confidence, regardless of whether it's correct. That distinction changes everything about how verification needs to work.
When a colleague sends you a report, you unconsciously calibrate trust based on their track record, their hedging language, their citations. You notice when they seem uncertain. Large language models don't give you those signals. A model like Claude Fable 5.2 or GPT-6 Astra will state a fabricated statistic with the same syntactic confidence as a verified fact. The prose is always polished. The structure is always coherent. The confidence calibration cues you've relied on for decades don't apply.
The Fluency-Accuracy Trap
This is what researchers call the fluency-accuracy conflation — the tendency to assume that well-written output is accurate output. Stanford's 2024 study on AI-assisted research found that participants who used AI were 35% more likely to include inaccurate citations in their work compared to those who didn't, precisely because the AI-generated text read as authoritative. The writing quality masked the factual gaps.
When you read a human-written paragraph full of hedging and awkward phrasing, your critical thinking activates. When you read a clean, confident AI-generated paragraph, it often doesn't. That's the core problem.
Domain Knowledge Doesn't Automatically Transfer
You might be an excellent critical thinker in your field. But AI critical thinking requires a specific kind of meta-cognition: thinking about how the model produced its answer, not just whether the answer sounds right. It means understanding that models predict likely next tokens rather than retrieving verified facts. It means knowing that a model's training data has a cutoff date, that it can't access real-time information unless connected to tools, and that it will confidently fill gaps with plausible-sounding fabrications.
This is why even experienced professionals score poorly. Their domain expertise gives them a false sense of security — they assume they'd catch errors in their own field, but the errors AI makes are often subtle enough to slip past domain knowledge alone.
The Verification Burden Has Shifted
In traditional knowledge work, the person producing the information bears the burden of accuracy. When you delegate to AI, that burden shifts entirely to you. Most people haven't internalised this shift. They treat AI output the way they'd treat output from a trusted junior colleague — worth a quick scan, but probably fine. The data suggests this default trust is misplaced.
5 Signs You're Not Thinking Critically About AI Output
If you recognise three or more of these patterns in your own AI use, your AI critical thinking skills likely need deliberate work. These aren't edge cases — they're the most common failure modes we observe across thousands of assessments.
1. You Accept the First Answer
The most common pattern: you prompt the model, get a response, and use it. No follow-up questions. No asking the model to reconsider. No testing the same question with a different framing. This is the AI equivalent of reading one Google result and closing the browser.
Strong AI critical thinkers treat the first response as a draft hypothesis, not a final answer. They probe, challenge, and iterate — not because the model is always wrong, but because the first response optimises for plausibility, not necessarily for accuracy or completeness.
2. You Don't Verify Factual Claims
AI models generate text that looks like it contains facts — names, dates, statistics, citations. But these are pattern completions, not database lookups. Hallucination detection is a skill, and most people don't practise it.
If you've ever used an AI-generated statistic in a presentation without checking the source, or quoted an AI-suggested citation without confirming it exists, you've fallen into this trap. The Anthropic AI Fluency Index identifies fact verification as one of the markers that separates competent AI users from developing ones.
3. You Ignore Confidence Calibration
Models don't naturally express uncertainty well. When you ask a model a question it can't reliably answer, it typically answers anyway — with the same tone and structure as when it's on solid ground. Strong critical thinkers explicitly ask models to flag uncertainty, rate their own confidence, or identify which parts of their response are most likely to be wrong.
If you've never prompted a model with something like "What parts of this answer are you least confident about?" or "What assumptions are you making here?", you're missing a key verification layer.
4. You Skip Source Checks
Source triangulation — verifying AI claims against independent sources — is the single most effective habit for catching errors. Yet it's also the most skipped step. The reasoning is understandable: if you're using AI to save time, adding a verification step feels like it defeats the purpose.
But the time calculation changes when you factor in the cost of acting on wrong information. A fabricated legal precedent, an invented API endpoint, a hallucinated research finding — these don't just waste time, they create downstream problems that cost far more to fix than the verification would have taken.
5. You Conflate Fluency With Accuracy
This is the meta-pattern that enables all the others. If the output reads well, you trust it. If it's formatted nicely with bullet points and headers, you trust it more. If it includes what look like citations, you trust it even more.
Fluency is a property of the model's language generation. Accuracy is a property of the underlying information. These are independent variables. A model can be perfectly fluent and completely wrong. Until you internalise this distinction, your critical thinking with AI will have a fundamental blind spot.
Where People Actually Score on AI Critical Thinking
The average critical thinking dimension score across 2,553 AISA assessments is 42.4 out of 100. To put that in context, the overall composite average is 46.2 — meaning critical thinking drags down most people's total score. It's the second-lowest of all five dimensions, ahead of only technical understanding at 39.2.
Role-by-Role Breakdown
The variation across roles is significant:
| Role | Critical Thinking Score | Overall Composite | n |
|---|---|---|---|
| Founders | 50.3 | 55.5 | 230 |
| Product | 49.5 | 55.0 | 149 |
| Engineering | 47.9 | 54.1 | 521 |
| Data | 47.0 | 50.0 | 56 |
| Design | 41.9 | 47.6 | 59 |
| Students | 34.4 | 36.2 | 231 |
Founders score highest at 50.3, which still sits in the Developing tier (28–59). Students score lowest at 34.4 — a 15.9-point gap. This pattern aligns with what you'd expect: founders tend to have more experience evaluating uncertain information and making decisions with incomplete data. Students, who are often newer to professional contexts, haven't built those habits yet.
But even the highest-scoring group barely cracks the midpoint. No role averages above 51. This isn't a problem confined to junior professionals — it's systemic.
The Overconfidence Factor
The prediction gap data makes this worse. Students predict their overall score will be 66.6 but actually score 35.1 — an overestimation of 31.5 points. Even founders, who are the most calibrated group, overestimate by 6.7 points (predicting 61.3, scoring 54.6). For a deeper look at this pattern, see AI Overconfidence by Role: Who's Worst?.
This means most people not only score poorly on critical thinking — they don't know they score poorly. That combination is what makes this dimension particularly dangerous. You can't fix a weakness you don't see.
Why Critical Thinking Lags Other Dimensions
Workflow & Application, the highest-scoring dimension at 47.2, is more visible. People know whether they use AI for tasks or not. Prompting (44.6) is practised every time someone types into ChatGPT. But critical thinking is invisible — it's the absence of a step, the check you didn't run, the question you didn't ask. You can use AI daily and never develop this skill because nothing in the interface prompts you to verify.
As explored in Is AI Making Us Dumber?, the convenience of AI output actively works against critical evaluation. The easier it is to get an answer, the less likely you are to question it.

Curious about your AI Fluency?
AISA helps you measure, prove and improve your AI skills — free report in a 20–40 minute chat.
7 Questions to Gauge Your AI Critical Thinking
Before taking a full AI fluency assessment, use these questions as a self-diagnostic. Be honest — the value is in the self-awareness, not the score.
Verification Habits
1. When AI gives you a specific statistic or citation, what do you do next?
- (a) Use it as-is — it sounds right
- (b) Occasionally spot-check if something feels off
- (c) Verify every factual claim against an independent source before using it
2. Have you ever asked an AI model to rate its own confidence in its answer?
- (a) No, I didn't know that was useful
- (b) Once or twice
- (c) Regularly — it's part of my standard workflow
Output Evaluation
3. When an AI response contradicts your intuition, what's your default reaction?
- (a) Assume the AI is probably right — it has more data
- (b) Feel uncertain but usually go with the AI
- (c) Investigate the discrepancy before deciding
4. Can you name three specific ways AI models produce inaccurate output?
- (a) Not really — I know they "hallucinate" but couldn't explain the mechanisms
- (b) I could name one or two
- (c) Yes — training data gaps, pattern completion without grounding, outdated knowledge cutoffs, and others
Process Design
5. Do you use different verification approaches depending on the stakes of the task?
- (a) I treat all AI output the same way
- (b) I'm more careful with important tasks but don't have a system
- (c) I have explicit stakes-based review criteria — low-stakes tasks get lighter checks, high-stakes tasks get full verification
6. When you need reliable information, do you cross-reference AI output across multiple models or sources?
- (a) No — I use one model and trust it
- (b) Sometimes, if I'm unsure
- (c) Yes — cross-model verification is a standard step for anything important
7. After using AI output in your work, have you ever gone back to check whether it was accurate?
- (a) No
- (b) Occasionally, when something went wrong
- (c) Yes — I do post-hoc accuracy audits to calibrate my trust
Interpreting Your Answers
If you answered mostly (a): You're likely in the Novice to Developing range for critical thinking. This is common — 69.3% of AISA test-takers fall into the Emerging or Developing tiers overall. The good news is that critical thinking habits are buildable.
If you answered mostly (b): You have some awareness but lack systematic habits. This is the most common profile — you know verification matters but don't do it consistently.
If you answered mostly (c): You likely have strong critical thinking habits. A formal assessment would tell you exactly where you stand relative to the 2,553 people who've taken AISA.
How to Improve Your AI Critical Thinking Skills
Improving critical thinking with AI isn't about becoming sceptical of everything — it's about building deliberate verification habits that match the stakes of what you're doing. Here's what the evidence and our assessment data suggest works.
Build a Stakes-Based Verification Framework
Not every AI interaction needs the same level of scrutiny. A brainstorming session for blog post ideas needs less verification than a financial analysis or a legal summary. Define three tiers for your own work:
- Low stakes (brainstorming, drafting, ideation): Scan for obvious errors, but don't verify every claim
- Medium stakes (internal reports, code suggestions, research summaries): Verify key facts, cross-reference critical claims, ask the model to flag uncertainties
- High stakes (client deliverables, published content, legal/financial decisions): Full verification against independent sources, cross-model checks, expert review
The point isn't to slow everything down — it's to allocate your verification effort where it matters most.
Develop Specific Verification Prompts
Add these to your regular AI workflow:
- "What assumptions are you making in this response?"
- "Which parts of this answer are you least confident about?"
- "Are there alternative interpretations I should consider?"
- "What sources would I check to verify this?"
These prompts don't guarantee accuracy, but they surface potential weak points that you can then investigate independently. The AERA/APA/NCME Standards for educational measurement emphasise that assessment quality depends on evidence — the same principle applies to evaluating AI output.
Practice Claim Decomposition
When AI gives you a complex answer, break it into individual claims and evaluate each one separately. A paragraph might contain five factual assertions — three correct, one outdated, and one fabricated. If you evaluate the paragraph as a whole, the three correct claims create a halo effect that masks the two problematic ones.
This is a learnable skill. The AISA critical thinking dimension specifically measures whether candidates can identify which parts of an AI response need verification and which can be trusted.
Cross-Reference Across Models
With models like Claude Fable 5.2, Gemini 4 Carbon, and GPT-6 Astra all available, cross-model verification has never been more practical. If two models independently produce the same factual claim, it's more likely (though not guaranteed) to be accurate. If they disagree, that's a strong signal to check an independent source.
This doesn't mean running every query through three models. But for medium- and high-stakes work, a quick cross-check takes seconds and catches a meaningful percentage of errors.
Track Your Own Accuracy
The Deloitte 2024 State of Generative AI report found that 68% of organisations using generative AI had no formal process for validating AI output quality. If organisations struggle with this, individuals are even less likely to have systems in place.
Start simple: for one week, verify every factual claim AI makes in your work. Track how often the claims are accurate, partially accurate, or wrong. This exercise recalibrates your intuition about how much to trust AI output — and the results usually surprise people.
Understand Why Models Fail
Knowing that models make errors is table stakes. Knowing why they make errors — and being able to predict when they're most likely to fail — is what separates competent from proficient AI users. Models are more likely to produce inaccurate output when:
- The topic is niche or recent (beyond training data)
- The question requires precise numerical reasoning
- The task involves synthesising across many sources
- The prompt is ambiguous or underspecified
Building this mental model of failure modes lets you calibrate your verification effort more intelligently. For a structured approach to building these skills, the Critical Thinking with AI guide covers the core competencies.
What Your Score Actually Means for Your Work
AI fluency isn't just about knowing how to prompt well or understanding transformer architecture. It's about the full loop: generating output, evaluating it, verifying it, and integrating it responsibly. Critical thinking is the evaluation and verification step — and it's the one most people skip.
The practical consequence is real. If you're using AI output in your work without adequate verification, you're introducing a source of error that scales with your AI usage. The more you use AI, the more critical thinking matters — and the more damage its absence can cause.
Across AISA's data, people motivated by leadership score an average of 51.4 on their overall composite, while those in career transition average 36.8. But regardless of motivation or role, critical thinking consistently underperforms other dimensions. It's the universal weak spot.
If you want to know exactly where you stand — not a self-assessment guess, but an evidence-linked score across all five dimensions — take the AISA assessment. It's a 20-minute conversation, and you get a detailed breakdown including your critical thinking score, your persona type, and specific areas for improvement. Every score is backed by evidence from your own words, evaluated against a published rubric.
The question isn't whether you think you're thinking critically about AI. It's whether you can demonstrate it.
Related reading: How Good Am I at AI Safety? [2026 Data] — the lowest-scoring dimension, and what it reveals about responsible AI use.
Related reading: Over-Reliance on AI at Work [2026 Data] — what happens when verification habits break down at scale.
Related reading: AI Fluency Levels: 5 Tiers Explained [2026] — where each tier sits and what it takes to move up.
Frequently Asked Questions
What is critical thinking in AI?
Critical thinking in AI is the ability to evaluate, verify, and question AI-generated output rather than accepting it at face value. It includes skills like detecting hallucinations, verifying factual claims against independent sources, understanding when models are likely to produce errors, and calibrating your trust based on the stakes of the task. AISA measures it across criteria like output evaluation, fact verification, and confidence calibration.
Why do people score low on AI critical thinking?
Critical thinking with AI scores low (42.4 average across 2,553 assessments) because it's an invisible skill — nothing in the AI interface prompts you to verify output, and the fluency of AI-generated text actively discourages scrutiny. Most people have strong general critical thinking skills but haven't adapted them to the specific challenges of evaluating AI output, where confidence signals are absent and errors are subtle.
How can I improve my AI critical thinking?
Start with three habits: build a stakes-based verification framework so you match scrutiny to importance, develop specific prompts that surface model uncertainty ("What assumptions are you making?"), and practise claim decomposition — breaking complex AI responses into individual assertions and checking each one. Cross-referencing output across multiple models also catches errors efficiently. The Critical Thinking with AI guide provides a structured approach to building these competencies.
How does AISA measure AI critical thinking?
AISA measures critical thinking through a conversational assessment where candidates discuss how they evaluate and verify AI output. A separate AI evaluator scores responses against a published rubric, looking for evidence of specific behaviours: questioning AI claims, describing verification processes, identifying failure modes, and demonstrating awareness of when AI output needs additional scrutiny. Every score requires evidence in the candidate's own words — there are no multiple-choice shortcuts.

Curious about your AI Fluency?
AISA helps you measure, prove and improve your AI skills — free report in a 20–40 minute chat.

