Test Your AI Knowledge: What You Actually Know vs What You Think You Know

You use ChatGPT every day. You've tried Claude, maybe Gemini. You know what a prompt is, what hallucinations are, and that you should probably verify AI output. You're pretty good at this.

Or are you?

Here's what we've observed across thousands of AI skills assessments: most people overestimate their AI knowledge. Not by a little — by a lot. Someone who rates themselves "advanced" often scores in the middle of the pack when their skills are actually tested. And the people who describe themselves as "beginners" sometimes demonstrate surprisingly sophisticated workflows they've built intuitively without realising how unusual their approach is.

The Dunning-Kruger Problem in AI Skills

The Dunning-Kruger effect — where people with limited knowledge overestimate their competence — is particularly acute with AI. There are two reasons:

First, AI tools are designed to be easy to use. You type something, you get something back. The friction is so low that it creates an illusion of mastery. But there's an enormous gap between using AI and using AI well. The difference between a novice and an expert isn't visible in the interface — it's in how they think about the interaction.

Second, there's no natural feedback loop. If you write bad code, it breaks. If you write a bad prompt, you still get a polished, confident-sounding response. The AI doesn't tell you your approach was inefficient, that you could have gotten a better answer with a different structure, or that you're missing a technique that would save you hours.

What Real AI Knowledge Looks Like

When we say "AI knowledge," we're talking about something broader than knowing which model is newest. Real AI knowledge spans five areas:

Prompting goes beyond typing a question. Do you structure your prompts with context, constraints, and output format? Do you iterate systematically when the first result isn't right? Do you manage context across long conversations?

Critical thinking means not trusting AI output at face value. But it goes further than "always double-check." It means understanding which outputs are most likely to be wrong, why certain tasks produce unreliable results, and having a specific verification method — not just a vague sense of caution.

Technical understanding doesn't mean you need to train models. It means understanding why a 4,000-word document might lose coherence when you paste it into a chat window, why the same prompt gives different results on different days, or why some tasks are fundamentally better suited to AI than others.

Workflow integration separates occasional users from genuinely fluent ones. Are you using AI for one or two tasks, or is it embedded across your work? Have you built repeatable processes, or do you start from scratch each time?

Safety awareness means thinking about what data you're exposing to AI systems, understanding the implications of AI-generated content in professional contexts, and recognising when AI use is inappropriate regardless of how convenient it would be.

Why Self-Assessment Doesn't Work

We've all filled out skills matrices. "Rate yourself 1–5 on the following..." The problem is that self-assessment requires you to know what you don't know — which is, by definition, the thing you're worst at judging.

Consider this: if you've never used system prompts, few-shot examples, or chain-of-thought prompting, you might not know these techniques exist. You'd rate your prompting ability based on your experience of typing questions and getting answers — which feels perfectly adequate until you see what structured prompting actually produces.

This is why external assessment matters. Not a quiz with right and wrong answers, but a test that explores how you actually think about and use AI — and compares it against a calibrated framework.

How to Get an Honest Measure

If you want to genuinely test your AI knowledge, look for something that:

1. Goes beyond recall. Picking the right answer from four options tests memory. A real test explores depth — follow-up questions, "how would you handle this," "walk me through your process."

2. Covers the full range. Many tests focus on prompting alone. AI knowledge includes evaluation, tool awareness, workflow design, and responsible use.

3. Gives you specifics. A single score is almost useless. You need to know which areas are strong and which need work — because someone who's great at prompting but weak on critical evaluation needs completely different development than someone with the reverse profile.

AISA's assessment takes this approach. It's a conversation, not a quiz. You talk to an AI interviewer who adapts based on what you say, probing deeper where you show strength and moving on where you don't. At the end, you get a detailed breakdown across 11 criteria, a personalised AI persona classification, and specific recommendations for what to learn next.

It takes 20–40 minutes. It's free for your first assessment. And it will almost certainly surprise you — in both directions.

The professionals who know the most about AI are usually the ones most aware of what they still don't know. That awareness is itself a skill — and it starts with an honest test.