The AISA Rubric: 5 Dimensions, 11 Criteria

Here's something we didn't expect when we started building AISA. The single strongest predictor of a high score isn't tool knowledge, prompt complexity, or technical vocabulary. It's whether a person can explain why they do what they do.

Two candidates can take the exact same action — structure a prompt the same way, pick the same tool, catch the same error. But the one who can tell you the reasoning behind their choice consistently outscores the one who just does it. That gap shows up across every dimension, every role, every score band. The rubric exists to capture it.

Why does this matter? Two reasons. First, if you can't see the reasoning, you can't validate the decision — AI becomes a black box, and the person using it is operating on instinct rather than judgment. Second, when you can see the reasoning, you can see exactly where the learning gaps are. The "why" is where the growth targets live.

This article walks through the full AISA scoring framework: 5 dimensions, 11 criteria, a 1–10 scale per criterion, and a weighted composite that produces a 0–100 proficiency score. Whether you're an engineering manager evaluating candidates, an L&D leader benchmarking your team, or a builder curious about where you stand — this is the reference document.

Why These Five Dimensions

Most AI skill frameworks fall into one of two traps. They either reduce proficiency to a single axis — "Can this person write good prompts?" — or they enumerate dozens of micro-skills that are impossible to assess reliably.

We started with extensive research into how AI proficiency was being measured across industry and academia, then worked with domain experts to narrow down what actually predicts effective AI use. The goal was the minimum viable set of dimensions that, together, tell you whether someone can do useful, responsible work with AI.

We tested frameworks with as few as 6 criteria and as many as 20. Six wasn't enough resolution — you couldn't tell a strong prompt engineer with no critical thinking from an all-rounder. Twenty was too granular — inter-rater reliability collapsed because the distinctions between adjacent criteria became subjective. Eleven criteria across five dimensions turned out to be the sweet spot: enough resolution to surface specific strengths and gaps, compact enough that every criterion stays meaningfully distinct.

The five dimensions themselves came from watching how effective practitioners actually work — not how they describe their work in a resume, but what they do in live problem-solving. Strong performers share a common loop: they communicate clearly with AI systems, evaluate the outputs critically, understand the underlying technology well enough to diagnose failures, integrate AI into structured workflows, and operate within safety and ethical boundaries. Remove any one of these and the person becomes unreliable in predictable ways.

The weighting reflects where real-world impact concentrates. Workflow & Application carries the highest weight (25%) because the gap between knowing AI concepts and actually shipping AI-assisted work is where most teams stall. Safety & Responsibility carries the lowest weight at 10% — not because safety is unimportant, but because the Safety criterion is still maturing. We have plans to expand it with sub-criteria as we collect more assessment data, and the weight may increase accordingly. For now, it functions as a threshold: candidates who score below 4 on safety exhibit patterns that disqualify them from unsupervised AI work regardless of their other scores.

Dimension 1: Prompting & Communication (23%)

This dimension measures how effectively someone communicates with AI systems. It's not about memorizing prompt templates — it's about whether the person treats the AI as a reasoning system that responds to structure, specificity, and context, or just types into it like a search bar.

P1: Prompt Design

P1 looks at the initial construction of prompts. At the Novice level (1–2), candidates type vague, conversational requests and accept whatever comes back. At Competent (5–6), they reliably include role framing, constraints, output format specifications, and examples. At Expert (9–10), they demonstrate principle-level understanding of why certain structures work — they can predict how changes to a prompt will affect output quality, and they adjust their approach based on the model they're using.

The key differentiator at the upper end isn't sophistication for its own sake. It's intentionality. An expert-level prompt might actually be shorter than a competent-level one — but every element is there for a reason the person can explain.

P2: Iterative Dialogue

P2 measures the ability to refine AI outputs through multi-turn conversation. This is where many people who score well on P1 plateau. They can write a strong initial prompt but don't know how to diagnose why an output missed the mark, or how to steer the conversation without starting over.

Proficient candidates (7–8) treat the conversation as a feedback loop. They reference specific parts of previous outputs, escalate or narrow the scope deliberately, and know when to abandon a thread and restructure. Expert-level performance here looks like someone pair-programming with a junior developer — they know how to ask the right follow-up question to get what they need in one or two turns.

P3: Context & Memory Management

P3 addresses how people manage the practical constraints of AI conversations: context windows, conversation history, and information persistence. A Novice doesn't know these constraints exist. A Competent candidate breaks large tasks into smaller conversations. An Expert designs their workflow around context management — they know when to summarize previous context, when to use system prompts, and how to structure information so the model retains what matters.

This criterion has become increasingly important as AI tools have moved from single-shot interactions to persistent, session-based workflows. Anyone building with tools like developer-focused AI assistants needs this skill daily.

Dimension 2: Critical Thinking (22%)

Critical Thinking is the dimension that most sharply separates the top quartile from everyone else. It measures whether someone evaluates AI outputs with the same rigor they'd apply to any other information source — and whether they understand where AI systems systematically fail.

T1: Output Evaluation

T1 scores the ability to assess whether an AI output is correct, complete, and fit for purpose. At the low end, people accept outputs at face value or reject them on gut feeling. At the high end, they apply structured evaluation: checking outputs against known facts, probing edge cases, testing for internal consistency, and being able to articulate what "good enough" looks like for a given use case.

The practical test for T1 is whether someone catches errors that the AI introduced confidently. Our assessment conversations are designed to surface moments where the AI facilitator provides a plausible but flawed response. How the candidate handles that moment is one of the highest-signal data points in the entire assessment.

T2: Limitation Awareness

T2 measures whether someone understands the systematic limitations of current AI systems — hallucination, training data cutoffs, reasoning failures in specific domains, sensitivity to prompt framing, and the difference between pattern matching and genuine understanding.

A Developing candidate (3–4) knows that AI "can be wrong sometimes." A Proficient candidate (7–8) can predict when AI is likely to be wrong for a specific task and adjusts their workflow accordingly. They don't waste time asking an LLM to do reliable arithmetic, and they don't blindly trust AI-generated code without testing it.

This criterion matters enormously for product managers and designers making decisions about where to deploy AI in products their users will rely on.

Dimension 3: Technical Understanding (20%)

Technical Understanding doesn't require a machine learning degree. It requires enough conceptual grounding to make informed decisions about AI tools and to diagnose problems when they come up.

U1: AI Fundamentals

U1 measures whether someone understands the core concepts behind the AI systems they use — a working mental model of how large language models generate text, what training data means for output quality, the difference between fine-tuning and prompting, and why temperature and other parameters affect outputs.

We're not looking for people to recite the transformer architecture. We're looking for whether their mental model is accurate enough to be useful. A data scientist should score higher here than a product manager, but every role benefits from a mental model that helps them predict system behaviour.

U2: Tool Landscape

U2 evaluates awareness of the current AI tool ecosystem and the ability to select appropriate tools for specific tasks. This isn't a trivia test — we don't care whether someone can name every model on the market. We care whether they understand the tradeoffs: when to use a general-purpose LLM versus a specialized model, when an API integration beats a chat interface, when a RAG pipeline is the right architecture versus fine-tuning.

The scoring rewards people who have hands-on experience with multiple tools and can explain why they chose one over another. Someone who has only used one AI tool but understands its strengths and limitations deeply will outscore someone who name-drops ten tools but can't articulate a selection rationale.

Dimension 4: Workflow & Application (25%)

This is the highest-weighted dimension because it measures what ultimately matters: can this person get real work done with AI? Knowing how AI works and actually shipping AI-assisted outcomes are different skills, and the gap between them is larger than most people assume.

W1: Workflow Integration

W1 measures how someone incorporates AI tools into their existing work processes. At the low end, AI use is ad hoc — open ChatGPT when stuck, close it when you get an answer. At the high end, AI is woven into a structured workflow with defined handoff points, quality gates, and human review stages.

Proficient candidates can describe their AI workflow as a repeatable process. Expert candidates have optimized that process — they know which steps benefit from AI involvement, which require human judgment, and they've built personal systems (templates, checklists, automated pipelines) that encode these decisions.

W2: Task Decomposition

W2 evaluates the ability to break complex problems into AI-appropriate subtasks. This is a core engineering skill applied to a new context. The question isn't "Can you use AI to do this task?" — it's "Can you figure out which parts AI can handle, which parts it can't, and how to reassemble the pieces?"

Strong task decomposition is especially critical for developers working on AI-assisted codebases. The difference between a developer who asks AI to "build me a login system" and one who decomposes the problem into authentication flow design, token management, UI components, and test cases — and knows which subtasks to delegate — is the difference between unreliable output and production-ready code.

W3: Domain Application

W3 scores the ability to apply AI tools effectively within a specific professional domain. A developer using AI for code review exercises different domain knowledge than a designer using AI for user research synthesis. W3 measures whether someone adapts their AI usage to the norms, constraints, and quality standards of their field.

This is where generalist AI knowledge meets specialist expertise. The highest scores go to people who've developed domain-specific AI workflows — not just using AI generically, but leveraging it in ways that reflect deep understanding of their field's requirements.

How Scoring Works

Each of the 11 criteria is scored on a 1–10 scale. The scores aren't arbitrary — they map to five proficiency bands that describe observable behaviours:

• 1–2 (Novice): Unaware that this is a distinct skill. May use AI but shows no intentionality.
• 3–4 (Developing): Aware of the skill and attempts to apply it, but inconsistently.
• 5–6 (Competent): Demonstrates functional, repeatable techniques. Can do the thing reliably.
• 7–8 (Proficient): Intentional and can explain why their approach works. Adapts to novel situations.
• 9–10 (Expert): Operates at a principle level. Has internalized the skill so deeply it has reshaped how they think about their work.

The composite score is calculated by weighting each dimension according to its assigned percentage (Prompting 23%, Critical Thinking 22%, Technical Understanding 20%, Workflow 25%, Safety 10%) and normalizing to a 0–100 scale.

Critically, scores are based on evidence observed during the conversation, not self-reported claims. AISA's dual-track architecture means the scoring AI independently evaluates every response — it doesn't take anyone's word for anything. If someone claims to be an expert prompt engineer but writes vague, unstructured prompts during the assessment, the score reflects the observed behaviour.

What 1,800+ Assessments Have Revealed

After scoring more than 1,800 professionals, a few patterns have emerged that challenge common assumptions about AI skills.

Subject matter expertise beats youth. The popular narrative is that younger, digitally native professionals will naturally be better with AI — the way Gen Y picked up smartphones faster than Gen X. The data says otherwise. Experienced professionals with deep subject matter expertise consistently outscore younger candidates. Domain knowledge turns out to be the foundation that AI skills are built on, not a separate track.

Technical doesn't mean better at AI. Product managers outscore developers on average. That surprised us too. Developers tend to score high on Technical Understanding but lower on Critical Thinking and Workflow Integration — they know how the technology works but don't always evaluate outputs as rigorously or integrate AI as systematically into their broader work. PMs bring structured decision-making habits that transfer well.

The confidence gap is real — and revealing. Across 1,800+ assessments, professionals rate themselves an average of 18.5 points higher than they actually score. This happens constantly: someone comes in confident they're an 80 and scores a 55. It's not that they're bad at AI — it's that they've never had an objective measure to compare against. They've never seen how the top performers actually use AI, and without that reference point, "I use ChatGPT every day" feels like expertise. AISA provides both the objective benchmark and the comparison point.

Score Interpretation: The Five Bands in Practice

Understanding what each band looks like in practice helps managers and L&D leaders act on AISA results.

A team with an average composite score of 35 (Developing) is a team where most people have tried AI tools but haven't built reliable habits around them. Training should focus on foundational skills: structured prompting, output verification, and basic tool selection. This is where most teams land today, and the gap is largest in Critical Thinking — people use AI but don't systematically evaluate what it gives them.

A team averaging 55 (Competent) has functional AI skills. They can get work done with AI tools. The upskilling opportunity is moving from "I know how to use this" to "I know why this works and when it will fail." Targeted training on limitation awareness and workflow optimization yields the highest ROI at this level.

A team averaging 75 (Proficient) is operating at a level where AI is genuinely integrated into their work. These teams benefit more from peer learning, advanced tool exploration, and domain-specific AI application workshops than from foundational training.

For a deeper analysis of how scores distribute across roles and dimensions, see our 2026 AI Skills Report. For guidance on turning score data into upskilling plans, read The AI Skills Gap: How to Benchmark and Upskill Your Existing Team.

Why This Rubric Works

The rubric is built on three principles that set it apart from simpler AI skill assessments.

It measures behaviour, not knowledge. Someone who knows the definition of "few-shot prompting" but never uses it effectively scores lower than someone who structures examples into their prompts intuitively. You can't fake behaviour over a 25-minute adaptive conversation the way you can select the right answer on a multiple-choice test.

It separates dimensions that are often conflated. Many teams assume that someone who writes good prompts also evaluates outputs carefully. In practice, these skills are independent — a person can be a Proficient prompt engineer and a Developing critical thinker. The multi-dimensional structure surfaces these asymmetries so that hiring and training decisions can be precise.

It scales across roles. The same rubric applies to developers, product managers, designers, and data scientists, but the conversation adapts to each role's context. A developer's W3 score reflects AI application in software engineering; a designer's W3 score reflects AI application in design workflows. The criteria are universal; the evidence is domain-specific.

The rubric isn't a static document. As AI tools evolve and the baseline of "what every professional should know" shifts, the criteria descriptions and band definitions will be updated — the Safety dimension in particular is one we plan to expand with sub-criteria as the data matures. But the five dimensions — communication, critical thinking, technical understanding, workflow integration, and safety — represent durable categories of AI proficiency that will remain relevant regardless of which specific tools dominate the market.