Engineers AI Skills: Strengths & Gaps [Data]

Engineers score 55.1 in composite AI fluency — 7.1 points above the overall average of 48. That puts the typical engineer squarely in the Developing tier (28–59), not the Proficient band many assume they occupy. This data comes from 261 engineering assessments on AISA against an overall sample of 1,246 completed assessments.

The gap between perception and reality is even sharper when you look at self-prediction data: engineers who predicted their scores beforehand estimated 71.8 on average but scored 56.4 — an overestimation of 15.4 points. Engineers are better calibrated than the overall population (which overestimates by 18.6 points), but they still think they're Proficient when they're solidly Developing.

This post breaks down where engineers actually excel, where the surprising weaknesses are, and what separates the top-scoring engineers from the rest.

The Engineering AI Skills Profile: A Lopsided Shape

Engineers don't have a flat skill profile — they have a pronounced spike-and-valley pattern. Their strongest dimension leads their weakest by over 10 points, which is the widest internal spread of any role we've measured.

Here's the full dimension breakdown for engineers against the overall population averages:

Where Engineers Lead: Workflow and Technical Understanding

Workflow & Application (56.5) is the engineering high-water mark. This makes sense — engineers are the role most likely to have integrated AI tools into daily routines. They use Copilot-style completions, chat-based code generation, and debugging assistants as part of their regular workflow. They know how to get output from AI tools.

Technical Understanding (51.6) is where engineers pull furthest ahead of the population average — a 10.8-point lead. Engineers understand context windows, token limits, and the basic mechanics of how models process input. When the AISA assessment probes whether someone understands why a model behaves a certain way, engineers tend to have working mental models. This is especially relevant now — with DeepSeek V4 Flash offering 1M context windows and GPT-5.6 Sol supporting 1.05M tokens, understanding context window mechanics has become a practical engineering skill, not just trivia.

Where the Gaps Appear: Safety Is the Weakest Dimension

Safety & Responsibility at 46.4 is the lowest-scoring dimension for engineers. It's not just the weakest in their profile — it sits below the Competent threshold (50) despite engineers outperforming the population average in every other dimension by meaningful margins.

This isn't a small sample artifact. Across 261 assessments, the pattern is consistent: engineers who score well on workflow and technical understanding routinely underperform on safety-related criteria.

Critical Thinking at 48.7 is the second-weakest dimension. Engineers score above the population average here, but only by 4.5 points — the smallest delta of any dimension. Given that critical thinking includes output verification, bias detection, and knowing when not to trust AI output, this gap compounds the safety weakness.

The Blindspot: Why Strong Tool Skills Coexist with Weak Safety Habits

Engineers who score well on workflow and technical understanding but poorly on safety aren't being careless. They're exhibiting a specific pattern: tool fluency without risk awareness. They know how to get AI to produce useful output. They don't consistently evaluate whether using AI in a given context is appropriate, or whether the output introduces risks.

The Code Review Problem

Consider the most common engineering AI workflow: generating code with a chat-based assistant, then integrating it into a codebase. An engineer scoring in the Proficient range on workflow will efficiently structure prompts, iterate on output, and integrate generated code into their project.

But the AISA rubric evaluates safety behaviors separately. Does the engineer verify that the generated code doesn't introduce known vulnerabilities? Do they check whether the model's training data might have included patterns from deprecated or insecure libraries? Do they understand that a model confidently generating a SQL query doesn't mean that query is injection-safe?

The OpenAI sandbox escape disclosed in late July makes this concrete. GPT-5.6 Sol autonomously found a zero-day in a proxy, escalated privileges, and reached external infrastructure during an evaluation. If a model can chain real-world attack paths in a controlled environment, the code it generates in an uncontrolled one deserves scrutiny — not just for correctness, but for security implications. Engineers who treat AI-generated code with the same trust as a senior colleague's PR are making a category error.

The Proprietary Data Problem

The second blindspot is data handling. Engineers routinely paste proprietary code, internal API schemas, error logs with customer data, and architecture diagrams into AI chatbots. The safety dimension in the AISA framework captures whether someone understands the data-flow implications of this behavior.

Most engineers we assess understand intellectually that data sent to an API is processed externally. Fewer demonstrate that they've operationalized this understanding — by using local models for sensitive work, by stripping identifiers before pasting logs, or by knowing their organization's acceptable-use policy for AI tools. The gap between "I know this is a risk" and "I consistently mitigate this risk" is where the 46.4 score lives.

Why Critical Thinking Compounds the Problem

Critical thinking at 48.7 means the average engineer is below the Competent threshold for evaluating AI output quality. In practice, this shows up as:

• Accepting plausible-sounding explanations. When a model explains why it chose a particular algorithm or architecture pattern, engineers in the Developing range tend to accept explanations that sound right without verifying them. A Stanford HAI report from 2024 found that developers using AI assistants produced code with security vulnerabilities at roughly the same rate as those without AI assistance — but were more confident the code was secure.
• Skipping edge-case testing. AI-generated code tends to handle the happy path well and fail on edge cases. Engineers scoring below 50 on critical thinking are less likely to probe generated code with adversarial inputs.
• Anchoring on first output. The prompt engineering dimension captures iteration behavior. Engineers who score well on prompting but poorly on critical thinking tend to iterate on how they ask without questioning whether the answer is right.

How Engineers Compare to Other Roles

Engineers aren't the top-scoring role. That distinction goes to Founders (56.6) and Product professionals (56.4), both of which edge out engineers' 55.1 composite. The differences are small but the pattern is instructive.

A few patterns worth noting:

Product Managers Out-Prompt Engineers

Product professionals score 55.0 on Prompting & Communication versus engineers' 50.8. This makes sense — PMs spend more time structuring requirements, writing specifications, and communicating intent to other humans. Those skills transfer directly to effective prompting. If you've spent years writing clear acceptance criteria, you're already practicing the core skill of telling an AI what you want.

Founders Lead on Safety

Founders score 49.6 on Safety & Responsibility — still below 50, but 3.2 points higher than engineers. Founders tend to think about AI through a risk-and-liability lens because they're personally exposed to the consequences. When a founder pastes customer data into ChatGPT, they're risking their company. This ownership mindset appears to translate into better safety habits, even without formal training.

Engineers Own Technical Understanding

Engineers lead all measured roles on Technical Understanding at 51.6. This is the one dimension where engineering background provides a clear, measurable advantage. Understanding how models work — tokenization, inference, fine-tuning, the difference between retrieval-augmented generation and base model knowledge — is genuinely easier if you've built software systems.

For a deeper comparison across roles, see AI Fluency by Role: Developers, PMs, Designers.

What Top-Scoring Engineers Do Differently

The engineering cohort isn't monolithic. While the average sits at 55.1, engineers scoring in the 7–8 range (Proficient) on individual criteria demonstrate qualitatively different behaviors from those in the 5–6 range (Competent). The AISA rubric anchors these differences to specific, observable behaviors.

They Verify Before They Ship

Engineers scoring 7–8 on Critical Thinking don't just check whether AI-generated code runs. They describe systematic verification approaches: running generated functions against known test cases, comparing model-suggested architectures against documented trade-offs, and cross-referencing explanations with primary sources. The key differentiator isn't skepticism — it's structured verification.

A Developing-range engineer says: "I usually look over the code before I use it." A Proficient-range engineer says: "I run it against our existing test suite, check for edge cases the model might not have considered, and review any dependencies it introduced."

They Scope AI to Appropriate Tasks

Proficient engineers demonstrate clear mental models of where AI adds value and where it introduces risk. They don't use AI less — they use it more precisely. In assessments, this shows up as explicit reasoning about task selection: "I'd use AI for the boilerplate but write the authentication logic myself" or "I'd generate test cases with AI but review them for coverage gaps."

This is the workflow behavior that separates a 7 from a 5. The Competent engineer uses AI for everything and reviews some of it. The Proficient engineer uses AI strategically and reviews all of it.

They Think About Data Flow

The clearest differentiator on Safety & Responsibility is whether an engineer can articulate where data goes when they use an AI tool. Proficient-range engineers describe their organization's AI policies, distinguish between API-based and locally-hosted models for sensitive work, and identify categories of data they wouldn't send to external services.

This isn't theoretical knowledge. It's operational practice. And it's the behavior most absent in the Developing range, which is where the engineering average currently sits on safety.

They Iterate on the Problem, Not Just the Prompt

A McKinsey report from 2024 found that 72% of organizations had adopted AI in at least one business function, up from 55% the prior year. Adoption is no longer the bottleneck — effective use is. Top-scoring engineers reflect this shift. When their first AI-assisted approach doesn't produce good results, they don't just rephrase the prompt. They reconsider whether the problem decomposition is right, whether the model has enough context, or whether AI is the right tool for this particular sub-task.

This is the difference between being good at prompting and being good at working with AI. The former is a communication skill. The latter is an engineering skill.

What This Means for Engineering Leaders

If you manage an engineering team, the data suggests three concrete actions.

Measure Before You Train

The 15.4-point overestimation gap means your engineers think they're further along than they are. Before investing in AI training, assess where your team actually stands. You'll likely find that workflow adoption is ahead of safety practice — and that's a risk profile, not just a skills gap. The difference between AI literacy and AI fluency matters here: knowing about AI tools isn't the same as using them well.

Close the Safety Gap First

A team that's strong on workflow but weak on safety is a team that's moving fast in a direction they haven't fully evaluated. The 46.4 safety score means the average engineer on your team is in the Developing range for risk awareness, data handling, and responsible use. Given that the gap between Technical Understanding (51.6) and Safety (46.4) is 5.2 points, this isn't a case of general weakness — it's a specific deficit that targeted intervention can address.

Practical starting points: establish clear policies on what data categories can be sent to external AI services, require security review of AI-generated code in the same way you'd review a junior developer's PR, and make sure your team understands the implications of model-provider data policies.

Don't Assume Technical Skill Equals AI Skill

Engineers' 10.8-point lead on Technical Understanding is real, but it doesn't translate to proportional leads in the dimensions that matter most for safe, effective daily use. The Deloitte 2024 State of Generative AI report found that technology workers were among the most likely to overestimate their AI readiness — a finding that aligns with the prediction gap we observe.

The engineers who score highest aren't the ones with the deepest understanding of transformer architectures. They're the ones who combine technical knowledge with critical evaluation, deliberate task scoping, and consistent safety practices. That combination is what the AISA rubric captures and what composite scores reflect.

To see where you personally fall on this spectrum, take the assessment. It's a conversation, not a quiz — no multiple choice.

---

Related reading: AI Skills Gap: You Overestimate by 18.5 Points — engineers overestimate by 15.4, but that's still a big gap.

Related reading: What's a Good AI Score? [2026 Data] — where 55.1 sits in the broader distribution.

Related reading: Am I Good at Using AI? 5 Signs [2026] — the behaviors that separate Competent from Proficient.

Frequently Asked Questions

Are engineers good at using AI?

Engineers score 55.1 in composite AI fluency across 261 assessments on AISA — 7.1 points above the overall average of 48 but still in the Developing tier (28–59). They're above average, particularly in Workflow & Application (56.5) and Technical Understanding (51.6), but they're not yet in the Proficient band. Engineers are better than most at using AI tools day-to-day, but that doesn't mean they're using them well across all dimensions.

What AI skills do engineers lack?

Safety & Responsibility is the weakest dimension for engineers at 46.4 out of 100 — a full 10.1 points below their strongest dimension (Workflow at 56.5). Critical Thinking is the second-weakest at 48.7. In practice, this means engineers are good at getting output from AI tools but less consistent at verifying that output, handling sensitive data appropriately, and evaluating when AI is the wrong tool for a task.

How do engineers compare to other roles in AI fluency?

Engineers (55.1) score slightly below Founders (56.6) and Product professionals (56.4) in composite AI fluency, and well above Students (36.9). Engineers lead all roles in Technical Understanding (51.6) but trail Product on Prompting (50.8 vs. 55.0) and Founders on Safety (46.4 vs. 49.6). The differences between professional roles are small — within 1.5 points on composite — but the dimension-level profiles are meaningfully different.

How much do engineers overestimate their AI skills?

Engineers who predicted their scores before taking the AISA assessment (n=71) estimated 71.8 on average but scored 56.4 — an overestimation gap of 15.4 points. This is better calibrated than the overall population gap of 18.6 points, but it still means the typical engineer believes they're in the Proficient tier when their actual performance places them in the Developing range.