AI Skills Assessment Accuracy [Wingie Case Study]

Every buyer of an AI skills assessment asks the same question first: does the score match the person? AI skills assessment accuracy is easy to claim and harder to show, so here it is shown on a real team. Wingie Enuygun, the online travel company, put its product team through AISA, and the director who knows each of them best compared the results with his own read.

His verdict: the results were a fair reflection of every person on the team, his own included.

This post covers how the check worked, what the team scored, why the result holds up, and how to run the same check on a team of your own.

What AI Skills Assessment Accuracy Means to a Buyer

AI skills assessment accuracy means the scores agree with how people actually work with AI. The most direct test is to compare the results, person by person, with the judgement of someone who sees that work every day. If the assessment places people where their manager would, it is measuring the real skill.

Why the manager is the yardstick

A director who works closely with a team has watched a great deal of real work. They know who reaches for AI first, who checks what it produces, who has built it into how the team ships, and who still treats it as an experiment. No questionnaire captures that. It is the closest thing an organisation has to ground truth about its own people.

What counts as a match

A match is not one number lining up. It is the whole picture: the strongest person comes out on top, the people a manager would place in the middle land in the middle, and the people still building the habit show up where the manager expects them. In assessment science this is a criterion validity check: do the scores agree with an informed judgement of the same people, made independently of the assessment? The AISA assessment quality framework names it as the test that matters most: when managers compare results with how their people actually work, the two should match.

The Test: A Product Team and the Director Who Knows Them

Wingie Enuygun's product team took AISA in late September 2026: eleven people, including their director, Behcet Mutlu, Director of Technology and Product Management. Everyone took the same assessment and received a full report. The director then compared the results with his own knowledge of each person on his team, himself included.

Who took it

The group was the product team at Wingie Enuygun, one of Turkey's online travel platforms: the director and the people on his team. He knows each of them through their work, which is what made the check possible.

An AI-native product team

Wingie Enuygun has made AI part of how its product organisation works. The team is developed through training and workshops, and pointed toward AI applications people can put into practice in their own work. It runs an agentic product management structure, alongside product skills it keeps developing. Behcet describes the result as a fully AI-native product management philosophy.

How the comparison worked

Each person talked through how they actually work with AI and showed it in live exercises. AISA scored what each of them demonstrated, against a published rubric, and every result was calibrated and integrity-verified before the report was released. The director's read came from working with them, not from the reports. The question was simply whether the two pictures would agree.

The Result: 92 at the Top, and a Fair Reflection

The director scored 92 out of 100. Nobody else on the team reached 80, and most scored between 60 and 79. Setting the results against his own judgement of each person, he found them a fair reflection of the team, his own score included.

The scores

• 11 people assessed, the director included
• The director: 92, in the top band of AISA's scale (92 to 100), a level reached by 1.3% of assessed professionals, as the AI fluency levels show
• Everyone else: below 80, with most between 60 and 79

That spread is what gives the comparison its strength. An assessment that put everyone in one band could not be checked against a manager's read at all. A team that runs from the middle of the scale to its top band gives the comparison something real to test: order, distance and outliers.

The director's verdict

"The results were a fair reflection of my team, myself included."

Behcet Mutlu, Director of Technology and Product Management, Wingie Enuygun

"Myself included" matters. The same standard that placed each member of his team placed him too, and he found both fair. A director at the top of his team, calling the picture fair: that is the check in one sentence.

Why AI Skills Assessment Accuracy Holds Up

The results held up against the director's view because AISA measures what people demonstrate, not what they say about themselves. Every score is tied to evidence from the person's own conversation, every result is calibrated and integrity-verified, and the results hold steady when people retake it. Self-ratings, by contrast, miss by a wide margin.

Evidence, not self-report

AISA asks people how they work with AI, then has them show it. Strong habits show up in specifics: keeping a human in the loop on decisions that matter, running a verification checklist before AI output is shared, knowing which tool suits which task. Confident talk without those specifics does not score highly.

The wider research on measuring people points the same way. In a 2022 re-analysis of decades of personnel selection studies, Sackett, Zhang, Berry and Lievens found that structured interviews had the highest mean validity of every procedure they reviewed, at .42 (Journal of Applied Psychology). Structure and evidence predict how people perform better than impressions do.

Self-ratings miss

People are poor judges of their own skill. In Kruger and Dunning's 1999 study, participants who scored in the 12th percentile on a logical reasoning test placed themselves in the 62nd (Journal of Personality and Social Psychology).

AISA's own data shows the same pattern for AI. Across 274 professionals who predicted their score before being assessed, the average prediction was 63.4 and the average result 44.9: an overestimate of 18.5 points, set out in the five AI skills professionals overestimate.

Wingie's team made predictions too. As a group they came close, in line with our finding that product managers are the most accurate self-assessors. Person by person, though, their predictions missed their actual score by 8.9 points on average, some too high and some too low. The director, reading the same results, found them a fair reflection of each person.

Results that hold on retake

A score that moved every time someone sat the assessment could never match anyone's judgement. AISA's results are stable: 98% of people who retake it land in the same or an adjacent tier, and scores correlate at r = 0.79 between attempts. The picture the director recognised comes from a measurement that holds when it is repeated.

From One Manager's Read to Every Team

A good manager's read is valuable, but it covers one team, through one person's eyes, and it lives in that person's head. An evidence-based assessment gives the same read, to the same standard, across every team in the organisation, so training goes where it is needed and progress can be measured again.

What a manager's read cannot do on its own

• Compare across teams. Two directors with different standards produce two pictures that cannot be lined up.
• Show its evidence. The judgement sits in one person's memory, with nothing anyone else can check.
• Measure progress. Without a recorded baseline, there is nothing to measure improvement against.

What a shared standard adds

With a shared baseline, an organisation can see which teams need what across AI fluency's core skills, from prompting and critical thinking about AI output to workflow and safety, and point training at the gaps instead of spreading it evenly. The AI Fluency Index gives the wider benchmark to compare against.

How to Run the Same Check on Your Team

You can repeat Wingie Enuygun's check in your own organisation. Pick a team you know well, write down your read of each person before anyone takes the assessment, then compare the results with your notes person by person. Where the two agree, you have a baseline you can build on. Where they differ, the report shows exactly why.

Step 1: Write your read down first

Behcet Mutlu's recommendation for anyone running this check is to make it blind. Before the reports arrive, write down where you expect each person to land, as a rough order or a band, then see who lands where you predicted and who does not. Writing it first keeps hindsight out of the comparison.

Step 2: Run the assessment for the whole team

Everyone takes the same assessment and gets a full report. With a team AI assessment, every result lands in one dashboard, alongside team-level analytics.

Step 3: Compare person by person

Set the results next to your notes. Check the order, the bands and the outliers. Every score links to quotes from the person's own conversation, so any difference becomes a specific conversation about specific work.

Step 4: Use it as your baseline

Once the picture is confirmed, it becomes the starting line. Plan training against the gaps, then measure again to see what moved.

Or start with yourself: take the assessment and see whether the result matches your own read.

---

Related reading: AI Skills Gap: You Overestimate by 18.5 Points. What 274 professionals predicted, against what they scored.

Related reading: Product Managers AI Skills: Most Accurate Self-Assessors. Which roles judge their own AI skills best.

Related reading: AI Skills Assessment for Teams: 7 Tools [2026]. How team assessment tools compare.

Frequently Asked Questions

How accurate is an AI skills assessment?

It depends on what it measures. Self-rating questionnaires miss badly: professionals overestimate their AI skills by 18.5 points on average. An assessment that scores demonstrated skill against a published rubric can be checked against the people who know the work best. At Wingie Enuygun, the product director found AISA's results a fair reflection of every person on his team, himself included.

Can a manager's judgement replace an AI skills assessment?

A manager's judgement is a strong check for the team they know, but it cannot be compared across teams, shown to anyone else, or repeated to a common standard. An AI skills assessment gives the same evidence-based read for every team, which is what an organisation needs to direct training and measure progress.

How do I check AI skills assessment accuracy for my own team?

Write down your read of each person before they take the assessment, run it for the whole team, then compare the results with your notes person by person. Accuracy shows up as agreement on order, bands and outliers, and the report's quotes explain any difference.

Why are self-assessments of AI skills unreliable?

People judge their own skill poorly, and the least skilled tend to overestimate most. In AISA's data, 274 professionals predicted an average score of 63.4 and scored 44.9. Even Wingie Enuygun's product team, which predicted well as a group, missed by 8.9 points per person on average.