In this guide · 8 sections
How does AI work? It is the part of AI fluency most professionals skip, and it shows. In the AISA AI Fluency Index, the two lowest-scoring skills of all eleven are AI Fundamentals and Tool Landscape, and both belong to Technical Understanding. You do not need to build a model to close that gap. You need a working picture of what happens between your prompt and the answer, and a clear sense of which tool fits which job. This guide covers both, with exercises to try this week and the best free resources to go further.
What Technical Understanding of AI Means
Technical understanding means having a model of how AI produces its answers that is accurate enough to change what you do. It is one of the five dimensions of AI fluency and covers two skills: AI fundamentals, meaning how models are trained and run, and the tool landscape, meaning which assistant, model or integration to use for a given job.
AI fundamentals
AI fundamentals is the difference between knowing the word "token" and knowing why a long conversation starts to lose details from the beginning. People who score well here explain concepts such as token prediction and context windows in their own words, then use them to decide what to do: start a fresh chat, attach the source document, or switch to a model that reasons step by step.
The tool landscape
The tool landscape is knowing your options and choosing between them on purpose. A strong answer to "which AI do you use?" is never one product name. It is two to four tools, each with a reason, plus an awareness of what sits underneath them: APIs, agents, and connectors built on the Model Context Protocol.
How Does AI Work? Five Core Ideas
A large language model predicts the next piece of text, one token at a time, from patterns it learned in training. Almost everything you notice about AI follows from that: what it knows, what it forgets, why it sometimes invents things, and why tools and retrieval make it more useful.
1. It predicts the next token
Text is split into tokens, short chunks of words, and the model assigns a probability to every possible next token. It is not looking anything up. A setting called temperature controls how adventurous the choice is, which is why the same prompt can give different answers.
2. Its knowledge stops at a training cutoff
What a model "knows" was fixed when it was trained. Anything newer reaches it only through search, an attached file or a connected tool. If an answer about last month's news sounds confident without a source, treat it as a guess. Training data cutoffs explains how to spot this.
3. The context window is its working memory
Everything in the conversation, your files and its own replies, has to fit in the context window. As it fills, recall of the middle and the beginning gets less reliable. A fresh conversation for a fresh task is the simplest fix.
4. Retrieval and tools add what training left out
Retrieval-augmented generation finds the relevant passages in your documents and places them in the prompt, so answers rest on your sources rather than on memory. Fine-tuning changes the model itself and is rarely the first thing to reach for.
5. Agents put the model in a loop with tools
An agent is a model that can call tools, look at the result and decide the next step, repeating until the job is done. That is powerful and it raises the stakes: an agent that can act can also act on bad instructions hidden in a web page or an email.
Capability is uneven, which is why this picture matters. Stanford's 2026 AI Index Report notes that a model earned a gold medal at the International Mathematical Olympiad, yet the top model reads analog clocks correctly just 50.1% of the time. Knowing how AI works is how you predict which side of that line your task falls on.
Technical AI Skills at Every Level
AISA scores Technical Understanding from 1 to 10, the same scale as the rest of the AISA rubric. This is what each band looks like in practice.
| Level | Score | What it looks like |
|---|---|---|
| Novice | 1–2 | AI is a black box. Uses one tool and is not aware of the alternatives. |
| Developing | 3–4 | Knows the vocabulary (tokens, context window) without applying it. Names several tools but cannot say when to use each. |
| Competent | 5–6 | Explains core concepts and uses them in real decisions. Uses two to four tools with a clear reason for each. |
| Proficient | 7–8 | Uses technical understanding to choose and deploy AI. Works with APIs, agents, MCP and automation tools. |
| Expert | 9–10 | Understands training, inference, fine-tuning and retrieval at a system level. Builds custom agents and integrations. |
Across 1,800 assessments, the average AI Fundamentals score is 5.0 and the average Tool Landscape score is 4.8 out of 10. The average professional sits at the very start of the Competent band, so a few weeks of deliberate practice can lift you above the average.
The Best Resources to Learn How AI Works
These are the resources we would hand a colleague who wants a real understanding of how AI works without a computer science degree. All of them are free. Start with the foundational ones and move down the list.
Start here
Foundational · 6- VideoFreeLarge Language Models explained briefly 3Blue1BrownA short animated explainer of how a model predicts the next word
- CourseFreeAI Capabilities and Limitations Anthropic AcademyA working model of why AI behaves as it does, with hands-on activities
- CourseFreeIntroduction to generative AI and agents Microsoft LearnHow models work, how prompts shape output, and the core ideas behind agents
- VideoFreeHow I use LLMs Andrej KarpathyA practical tour of choosing models, search, deep research and file uploads
- GuideFreeAn opinionated guide to which AI to use Ethan MollickA current map of the assistants and which tier to use for which work
- ReportFreeThe 2026 AI Index Report Stanford HAIThe yearly evidence on what AI can and cannot yet do
Build on it
Intermediate · 5- InteractiveFreeTransformer Explainer Georgia TechA small model running in your browser: see tokens, probabilities and temperature live
- VideoFreeDeep Dive into LLMs like ChatGPT Andrej KarpathyA long general-audience lecture on how models are trained and why they hallucinate
- DocsFreeContext windows Claude DocsWhat counts toward working memory and why recall degrades as it fills
- DocsFreeWhat is the Model Context Protocol? MCP docsThe open standard that connects assistants to files, tools and data
- GuideFreeA practical guide to building agents OpenAIWhat an agent is, when it beats a fixed workflow, and where humans stay in the loop
The landscape moves weekly, and so does the price of using it: the 2025 AI Index found the cost of running a model at GPT-3.5's level fell more than 280-fold between November 2022 and October 2024. To keep your tool knowledge current, read the latest AI news once a week.
How to Learn How AI Works: 6 Exercises
The fastest way to learn how AI works is to watch it work on your own material. Each exercise below takes 10 to 25 minutes with the assistant you already use.
- Exercise 110 min
Watch next-token prediction
Open the Transformer Explainer, type the first half of a sentence from your own work, and look at the predicted next tokens and their probabilities. Move the temperature slider low and high and generate a few times. Notice that the model is choosing likely continuations, not retrieving facts.
- Exercise 215 min
Same prompt, three fresh chats
Run one real work prompt, unchanged, in three new conversations and put the answers side by side. What stays stable is more trustworthy; what changes in substance shows where the model is uncertain. For facts, never rely on a single run.
- Exercise 315 min
Find the knowledge boundary
With web search off, ask about something that happened in your field last month and ask for the model's training cutoff. Then turn search on, or attach the source, and ask again. Notice whether it flagged its uncertainty or answered confidently about something it could not know.
- Exercise 420 min
Needle in your own document
Upload a long document you know well and ask five questions: two answered on the first and last pages, two buried in the middle, and one the document does not answer. Notice which answers were exact and whether it admitted the missing one or invented it.
- Exercise 515 min
Teach it back
Without looking anything up, write 150 words on the difference between pasting documents into the prompt, retrieval and fine-tuning, and say which you would use for one real need in your team. Ask the AI to mark your explanation for errors and gaps. The fuzzy spots become your reading list.
- Exercise 625 min
Two tools, one task
Run a task with a checkable answer in two assistants, or in one assistant's standard and reasoning modes. Score each on accuracy, usefulness and time to a usable result, and start a one-page "which tool when" note you add to every week.
Curious about your AI Fluency?
AISA helps you measure, prove and improve your AI skills — free report in a 20–40 minute chat.
AISApedia Concepts to Learn Next
These are the AISApedia entries behind Technical Understanding, grouped from first steps to advanced. Each one explains a concept in plain language and links a hand-picked resource to go further.
Foundational
Start here11 concepts- AI-Powered Search
AI-powered search engines like Perplexity, Google AI Overviews, and ChatGPT with web access use language models to understand the intent behind a query, retrieve relevant sources, and synthesise a coherent answer — replacing the traditional model of returning a list of links with direct, conversational responses grounded in cited sources.
DocsHow does Perplexity work? - ChatGPT Basics
ChatGPT is OpenAI's conversational AI assistant, one of the most widely used language model interfaces available.
DocsMemory FAQ - Claude Projects
Claude Projects is Anthropic's feature for creating persistent AI workspaces where uploaded documents, custom instructions, and accumulated context carry across conversations.
ArticleCollaborate with Claude on Projects - Context Windows
A context window is the maximum amount of text — measured in tokens — that a language model can process in a single interaction.
ArticleLost in the Middle: How Language Models Use Long Contexts - GitHub Copilot
GitHub Copilot is an AI-powered code completion tool that integrates directly into code editors, generating suggestions based on the current file's context, open tabs, and natural language comments.
DocsAI Readable Naming Convention — Copilot Patterns & Exercises - Hallucination Causes
AI hallucinations occur when language models generate plausible-sounding but factually incorrect information, fabricating details, citations, events, or relationships that do not exist.
ArticleLLM Hallucinations in 2026: How to Understand and Tackle AI's Most Persistent Quirk - Model Comparison
Model comparison is the skill of evaluating different AI models against specific task requirements rather than relying on general benchmarks or marketing claims.
ArticleArena AI: The Official AI Ranking & LLM Leaderboard - NotebookLM
NotebookLM (rebranded as Gemini Notebook in July 2026) is Google's AI research tool that allows users to upload multiple documents and query across them as a unified knowledge base.
TutorialNotebookLM: A Guide With Practical Examples - Perplexity for Research
Perplexity is an AI-powered research tool that combines language model reasoning with real-time web search, delivering synthesised answers with inline source citations.
ArticleHow to Use Perplexity AI for Research: The Complete 2026 Guide - Token Limits
Token limits define the maximum amount of text a language model can process in a single interaction, encompassing both the input (prompt, context, conversation history) and the output (generated response).
DocsLLM Context Window Explained: Tokens, Limits, and How to Work Within Them - Token Prediction
Token prediction is the core mechanism by which large language models generate text.
ArticleThe Illustrated GPT-2 (Visualizing Transformer Language Models)
Intermediate
Build on it9 concepts- API vs Chat Interfaces
API and chat interfaces represent two fundamentally different modes of accessing AI language models.
ArticleChatGPT vs OpenAI API - Key Differences and Use Cases - Cursor IDE
Cursor is an AI-native code editor built on the VS Code foundation that integrates language model capabilities directly into the development workflow.
TutorialCursor AI: A Guide With 10 Practical Examples - Custom GPTs
Custom GPTs are user-configured ChatGPT instances with persistent instructions, uploaded knowledge documents, and optional tool integrations that create specialised AI assistants for recurring tasks.
ArticleAlmost an Agent: What GPTs Can Do - Model Selection Criteria
Model selection criteria are the evaluation dimensions professionals use to match AI models to specific tasks.
ArticleHow to Choose the Best Open Source LLM for Your Project - Temperature Settings
Temperature is a generation parameter that controls the probability distribution over tokens during AI text generation.
TutorialUnderstanding Temperature, Top P, and Maximum Length in LLMs - Token Economics
Token economics is the practice of understanding and optimising the cost structure of AI API usage, where pricing is determined by the number of tokens processed as input and generated as output.
ArticleUnderstanding LLM Cost Per Token: A 2026 Practical Guide - Tokenization Mechanics
Tokenization is the process by which AI language models split input text into discrete processing units called tokens before any computation occurs.
CourseTokenizers - Hugging Face LLM Course - Tool Selection Criteria
Tool selection criteria are the evaluation dimensions professionals use to choose the right AI tool for a given task, moving beyond familiarity or brand loyalty to systematic matching of tool capabilities to task requirements.
ArticleHow to Choose AI Tools For Work - Voice AI Interfaces
Voice AI interfaces are conversational systems where users interact with AI models through spoken language rather than text input.
ArticleAI Voice Agents: 2025 Update
Advanced
Go deep21 concepts- A2A Protocol
An Agent-to-Agent (A2A) protocol is a standardised communication format that allows AI agents to exchange structured data directly, bypassing the lossy translation of natural language.
ArticleAnnouncing the Agent2Agent Protocol (A2A) - Agent Memory Systems
Agent memory systems provide AI agents with persistent state across conversations, enabling them to recall user preferences, project context, prior decisions, and accumulated knowledge between sessions.
ArticleLLM Powered Autonomous Agents - API Integration Patterns
API integration patterns for AI systems are architectural approaches to connecting applications with AI model endpoints in ways that handle the unique reliability challenges of AI services: variable response times, rate limiting, non-deterministic outputs, and service degradation.
DocsProduction Best Practices - Attention Mechanisms
Attention mechanisms are the computational process by which transformer-based language models determine which parts of the input to focus on when generating each token of output.
ArticleThe Illustrated Transformer - Chunking Strategies
Chunking strategies determine how documents are split into segments for AI processing and retrieval.
ArticleChunking Strategies for RAG - Context Caching
Context caching is a technique that stores frequently used prompt content — system instructions, reference documents, few-shot examples — in a reusable form so that subsequent API calls can reference the cached content without retransmitting it.
DocsPrompt Caching - CrewAI Framework
CrewAI is an open-source Python framework for building multi-agent AI systems where specialised agents with defined roles, goals, and tools collaborate on complex tasks.
DocsIntroduction to CrewAI - Embedding Models
Embedding models convert text, images, or other data into dense numerical vectors in high-dimensional space, where semantic similarity corresponds to geometric proximity.
ArticleVector Embeddings Explained - Fine-Tuning Workflows
Fine-tuning adapts a pre-trained language model to a specific task or domain by training it on a curated dataset of examples, embedding behavioral patterns directly into the model's weights rather than providing them through prompts at inference time.
TutorialHow to Fine-Tune Open LLMs in 2025 with Hugging Face - LangGraph Agents
LangGraph is a framework for building stateful, multi-step AI agent workflows as directed graphs, where nodes represent processing steps (LLM calls, tool invocations, data retrieval, validation checks) and edges define control flow including conditional branching and cycles.
DocsLangGraph Quickstart - Local Model Deployment
Local model deployment runs AI language models on your own hardware — laptops, workstations, or on-premises servers — rather than calling cloud-hosted APIs.
ArticleRunning LLMs Locally: Ollama, llama.cpp, and Self-Hosted AI for Developers - MCP Protocol
The Model Context Protocol (MCP) is an open standard that enables AI assistants to interact directly with external tools, data sources, and services through a structured client-server interface.
ArticleIntroducing the Model Context Protocol - ML Model Evaluation
ML model evaluation with AI uses language models to design comprehensive test strategies for machine learning systems, going beyond standard accuracy metrics to systematically probe failure modes, boundary conditions, distributional blind spots, and adversarial weaknesses that conventional test sets miss.
ArticleTask-Specific LLM Evals that Do & Don't Work - Model Benchmarking
Model benchmarking is the practice of systematically comparing AI models on standardized or custom tasks to determine which performs best for a specific use case, balancing quality, cost, latency, and operational constraints.
ArticleMMLU-Pro Benchmark Leaderboard - Model Cards
Model cards are standardized documentation artifacts for machine learning models that disclose training data characteristics, intended use cases, performance benchmarks, known limitations, ethical considerations, and disaggregated evaluation results across demographic groups and input categories.
DocsModel Card Guidebook - Model Distillation
Model distillation transfers the task-specific capabilities of a large, expensive 'teacher' model into a smaller, faster 'student' model by training the student to reproduce the teacher's outputs on a representative dataset.
ArticleEverything You Need to Know about Knowledge Distillation - Quantization Tradeoffs
Quantization reduces a language model's memory footprint and computational cost by representing its weights with fewer bits — typically converting 16-bit floating point values to 8-bit, 4-bit, or even lower precision integers.
TutorialLLM Quantization - RAG Fundamentals
Retrieval-augmented generation (RAG) is an architecture pattern that connects a language model to external data sources.
ArticleRetrieval-Augmented Generation (RAG) - Semantic Caching
Semantic caching stores AI-generated responses and serves them for subsequent queries that carry the same intent, even when phrased differently.
ArticleWhat Is Semantic Caching? Guide to Faster, Smarter LLM Apps - Tool Use Patterns
Tool use patterns define how AI agents interact with external systems — databases, APIs, file systems, and web services — in a controlled and predictable manner.
ArticleFunction Calling Using LLMs - Transformer Architecture
The transformer is the neural network architecture underlying virtually all modern language models, introduced in 2017.
VideoAttention in Transformers, Step-by-Step (Deep Learning Chapter 6)
Measure How Well You Understand AI
Reading builds the model; using it is what counts. In the AISA assessment you talk with Aisa, an AI interviewer, about how you actually work with AI, and your report scores Technical Understanding alongside the other four dimensions of AI fluency, with the specific concepts to work on next. Take the free AI fluency assessment to see your score, then come back to this guide for the level above it.
Related reading: 44% Can't Explain How AI Works — what the data says about AI fundamentals. Related reading: What Is a Good AI Score? — how to read your result against everyone else's. Related reading: How to Use AI at Work — turn what you know about AI into a better workflow.
Frequently Asked Questions
How does AI work in simple terms?
A large language model predicts the next piece of text, one token at a time, based on patterns it learned from a huge amount of training data. It does not look facts up unless it is connected to search, your files or other tools. That is why it can be fluent and wrong at the same time.
Do I need to know how AI works to use it well?
You do not need to build models, but a working picture of how AI works changes your results. It tells you when to start a fresh conversation, when to attach a source, which tool to use, and when an answer needs checking. In AISA's data, Technical Understanding holds the two lowest-scoring skills of all eleven, so it is also where practice pays off fastest.
Which AI tool should I use?
Use two to four tools, each for a reason: one general assistant for everyday work, a reasoning or deep-research mode for hard problems, and specialist tools for your field, such as coding assistants or meeting tools. Test them on tasks with answers you can check and keep a short note of which tool wins for what.
What is the best free course to learn how AI works?
For a quick start, watch 3Blue1Brown's short explainer, then take Anthropic Academy's AI Capabilities and Limitations or Microsoft Learn's Introduction to generative AI and agents. Both are free. When you want depth, Andrej Karpathy's Deep Dive into LLMs like ChatGPT covers the full picture in plain language.
Curious about your AI Fluency?
AISA helps you measure, prove and improve your AI skills — free report in a 20–40 minute chat.
