It starts with a language model, not magic
Underneath every AI coding tool is a large language model — a system trained on enormous amounts of text and code that predicts, step by step, what should come next given everything it's seen so far. It doesn't "know" your codebase the way a human teammate does; it reasons over whatever text is currently in front of it and generates a plausible continuation, which for a coding task means plausible code, explanations, or commands.
This matters practically: the model is only as good as what it can actually see and how clearly the task was described. Ambiguous instructions produce plausible-but-wrong output because the model is filling gaps with an assumption, not asking you to clarify.
Context: what the model can actually see
Every model has a context window — a limit on how much text (your prompt, relevant files, prior conversation) it can consider at once. Coding tools spend a lot of engineering effort deciding what to put into that window: which files are relevant, which parts of a large file matter, what past steps in the session are worth keeping.
This is why the same underlying model can perform very differently across tools — a tool that retrieves the right three files performs better than one that dumps in the wrong twenty, even with an identical model doing the reasoning.
Tool use: how an agent edits files and runs commands
A chat model that only outputs text can't edit your project by itself. What turns it into a coding agent is tool use (sometimes called function calling): the model can emit a structured request — "read this file," "write this diff," "run this shell command" — which the surrounding application actually executes, then feeds the result back to the model as more context.
This is the mechanical basis for everything an agent like Claude Code or OpenAI Codex does: read, edit, run, observe the output, decide the next step. None of it requires the model to have special access to your machine beyond what the tool explicitly grants it.
The agentic loop: plan, act, observe, repeat
A single-shot suggestion (autocomplete) and an agentic workflow differ mainly in how many times this loop runs before it hands control back to you. An agent will often: form a rough plan, make an edit, run a command to check it (a test, a build, a linter), read the result, and decide whether to continue, adjust, or stop and ask you something.
- Plan — break the request into a rough sequence of steps
- Act — make an edit or run a command
- Observe — read the actual output: a passing test, an error, a diff
- Decide — continue, backtrack, or surface something to the human
Why tests and error output matter so much
A model that only "thinks" it wrote correct code, with no way to check, is guessing. A model that can run your test suite and read a real failure has grounding — it's reasoning about actual, current information instead of its own prior assumption. This is a large part of why agentic tools that can execute code tend to produce more reliable results than pure chat, and why having any tests at all in a project meaningfully improves what an AI can do with it.
Where it still breaks down
None of this makes the model infallible. It can misread an ambiguous requirement, over-fit a fix to make a specific test pass without solving the underlying problem, or make a confident-sounding change that's subtly wrong in a way no test happens to catch. Understanding the mechanism doesn't remove the need for review — it explains why review matters. Our guide to debugging AI-generated code picks up directly from here.