Prompt Engineering is a Legacy Skill

Context over Tricks

In 2024, we spent hours optimizing "system prompts" and finding magic phrases like "take a deep breath" to get better results. In mid-2026, with models like GPT-5.6 and Claude Sonnet 5, the model understands intent instantly. The era of "Prompt Engineering" as a pseudo-mystical art is over.

It is worth being precise about what died, because the claim is easy to overstate. Clear instructions still matter enormously. What stopped mattering is wording as a lever — the belief that a better phrasing of the same request unlocks capability the model was withholding. Frontier models now infer intent from a plainly-stated task about as well as they do from an elaborately-tuned one, so the returns on rewording collapsed to roughly zero while the returns on giving the model better information stayed enormous.

What Replaced It: Context Engineering

The successor skill has a name, and it is a broader job than retrieval. Everything that occupies the model's window on a given request is engineered, and there are five distinct inputs to design:

• Instructions — the stable description of the task, the output contract, and the constraints. Written once, versioned, reviewed like code.
• Retrieved knowledge — the documents, code, or records fetched for this specific request. The part most people mean when they say RAG.
• Memory — what the system carries forward about this user or this session, and just as importantly what it deliberately forgets.
• Tool definitions — the functions the model can call, and the descriptions that tell it when each one is appropriate.
• State — the transcript so far, the results of prior tool calls, and whatever the current step of a workflow needs to know.

Prompt engineering, in the old sense, was optimizing the first of those five. Context engineering is designing all five as a system, under a fixed budget, with an eval suite telling you whether each change helped.

The Context Budget

Large windows created a trap. When you can send a million tokens, the temptation is to send a million tokens, and long-context models do not attend uniformly across everything you give them. Material buried in the middle of a very long context is reliably recalled less well than the same material placed near the beginning or the end — a well-documented effect that no amount of window size makes disappear. Sending more is not the same as being understood better.

Treat the window as a budget with competing claimants. Instructions and the output contract go at a stable position where they are always attended to. Retrieved evidence should be the minimum set that actually supports an answer, ordered so the most relevant sits at an edge rather than the middle. Transcript history gets compacted rather than carried indefinitely. And every token you add has a cost as well as a risk — our breakdown of token economics covers why over-retrieval is simultaneously a quality problem and a line item.

The New Skill: Data Curation

Instead of prompt engineering, successful developers focus on context curation—feeding the model the right documents and examples to ground its reasoning. Garbage in, garbage out still applies, but now it's about the data, not the prompt syntax.

RAG (Retrieval Augmented Generation) pipelines are the new prompt engineering. How do you chunk your data? How do you rank it? How do you present it to the model? These are the high-leverage questions today.

Chunking is where most pipelines are quietly broken. Fixed-character splitting is the default in every tutorial and the wrong answer for almost every real corpus, because it cuts through tables, numbered procedures, and code blocks at arbitrary points. Splitting on document structure — headings, sections, function boundaries — and attaching the parent heading path to every chunk as metadata is a single afternoon of work that improves answer quality more than any prompt change will. The reason is unglamorous: a chunk that says "set this to 30 seconds" is useless without the heading that says which setting it belongs to.

A Worked Example: Support Bot Grounding

Consider a support bot built on GPT-5.6 Terra. Two years ago, a team would have spent a week iterating on the system prompt, trying phrasings like "you are a world-class support agent, think step by step" to squeeze out better answers. Today the higher-leverage work is entirely upstream: chunking the help-center docs by semantic section rather than fixed character count, tagging each chunk with product-version metadata so stale docs don't get retrieved for current users, and re-ranking retrieved chunks by recency before they ever reach the model. Teams that made this shift report far fewer hallucinated answers than teams still tweaking prompt wording — the model was never the bottleneck; the retrieval pipeline was.

Evaluation Driven Development (EDD)

The other side of the coin is evaluation. You don't improve prompts by guessing; you improve them by running benchmarks. Tools that allow you to systematically test your prompts against 100 test cases are the IDEs of the prompt era.

A minimal EDD loop looks like this: maintain a golden set of real user queries with expected answer characteristics, run every prompt or retrieval change against that set before shipping, and track a small number of metrics — factual accuracy against your source docs, refusal rate on out-of-scope questions, and latency — over time. This turns prompt and context changes from a vibes-based guessing game into something closer to normal software regression testing, which is exactly the point.

Building the golden set is the part teams postpone and shouldn't. It does not need to be large; fifty to a hundred real queries pulled from your logs, each annotated with what a correct answer must contain and what it must not claim, is enough to catch most regressions. Include the awkward cases deliberately: questions your docs genuinely can't answer, questions where two documents conflict, and questions phrased the way real users phrase them rather than the way your team would.

Measure the retrieval layer separately from the generation layer, because otherwise you cannot tell which one broke. Retrieval has its own metric — whether the chunk containing the answer appeared in the top results at all. If it didn't, no model and no prompt can save the answer, and tuning either one is wasted effort. Generation quality is then a question of groundedness: does every claim in the answer trace to something in the retrieved context, or did the model fill a gap from memory? Separating those two measurements is the single highest-leverage thing most teams could do to their eval setup.

Tool Definitions Are the New Prompt

In agent-shaped systems the leverage has moved again, this time into the function schemas. An agent decides which tool to call almost entirely from the tool's name, its description, and its parameter documentation — that text is a prompt, whether or not anyone on the team treats it as one. Two tools with overlapping descriptions produce an agent that picks unpredictably between them, and the resulting bug looks like model unreliability while actually being an interface design problem.

The practices that work here are ordinary API design practices. Keep the tool surface small; an agent choosing between six well-separated tools behaves far more predictably than one choosing between twenty overlapping ones. Say explicitly in the description when not to use a tool. Make parameters typed and validated so a malformed call fails with a message the agent can act on rather than silently doing the wrong thing. Our piece on agentic engineering covers the review side of this; the point here is that the interface is context, and it deserves the same versioning and eval coverage as any prompt.

What's Actually Left to "Engineer"

None of this means prompting doesn't matter at all. Clear task framing, explicit output format constraints (JSON schemas, XML tags), and well-chosen few-shot examples for genuinely novel task types still move the needle. What's gone is the need for incantations and superstition. The skill has moved from "finding the magic words" to "building the pipeline that gets the model the right information at the right time" — which is a data engineering problem, not a wordsmithing one.

Two places where careful wording still earns its keep are worth naming, because blanket claims are how good advice becomes wrong. Small and on-device models are far more sensitive to phrasing and formatting than frontier models are, so if you are deploying at that tier the old craft has not depreciated. And any output that another program will parse deserves an explicitly specified schema rather than a polite request, because "return JSON" and "return an object matching this schema, with no prose before or after" fail at very different rates.

The Skill Stack, Then and Now

The 2023 Skill	Its 2026 Replacement
Finding phrasings that unlock capability	Writing a task and output contract once, then versioning it
Chain-of-thought prompting by hand	Choosing a reasoning tier per code path and paying for it deliberately
Stuffing every relevant document into the window	Retrieval evaluation and a deliberate context budget
Judging output quality by reading a few samples	A golden set, separated retrieval and groundedness metrics, and regression runs in CI
Prompt libraries and shared prompt spreadsheets	Versioned instructions, tool schemas, and eval suites in the repository

The pattern across every row is the same: an artisanal, unversioned activity turning into a measured, reviewable engineering discipline. That is usually what "a skill became legacy" actually means — not that the problem went away, but that the professional version of the answer stopped being a craft secret.

The New Job Title: Context Engineer

If prompt engineer was the job title of 2023, "context engineer" is quietly becoming the job title of 2026. The role owns the retrieval pipeline end to end: what gets embedded, how it's chunked, how it's ranked, what metadata rides alongside each chunk, and how stale or conflicting information gets resolved before it reaches the model. This is a much closer cousin of a data engineer or a search relevance engineer than it is to the "prompt whisperer" archetype of a few years ago, and it's being compensated accordingly — companies are hiring specifically for RAG pipeline expertise now, not generic "AI prompting" skills.

Common Failure Modes in Context Pipelines

Even with frontier models like GPT-5.6 and Claude Sonnet 5 handling the reasoning, a poorly built context pipeline still produces bad answers. The most common failure modes we see in the wild:

• Chunking that ignores document structure. Splitting a table or a numbered procedure across two chunks destroys the meaning of both halves.
• No recency signal. Retrieving a technically-relevant but outdated document over a newer, more accurate one because the ranking only considers semantic similarity, not freshness.
• No conflict resolution. Two documents disagreeing on a fact, both retrieved, with no mechanism telling the model which one to trust.
• Over-retrieval. Stuffing twenty marginally relevant chunks into context "just in case," which dilutes the model's attention and often produces worse answers than five well-chosen ones.

Every one of these is a data and pipeline problem, solvable with better engineering discipline, not a better prompt. That's the whole thesis of this piece: the leverage moved upstream, and the teams that noticed early have a real, compounding advantage over the ones still polishing system prompts.

Where to Start This Week

If you want the shortest path from "we tweak prompts" to "we engineer context," do these four things in order. Pull fifty real queries from your logs and annotate what a correct answer must contain — that is your golden set. Add a retrieval metric so you know how often the right chunk even reaches the model. Re-chunk on document structure instead of character count and attach heading paths as metadata. Then, and only then, revisit your instructions, which you will probably find need shortening rather than lengthening.

None of that is glamorous and all of it compounds. The wider argument for why this style of work has become the constraint sits in the vibe coding manifesto — specification and verification beat clever authorship — and the same insight applied to project management is in why Linear's method wins. If you are choosing which model to build this on top of, our Gemini 3 Pro deep dive covers what a long-context frontier tier actually buys you, and the best AI assistants ranking covers the wider field.