Prompt Engineering Had a Good Run. It’s Over Now.
Two years ago, the hottest skill in tech was writing clever prompts. People built entire careers around it. LinkedIn was drowning in “prompt engineer” job titles. Courses selling for $500 promised to teach you the secret sauce of talking to AI.
Then, in June 2025, two of the most influential voices in tech — Shopify CEO Tobi Lütke and former OpenAI/Tesla researcher Andrej Karpathy — both said the same thing within a week of each other: prompt engineering is dead. The real skill is context engineering.
Karpathy put it bluntly: “Context engineering is the delicate art and science of filling the context window with just the right information for the next step. Too little or of the wrong form and the LLM doesn’t have the right context for optimal performance. Too much or too irrelevant and costs go up while performance comes down.”
Lütke was even more direct: “I really like the term ‘context engineering’ over prompt engineering. It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM.”
This wasn’t a semantic debate. It was a public acknowledgment that the way most people use AI is fundamentally broken — and that fixing it requires a completely different mindset.
What Context Engineering Actually Is (And Why It’s Not Just “Better Prompting”)
Here’s the simplest way to understand the difference. Prompt engineering asks: how do I phrase my question? Context engineering asks: what information does the AI need to see before I even ask the question?
Think of it this way. If you hire a brilliant consultant but give them zero background on your company, your industry, or your constraints, they’ll produce generic advice no matter how well you phrase your question. That’s prompt engineering: perfecting the question while neglecting everything around it.
Context engineering flips the equation. You spend 80% of your effort curating what the consultant sees — the right documents, the right data, the right examples, the right constraints — and 20% on the question itself. The result? Output that actually fits your specific situation.
Anthropic’s official definition captures it well: context engineering is “the set of strategies for curating and maintaining the optimal set of tokens during LLM inference, including all the other information that may land there outside of the prompts.”
That “outside of the prompts” part is critical. Your prompt is just one piece. The system instructions, retrieved documents, conversation history, tool definitions, memory from past sessions, example outputs: all of it shapes what the AI produces. Context engineering is the discipline of orchestrating all of those pieces together.
The Numbers Don’t Lie: 30% vs. 90%
This isn’t theoretical. A 2026 study across multiple AI coding agents found that agents operating without proper context files completed tasks correctly roughly 30% of the time. The same agents, given well-crafted context files, hit 90% success rates. Same model. Same task. The only variable was context.
Sourcegraph’s CodeScaleBench study (March 2026) showed equally dramatic results. When testing identical agents on 370 enterprise-scale tasks, agents with proper code context retrieved files 2.2x more accurately, used 95% fewer tool calls, and completed a Kubernetes monorepo task in 89 seconds that previously timed out after 2 hours.
And then there’s the Chroma research on “context rot,” a phenomenon that should terrify anyone dumping entire documents into AI. They tested 18 frontier models (including GPT-4.1, Claude Opus 4, and Gemini 2.5) and found that every single model showed significant performance degradation well before their context windows filled. A model with a 200K-token window started producing notably worse outputs by 50K tokens.
The takeaway is counterintuitive: bigger context windows don’t solve the problem. They actually make it worse if you’re not curating what goes in.
The “Lost in the Middle” Problem Nobody Talks About
In 2023, researchers Liu et al. published a paper that should have changed how everyone uses AI but somehow flew under the radar. They discovered that language models exhibit a U-shaped attention pattern: they pay strong attention to information at the beginning and end of their context, and systematically underweight everything in the middle.
This means if you paste a 50-page report into ChatGPT and the answer you need is on page 25, the model is more likely to miss it than if it were on page 1 or page 50. The information is there, the model just doesn’t prioritize it correctly.
Context engineers account for this. They structure information deliberately: critical instructions and the most relevant data go at the top, secondary context in the middle, and key constraints or reminders at the end. It sounds simple. Almost nobody does it.
The Six Layers of Context (A Practical Framework)
After analyzing dozens of production AI systems and the research literature, here’s the framework that actually works. Context isn’t one thing: it’s six distinct layers, and each one needs deliberate engineering.
Layer 1: System Instructions
The behavioral foundation. This is where you define the AI’s role, tone, constraints, and rules. Not a two-line system prompt like “You are a helpful assistant.” Think 50 to 200 lines of specific, tested instructions that handle edge cases your users will inevitably hit. Example: “If the user asks about pricing you don’t have, say you don’t know. Never guess a number.”
Layer 2: Persistent Memory
Cross-session knowledge that accumulates over time. ChatGPT’s memory feature, Claude’s project knowledge, or custom implementations using databases. This is what makes an AI feel like it actually knows you: your profession, your writing style, your ongoing projects, your preferences. Without it, every conversation starts from zero.
Layer 3: Retrieved Knowledge (RAG)
On-demand extraction of relevant information from your own data. This is the backbone of enterprise AI: querying your company’s documentation, past reports, customer data, or knowledge base in real time. The key insight from practitioners: 40 carefully selected highlights outperform uploading an entire 200-page PDF. More is not better. Relevance is everything.
Layer 4: Tool Use
The model’s ability to take actions and access live data: running calculations, executing code, searching the web, querying APIs, reading calendars. Every tool definition you give the model consumes tokens. Anthropic flagged this repeatedly: “The most common failure mode is bloated tool sets that cover too much functionality or lead to ambiguous decision points about which tool to use.” Give your AI 50 tools, and it wastes time deciding which one to call. Give it 5 precisely-scoped tools, and it moves fast.
Layer 5: Attachments and Session Data
Files, screenshots, URLs, and documents relevant to the current task. The context engineering principle here: be surgical. Don’t upload a full annual report when you only need the Q3 revenue table. Don’t send 20 screenshots when 3 capture the issue.
Layer 6: Conversation History
Previous messages in the current thread, including your corrections, clarifications, and stated preferences. This layer is why long conversations degrade: the history fills the context window with old, potentially irrelevant exchanges while the current task gets squeezed. Smart context engineers compress old history into summaries and keep the active window focused.
How Developers Are Already Using This: CLAUDE.md, AGENTS.md, and .cursorrules
The developer community hasn’t waited for the industry to catch up. Three file standards have emerged as the practical expression of context engineering for AI coding agents:
AGENTS.md is the universal context file. Drop it in your repository root, and every major AI agent (Claude Code, Cursor, GitHub Copilot, OpenAI Codex) reads it automatically. It contains your project overview, tech stack with exact versions, build commands, code style rules, architecture decisions with reasoning, and explicit anti-patterns. Think of it as a new-hire onboarding document, except the new hire processes it in 200 milliseconds.
CLAUDE.md is Claude-specific configuration that goes deeper: when to use extended thinking, which commands the agent can run without asking, project-specific gotchas (like “our get_db() dependency yields async sessions, always await”), and concrete do/don’t rules.
.cursorrules handles Cursor IDE-specific preferences: language, framework, formatter, testing requirements, and behavioral rules.
The pattern is the same across all three: give the AI structured, project-specific context before it writes a single line of code. The developers who do this report dramatically fewer corrections, faster task completion, and code that actually follows their team’s conventions instead of generic best practices from the training data.
Context Engineering for Non-Developers: Yes, This Applies to You
If you’re not a developer, you might think this is purely a coding concern. It’s not. Every knowledge worker who uses AI is doing context engineering. They’re just doing it badly.
A consultant who copies their entire client brief into ChatGPT is doing poor context engineering. A consultant who extracts the 5 most relevant data points, specifies the audience, sets the output format, and includes one example of their firm’s writing style is doing good context engineering.
Practical applications outside of coding:
Content creators: Instead of asking “write me a blog post about X,” context-engineer the AI with your brand voice guidelines, 2-3 examples of your best posts, your target audience profile, and SEO constraints. The output won’t be perfect, but it’ll be 70% there instead of 20%.
Researchers: Build a curated library of highlights from your reading. When you need the AI to synthesize findings, feed it your annotated bibliography, not raw search results. Your highlighted passages carry higher signal than anything the model can retrieve on its own.
Business analysts: Don’t ask the AI to “analyze this data.” Provide the dataset, specify the metrics that matter, define what “good” looks like in your organization, and include one example of a previous analysis your team approved. Context transforms a generic summary into actionable insight.
The Five Mistakes That Kill AI Output Quality
After reviewing the research and testing extensively, these are the context engineering failures I see most often:
1. The “dump everything” approach. Pasting a 200-page document because “the AI has a big context window” is the equivalent of giving a new employee every file in the company drive and saying “figure it out.” Chroma’s research proved that even frontier models degrade with too much context. Curate ruthlessly.
2. Ignoring information placement. Burying critical instructions in the middle of a long prompt. Put the most important context at the beginning and end. The “lost in the middle” effect is real and documented.
3. No examples. A single example of the output you want teaches the model more than three paragraphs of description. Few-shot examples are the most underused context engineering technique, and consistently the most effective.
4. Mixing instructions with data. When your prompt blends behavioral rules, background information, examples, and the actual task into one wall of text, the model struggles to distinguish what’s a rule versus what’s information. Use clear delimiters: XML tags, markdown headers, or labeled sections.
5. Trusting the model’s memory over fresh data. Language models are confidently wrong about facts that have changed since training. If your task involves current data (prices, policies, personnel, features), use tool calls to fetch live information rather than relying on the model’s knowledge.
Where This Is All Heading
Context engineering won’t be a buzzword for long. It’s quickly becoming infrastructure.
Anthropic, Google, and OpenAI are all building context management directly into their platforms, from Claude’s project-level context and memory features to Google’s Gemini context caching to OpenAI’s structured outputs. The Model Context Protocol (MCP), originally created by Anthropic, is becoming the standard for connecting AI to external data sources, adopted by Microsoft, Google, and dozens of startups.
Manning Publications already has a book titled “Context Engineering” in development. Gartner officially flagged the shift from prompt to context engineering in their 2025 analyst briefings. Job postings for “context engineer” are appearing on LinkedIn, and they pay significantly more than prompt engineering roles ever did.
The trajectory is clear: within 12 to 18 months, context engineering will be as fundamental to working with AI as knowing how to use a search engine is to working with the internet. The professionals who master it now will have a head start that compounds. The ones who keep tweaking prompts will wonder why the same model gives their competitor dramatically better results.
Start Here: A 15-Minute Context Engineering Checklist
You don’t need to overhaul your entire AI workflow overnight. Start with these five changes on your very next AI interaction:
1. Define success first. Before you type anything, write one sentence describing what a perfect output looks like. This becomes your first line of context.
2. Provide one example. Show the AI what “good” looks like. A previous report, a writing sample, a code snippet in the right style. One concrete example beats a page of instructions.
3. Separate your context. Use headers or tags to distinguish your rules from your data from your question. Even simple labels like “BACKGROUND:”, “RULES:”, “TASK:” dramatically improve output quality.
4. Put critical context at the top and bottom. Don’t bury your most important constraints in paragraph 7 of a 10-paragraph prompt.
5. Cut what’s not relevant. Before hitting send, remove anything that doesn’t directly serve the task. If a sentence is “nice to know” rather than “need to know,” delete it. Your AI will thank you with better output.
Context engineering isn’t complicated. It’s just deliberate. And in 2026, being deliberate about what your AI sees is the difference between generic output and genuinely useful results.