Series: Overview | Process & Methodology | Engineering Thinking | Meta-Skills (you are here) | Maintenance & Handover | Glossary
Contents
15. Post-Incident Learning
The principle: When something goes wrong, capture the mechanism that allowed it — not just the fix. “Only get something wrong once.”
The SWE vocabulary: A “postmortem” (or “post-incident review”) is a structured analysis of what went wrong after an incident. The key discipline is separating what happened (the timeline) from what allowed it to happen (the mechanism). SWEs also distinguish between the root cause (the actual source) and contributing factors (things that made it worse). “Blameless postmortems” focus on systems, not people — the question isn’t “who screwed up?” but “what about our process allowed this to happen?” A “retrospective” is a lighter version: a regular team check-in on what’s working and what isn’t.
Why it matters for AI orchestration: You’ll make the same mistakes repeatedly unless you build a feedback loop. Cross-session continuity depends on what the tool saves and retrieves; a correction in chat may not be available in a later session. And the mistakes that matter most aren’t the ones where code crashed — those are obvious. The expensive mistakes are process mistakes: spending 3 hours on something that should have been scoped to 30 minutes, or trusting AI output that should have been verified, or not checkpointing before a risky change.
The failure mode it prevents: The “Groundhog Day” problem. You hit the same frustration every few weeks — the AI clobbers a file you cared about, or a project spirals out of scope, or you deploy something broken. Each time you fix it in the moment but don’t change anything about how you work. The same situation triggers the same mistake.
The method: After something goes wrong (or surprisingly right), spend 2 minutes answering:
- What happened? (The facts, not the narrative.)
- What allowed it to happen? (The mechanism — what check was missing, what assumption was wrong?)
- What would prevent it next time? (A specific, checkable change — not “be more careful.”)
The third answer is the valuable part. “Be more careful” is not a mechanism — it’s a wish. “Always run the script on test data before real data” is a mechanism. “Add a dry-run flag to every destructive script” is even better — it changes the system, not just your intentions.
Carry the lesson forward: Put agreed, reusable lessons in existing project instructions or runbooks. In a later session, check that the lesson was retrieved and that behaviour changed. Review lessons before adopting them; mistaken rules can spread too.
Example: You ask the AI to reorganise your project’s file structure. It moves files around but breaks all the internal links between them. You fix the links manually — takes an hour. Postmortem: What happened? The AI moved files without updating references. What allowed it? You didn’t tell it about the internal links, and you didn’t check before approving the changes. What would prevent it? Before any file-moving operation, prompt: “List all references to these files across the project and update them too.” And checkpoint before the operation so you can rewind if it misses some.
Try this: Next time something goes wrong in an AI session, write down one sentence: “This happened because ___.” If the blank is “I wasn’t careful enough,” push harder. What specific check or step would have caught it?
16. Context Management
The principle: Keep relevant information available. Diagnose missed constraints or repeated failures before deciding to continue, compact or restart.
The SWE vocabulary: A context window limits how much information a model can receive at once. Tokens measure that input; their relationship to words varies with language and content. Check the tool’s documented limits and what it does when they are reached. This guide establishes no universal fill percentage or elapsed-time threshold for restarting.
In practice:
- Keep unrelated work in separate conversations.
- Supply the relevant files and evidence; avoid broad, unbounded exploration.
- When answers deteriorate, check the requirements, evidence and approach as well as the tool’s context or compaction indicators.
- Preserve failed approaches and what ruled them out so a fresh session does not repeat them.
Try this: When corrections keep failing, identify the mistaken assumption or missing evidence before continuing, compacting or restarting. Preserve the goal, decisions, failed approaches, checks and remaining work in the handover. On resuming, read the relevant files and confirm their state. Reuse the existing maintenance and handover guide for durable work.
17. Prompt-as-Spec
The principle: The best prompts function like engineering specifications — they define what success looks like, provide relevant context, and constrain the solution space. The difference between a mediocre result and an excellent one is almost always the quality of the prompt, not the capability of the AI.
The SWE vocabulary: A “specification” (spec) is a document that describes exactly what a piece of software should do — its inputs, outputs, constraints, and success criteria. A “requirements document” describes what needs to be built from the user’s perspective. A “design document” describes how it will be built from the engineer’s perspective. SWEs spend significant time on specs before writing code because a clear spec prevents rework. The same principle applies to prompts.
“Constraints” are boundaries on the solution. In engineering: “must run on a Raspberry Pi” or “must handle 10,000 users” or “must work offline.” In prompting: “use Python, not JavaScript” or “keep it under 50 lines” or “don’t use any external libraries.” Constraints are powerful because they narrow the solution space — the AI has fewer wrong directions to wander in.
Why it matters for AI orchestration: A vague prompt gets a vague result. “Make me a website” is a feature request — it conveys intent but not enough information to produce what you want. “Build a single-page site with my name, a bio paragraph, and links to three projects. Use plain HTML and CSS, no frameworks. Make it look clean and minimal — think Hacker News, not Apple” is a spec — it conveys intent plus constraints plus examples plus aesthetic direction. The AI can execute the second prompt on the first try. The first prompt will require 5 rounds of “no, not like that.”
The spectrum from bad to good:
- Feature request: “Build me a dashboard.” (What dashboard? For what data? What should it show?)
- Wish: “Build me a dashboard that shows my sales data.” (Getting warmer — but which metrics? What time range? What data source?)
- Brief: “Build a dashboard that reads from sales.csv and shows monthly revenue as a bar chart and top 10 products as a table.” (This is workable.)
- Spec: “Build a dashboard: (1) Read sales.csv (columns: date, product, amount, quantity). (2) Show a bar chart of monthly revenue (sum of amount, grouped by month). (3) Show a table of top 10 products by total revenue. (4) Use Python with matplotlib for the chart and print the table to stdout. (5) Handle missing or malformed rows by skipping them and printing a count of skipped rows at the end.” (This gets built correctly on the first try.)
The “let AI interview you” pattern: For larger features, don’t try to write the perfect spec yourself. Instead, describe what you want at a high level and ask the AI to interview you: “I want to build X. Before you start, ask me clarifying questions about requirements, constraints, and edge cases.” The AI will surface questions you wouldn’t have thought to answer — and the answers become the spec.
Example: Instead of “write a script that backs up my photos,” try: “Write a Python script that: (1) takes a source directory and destination directory as command-line arguments, (2) copies all .jpg and .raw files from source to destination, preserving the folder structure, (3) skips files that already exist in the destination (compare by filename and size), (4) prints a summary at the end: X files copied, Y files skipped, Z errors. If a file can’t be copied, log the error and continue — don’t crash.”
Try this: Before your next prompt, add one constraint and one success criterion that you wouldn’t have included otherwise. “Use Python 3, no external libraries” (constraint). “I’ll know this works when it processes my test folder of 50 photos in under 10 seconds” (success criterion). Notice how the result improves.
Previous: Engineering Thinking — technical debt, separation of concerns, state, debugging, AI reliability
Next: Maintenance & Handover — preserving context, scheduling revalidation and knowing when to refactor