Prompts that survive a model update

Prompts break at model updates because most of a long prompt is not about the job — it is a pile of corrections for one model's habits. "Do not use bullet points." "Do not start with 'Certainly'." "Keep going, do not stop early." When the model changes, those corrections stop matching the behaviour they were written against, and some of them start actively pushing the output in the wrong direction. The prompt that survives is the one where the job specification is separated from the workarounds, and where you have kept a few fixed test inputs that tell you which of the two broke.

What actually goes wrong

An update rarely makes a prompt fail outright. It shifts the default behaviour: the model becomes more verbose, or less willing to guess, or it starts obeying formatting requests it used to ignore. Your prompt was tuned against the old defaults, so a correction that used to pull the output to the middle now overshoots.

The instructions that age worst are the ones phrased as negations. "Don't be generic", "avoid clichés", "no fluff" describe a failure of the old model rather than the thing you want. Replace them with the positive specification — "every claim must name a file, a number or a date" — and the instruction keeps working regardless of which defaults it meets.

Three layers, kept apart

Write a prompt in three clearly separated blocks, in this order.

The job. What is being produced, for whom, and what it must contain. This layer is about your work, not about any model, and it should still make sense read aloud to a person. It rarely needs to change.

The input and output contract. What you will paste in, and the exact shape coming back — sections, field names, ordering, length limits, what to do with missing information. Also stable, because it is determined by whatever consumes the output downstream.

Model handling. Everything else: tone corrections, the reminder not to apologise, the phrasing that made it stop truncating, the temperature or reasoning setting. This is the disposable layer. Keep it last, keep it short, and label it so that when something changes you know precisely which paragraphs to suspect.

Most unmaintainable prompts are unmaintainable because these three are interleaved across nine hundred words.

Specify the output before the instructions

Write the output shape first — literally, as a filled-in example of a good result. Then write the instructions that would produce it.

A worked example does more than a paragraph of description, and it survives updates better, because a model matching a concrete example has less room to drift than one interpreting an adjective. If the output feeds a spreadsheet or a document template, state the field names and the order, and state what should appear when a field cannot be filled — "unknown", an empty string, an omitted line. Left unspecified, that case is where model updates show up first, usually as an invented value.

Keep a fixed test set

This is the part almost nobody does, and it is the part that turns prompt maintenance from an afternoon of poking into ten minutes of checking.

Save three to five real inputs, unchanged, in a file next to the prompt. Alongside each, save the output you were happy with and one line saying why it was good. Choose them to cover the awkward cases as well as the normal one: the short input, the very long one, the one with missing information, the one that made an earlier version fail.

When anything changes — the model, the prompt, the tool — re-run those inputs and compare against the saved outputs. You are not looking for identical text. You are looking for whether the structure held, whether it invented anything, and whether the failure case still fails safely.

Ten minutes of that tells you more than any amount of reading the release notes, and it is the difference between "it feels worse lately" and "step 2 stopped returning the decision list when the transcript has no clear owner".

Version the prompt like a document

Every prompt file gets a version number, a date, the model it was last tested against, and a two-line note on what changed. Not because it is tidy — because six weeks later you will need to answer "did this get worse, or did I change it?", and without the record the answer is a shrug.

Keep the previous version. Rolling back is often the correct response to a bad update, and it is only available if you did not overwrite the file.

The advice that ages worst

• Role-play preambles. "You are a world-class strategist with twenty years of experience" was worth something on older models and now mostly consumes context and adds a register you did not ask for. Describing the audience and the constraints does the same work with less drift: "the reader is a finance director who will skim this in two minutes".
• Magic phrases. Step-by-step incantations, threats, offers of tips, all-caps insistence. Some of these did shift behaviour on particular models at particular times. None of them are properties of the task, which means every one of them is a maintenance liability.
• Enormous prompt libraries. A file of five hundred prompts ages five hundred times. A handful of prompts you actually run weekly, each with a test set, is worth more and costs less to keep.
• Chained steps with an undocumented handoff. When step 1 feeds step 2, the format passed between them is an interface. Write it down. Otherwise a change in step 1's verbosity silently breaks step 2, and the symptom appears in the wrong place.

The week a model changes

A short procedure, in order: run the test set before changing anything, and keep the outputs. Read them against the saved ones and note what shifted rather than fixing immediately. Then delete the model-handling layer entirely and re-run — often half of it was correcting something the new model no longer does, and removing it fixes more than editing it. Add back only the corrections that prove necessary against the test set. Update the version, the date and the model name.

Resist rewriting the job layer during this. If the job layer needs changing at the same moment a model changed, you will not know which one caused the improvement.

We sell one worked example of all this: a Prompt System: Meeting Notes → Decisions → Follow-ups — a two-step chain with named variables, the handoff format between the steps written out, an input checklist, an output rubric, a list of the failure modes it is known to have, and a version log. It ships as Markdown and Google Docs, it works with any assistant that accepts a pasted prompt, and it makes no claim about what your outputs will be worth.

When to stop prompting

If the task has a deterministic answer — extracting a number, checking a format, looking something up in a list — a script or a formula will do it correctly every time and will not change behaviour next quarter. Prompt maintenance is a recurring cost, and it is only worth paying where the task genuinely needs judgement about language.

The other stopping point is the prompt that has been rewritten five times and still fails one of the test cases. That usually means the job is underspecified rather than the prompt underwritten, and the fix is upstream: a better input, a narrower scope, or splitting one prompt into two.

Related

• Structuring an SOP people actually follow
• Client tracking: when a spreadsheet is enough
• Prompt systems in the shop: AI Prompt Systems