A GitHub Action that rebuilds a pull request's description on every push looks idempotent — same inputs, same output. It isn't, if a human has edited that output in between runs. Two separate incidents, same root cause, both found live in production: the fix is to read before you overwrite, not to trust that regeneration is harmless.
The Problem
We built a small release pipeline: a GitHub Action keeps a single "ready to release" pull request open automatically, regenerating its description from scratch on every push to the integration branch — the commit list, a checklist of pre-merge steps, some guidance text. A second Action gates the actual merge, checking that the checklist in that PR's body is fully checked off.
This looked clean. Fully declarative, no state to manage — every run just recomputes the body from the current repo state and writes it. Idempotent, in the sense that matters to most people: same inputs, same output.
Except "regenerate from scratch" and "idempotent" aren't the same property, and the gap between them cost us twice.
Incident one. A maintainer checked all four boxes in the release checklist — deployed to test, config verified, the works — and the merge gate passed. Then someone else pushed an unrelated commit to the integration branch, as happens constantly on a shared branch. The PR-maintaining Action fired, rebuilt the body from the current repo state, and wrote a fresh checklist — unchecked, because the template always starts unchecked. The merge gate, which had just passed, failed again. The maintainer's verification was real; the automation just didn't know it had happened, because it never looked at what was already there before overwriting it.
Incident two. Months earlier, we'd added a title-convention check — a PR's title has to match a pattern, because it becomes the commit message on squash-merge and downstream tooling parses it. A release PR got created before that convention existed, with a plain-English title. Later, the same release got re-cut after more work landed — the Action regenerated the PR's body with the new content, correctly. It never touched the title, because nobody had written that code path. The PR sat there for days with a title that had been wrong since before the rule even existed, failing a check that had no way to know the title was stale rather than freshly violated.
Different symptom, same shape: a process assumes "regenerate" is a safe default action, and never asks whether something it's about to overwrite already carries information nothing else captured.
The Fix
The fix for both was the same move: before writing the new state, read the old state and carry forward whatever the new state doesn't actually know how to decide for itself.
For the checklist: before rebuilding the PR body, fetch the current body first, extract which checklist items are already checked, and write the freshly-regenerated body with that same checked state applied — positionally, since the checklist items themselves are a fixed, known set. The commit list and guidance text still regenerate completely clean every time; only the human-decided checkbox state survives across rebuilds.
For the title: once we noticed the Action had two code paths — "open a new PR" and "refresh an existing one" — and only the first one set a title, the fix was one line: pass the title through on both paths. The bug wasn't subtle logic; it was an asymmetry between two branches that were supposed to produce equivalent results and quietly didn't.
Neither fix required abandoning the "regenerate from scratch" design — that part was genuinely fine, and simpler than any incremental-patching alternative would have been. The fix was narrower: identify exactly which fields in the regenerated output represent a human decision rather than a fact derivable from the current repo state, and treat only those fields as sticky.
What It Caught — and Why Review Missed It
Neither of these showed up in design review, because on paper the design is correct: a pure function from repo state to PR body is a reasonable thing to want. The bug only exists at the intersection of two true things that don't individually look suspicious — "this regenerates every push" and "a human occasionally writes into the same surface it regenerates." Review each sentence in isolation and nothing's wrong. Only running the sequence — check a box, push again, watch the box reset — actually surfaces it.
That's the general shape worth remembering: automation that writes into a surface a human also writes into needs an explicit merge strategy for that surface, even when the automation's own logic is otherwise stateless and clean. "It's just regenerated from source" is true and irrelevant if the source doesn't include the human's last action.
What It Provided
- ▸A reusable rule, not just two patches. Any field a human can edit on the same artifact an Action rewrites needs its old value read back before the new value is written — checkboxes, titles, labels, assignees, anything.
- ▸A cheap test for the next one. Before trusting a "regenerate from scratch" design, ask: can a human write into this same surface between two runs? If yes, the design needs a merge step, independent of how clean the regeneration logic looks.
- ▸Confidence the merge gate was actually telling the truth. Once the checklist stopped silently resetting, a red check meant "someone hasn't verified this yet" again, instead of sometimes meaning "the automation forgot."
Future Implications
This category of bug is going to get more common, not less, as more of the glue between tools gets written as "regenerate this artifact from current state" automation — it's a genuinely good default, and AI-assisted code generation makes it trivially easy to produce more of it, fast. The failure mode doesn't show up in the generated code's logic; it shows up in the interaction between that code and whatever else is allowed to touch the same artifact. Before shipping anything that rewrites a shared surface — a PR, a document, a config file — the question worth asking isn't "is this logic correct," it's "what else writes here, and what happens to that when I overwrite it."