When AI contributes to a pull request that causes a failure, the postmortem should not end with “the model wrote it.” That sentence names a circumstance, but it does not explain which requirement was unclear, which review missed the risk, which test was absent, or which signal arrived too late.

A useful postmortem reconstructs four things: the change that shipped, the review and delivery context, the runtime impact, and the control that will change. Every claim gets evidence or an explicit label: observed, inferred, hypothesized, or unknown. Every action has an owner, an acceptance test, and a due date. This complements the engineering decision record for agent-written code, which covers decisions made before or during implementation.

Diagram connects change, review, and runtime impact to a control action in an AI-assisted code postmortem.

Short answer

  • Reconstruct the change from the commit, pull request, and agent trace when that trace exists.
  • Separate review, CI, approval, and deployment from the account of what happened at runtime.
  • Label each claim as observed, inferred, hypothesized, or unknown.
  • Replace “review more carefully” with a verifiable control, an owner, and an acceptance check.

What should the postmortem answer?

Start with an impact sentence that someone outside the incident can verify: what changed, what behavior was observed, who was affected, and how recovery ended. Do not start by assigning intent to the model or to a human author.

Then answer these questions in order:

  1. What requirement, decision, or assumption was the change meant to satisfy?
  2. What was produced, and which tools participated?
  3. What did human review, automation, and approval actually check?
  4. What was the first observable signal in production or delivery?
  5. Which control was missing, weak, or applied too late?
  6. How will we know the corrective action works?

This follows a blameless analysis. The Google SRE postmortem culture guide recommends identifying contributing causes without indicting a person or team. GitHub's guidance for reviewing AI-generated code also places human expertise, tests, static analysis, and context checks alongside automation.

Build three evidence lanes

An AI-assisted change leaves evidence in different places. Do not blend all events into one story before deciding what each source can prove.

Lane Question Possible evidence Limit
Change What entered the repository and where did it come from? commit, diff, pull request, instruction, changed files, agent trace A trace can show a tool call without proving it caused the defect.
Review and delivery What was reviewed, tested, and approved? comments, checks, CI artifacts, approval, deployment, configuration A green check proves what it ran, not everything the system needed.
Runtime What happened after delivery? redacted log, metric, alert, ticket, rollback, state query A nearby timestamp does not prove causality by itself.

If your harness records traces, use them to connect generations, tool calls, handoffs, and guardrails. The Agents SDK tracing documentation documents traces and spans for these events and warns that captured data may be sensitive. Store references, hashes, or redacted excerpts in the postmortem. Do not copy prompts, tokens, secrets, or an entire transcript just because it exists.

For each source, record its identifier and the claim it supports. ci-184 may prove that a suite ran against a particular commit. It does not prove that an unrepresented business rule was considered.

Separate fact, inference, hypothesis, and unknown

Use explicit labels so a convenient hypothesis does not become the official cause while the document is being edited.

Label Meaning Example
observed Direct evidence is available and referenced. “The checkout-error-rate alert opened at 14:07.”
inferred The conclusion combines evidence but is not directly observed. “The change likely affected checkout because the error followed deployment and appeared only on that path.”
hypothesized A plausible explanation still needs a sufficient test. “The missing currency case may have left validation incomplete.”
unknown The necessary source is missing, too redacted, or inconclusive. “We do not know which instruction the agent received in this attempt.”

A CI timestamp, a review comment, and a production log can form a coherent timeline. Still, distinguish “happened after” from “caused.” If the cause remains hypothesized, an action can be a test that confirms or rejects it.

Turn the timeline into a matrix

A short table is often easier to audit than several paragraphs. Use stable IDs so one claim can point to more than one source.

14:02  C1  commit 8f2a opened the PR              [observed]
14:05  R1  two CI checks finished green            [observed]
14:09  D1  deployment of 8f2a finished            [observed]
14:17  P1  checkout errors increased              [observed]
14:31  A1  rollback to 7c1d finished              [observed]

Then write a claim matrix:

ID Claim Status Evidence Next step
C1 The pull request changed currency normalization. observed diff 8f2a None, established fact.
C2 The missing-currency case had no test. inferred suite ci-184, module coverage Add an acceptance test.
C3 The agent received an ambiguous instruction. unknown redacted trace without the prompt Improve safe retention or do not claim it.

This matrix lets a reviewer challenge one claim without reopening the whole narrative. The Atlassian postmortem handbook offers a similar practical combination of timeline, causes, owners, and actions. Treat it as process guidance, not proof that every incident needs one fixed form.

Validate the document before publishing

The format can have a small deterministic check. The fixture below does not analyze agent code or prove a production cause. It only enforces two document invariants: an observed claim points to evidence, and every action has an owner and acceptance criteria.

import assert from "node:assert/strict";

const report = {
  claims: [
    {
      id: "C1",
      status: "observed",
      evidence: ["diff-8f2a", "ci-184"],
    },
    {
      id: "C2",
      status: "hypothesized",
      evidence: [],
    },
  ],
  actions: [
    {
      id: "A1",
      control: "Add the missing currency case to the merge test",
      owner: "checkout-team",
      acceptance: "ci-201 fails before the fix and passes after it",
      due: "2026-12-01",
    },
  ],
};

function validatePostmortem(value) {
  assert.ok(value.claims.length > 0, "postmortem needs claims");

  for (const claim of value.claims) {
    if (claim.status === "observed") {
      assert.ok(claim.evidence.length > 0, `${claim.id} needs evidence`);
    }
  }

  for (const action of value.actions) {
    assert.ok(action.owner, `${action.id} needs an owner`);
    assert.ok(action.acceptance, `${action.id} needs acceptance criteria`);
    assert.ok(action.due, `${action.id} needs a due date`);
  }
}

validatePostmortem(report);
console.log("postmortem evidence checks passed");

Save it as postmortem-check.mjs and run node postmortem-check.mjs. In a real system, connect the validation to the postmortem repository or review process. Do not make the script pretend an action is complete merely because its fields exist.

Turn the cause into a control

A corrective action should change the system that allowed the failure. For example:

Missed control Verifiable action
The requirement had a gap Add an acceptance case that fails on the old behavior and passes on the new one.
Review lacked operational context Attach a redacted summary of the change, risk, and executed checks to the PR.
CI missed a boundary Add a regression test and make it required for the affected path.
Permissions were too broad Restrict the tool or environment and verify the policy in CI.
The first signal arrived late Add an alert with a threshold, owner, and firing test.

“Review more carefully” has no completion criterion. The coding-agent observability post helps connect the operational trail to the PR, while the human-approval test for AI agents covers a human barrier as testable behavior.

What does not belong in the document?

Do not publish a full agent transcript, secrets, private prompts, tokens, personal data, or a story that assigns intent without evidence. Do not turn a community discussion into technical proof. Keep what enables learning and remove what increases exposure without improving the explanation.

Do not claim that AI was the cause merely because it participated in the change. Compare the specification, diff, checks, and observed behavior. If the agent's contribution cannot be safely reconstructed, mark it unknown and make safe observability a separate action.

Method and limits

Samuel Fajreldines wrote this article as a synthesis of public SRE, AI code-review, agent tracing, and postmortem guidance, combined with the local fixture above. The fixture was run with Node.js, but it is not a production incident, benchmark, or measure of model effectiveness. AI assistance helped organize research and wording; it did not run a production test or replace the author's responsibility. The author's tool is RemoteCode.

Adapt the result to your access controls, log retention, and privacy requirements. If an agent trace contains sensitive data, omitting a copy from the postmortem may be the right decision, provided a secure and auditable reference remains.

Final checklist

  • [ ] The timeline points to commits, checks, deployments, and observable signals.
  • [ ] Change, review, and runtime are separate lanes.
  • [ ] Each claim is labeled observed, inferred, hypothesized, or unknown.
  • [ ] The action changes a control and has an owner, acceptance test, and due date.
  • [ ] The document exposes no secrets, personal data, or unnecessary transcript.
  • [ ] The cause stays proportional to what the evidence supports.