Postmortem action items: why they never get done

Why postmortem fixes lose to feature work, and how to write, own and rank action items so incidents stop repeating.

7 min read

Two rows of rounded blocks with a check mark on the last one, like a pipeline that ran to the end and a step still waiting

Postmortem action items stall because writing the postmortem closes the work in everyone’s head, while the fix lands in a backlog with no owner, no date and no priority over feature work. Repeats come from that gap. Fix it by giving each preventive item one owner, a deadline and a review that checks it before the next incident does.

Google’s SRE book defines a postmortem as a written record that ends with follow-up actions to prevent the incident from coming back. So the document is supposed to produce work. That work is the part that gets dropped. If nobody on your team owns the reliability backlog between incidents, that is exactly the gap SRE and observability work is meant to close.

Why do postmortem action items never get done?

Because nothing forces them to compete with the roadmap on equal terms. The incident had urgency. Users were affected, someone got paged, and the clock was visible to everyone. The action items inherit none of that. They land in a backlog next to feature tickets, and every feature ticket has a product manager defending it. The action item has nobody.

Google’s SRE book is blunt about the result. Without a formal process for learning from incidents, they may recur indefinitely. The key word is process. A postmortem document on its own runs nothing. It sits in a wiki while the same contributing factor waits for the next busy Friday.

Four patterns are enough to stall an action item, and none of them needs bad intent.

The first is vagueness. An item like improve monitoring has no finish line, so it never finishes. The second is blame in disguise. AWS lists it as an anti-pattern to identify only human error and then provide no training or automation to prevent it. Be more careful is a wish. Nobody can close it.

The third is team ownership. When an item belongs to the platform team, it belongs to whoever has a spare afternoon. The fourth is the missing review. If nobody looks at open items until the next page, the next page becomes the review.

What makes a postmortem action item worth doing?

A good action item changes the system so the same contributing factor cannot cause the same impact again. It is also written so one person can finish it and someone else can verify it.

The clearest public example is in Google’s SRE workbook. A routine rack decommission ran into a bug in the maintenance automation. Part of the decommission had failed, so the team retried it. On the retry the machine list was empty, and the API treated an empty list as no filter at all. Within minutes, the disks of every satellite machine worldwide were erased. Good capacity planning meant very few users noticed during the two days of reinstalls.

The follow-up was unglamorous. The team spent several weeks auditing the automation and adding sanity checks so the decommission workflow became idempotent. Three years later, a similar incident drained a number of satellites. Google credits the action items from the original postmortem with dramatically reducing what happened the second time.

That is what a finished action item buys you. The payoff can arrive years after everyone has forgotten the ticket. A fix that lives in the automation does not depend on anyone remembering the incident.

AWS adds a check worth running on every postmortem. Ask why existing testing did not catch the issue, and add a test for the case if one does not exist. That turns one incident into a permanent guard.

Here is how I would rewrite weak items like these before the postmortem is published.

Weak action item Stronger version Why it holds
Be more careful when retrying automation Make the workflow idempotent and reject an empty target list The guard lives in code and survives turnover
Improve monitoring Alert on the symptom users saw, so on-call finds it before a customer does Google treats a monitoring failure that forced manual discovery as a postmortem trigger
Add more tests Add one test that reproduces this incident’s exact input AWS asks why existing testing missed the issue
Share the learnings Add the pattern to the pre-deployment checklist other teams use AWS recommends turning patterns into developer guides and checklists
Investigate root cause further Name the remaining unknown, one owner and a date to report back Open questions get an end date instead of living forever

Who should own postmortem action items?

One named person per item, plus one person who owns the follow-up process across all of them. Those are different jobs.

AWS’s Well-Architected guidance says to assign clear ownership for the post-incident analysis process. It also says to designate a team or individual to oversee investigations and follow-ups. Assigning each ticket is the easy half. If you only do that half, items rot quietly. Every owner believes their own item is fine, and nobody sees the pile.

The follow-up owner does not do the work. They keep the list honest. I would give that person two things. The first is a standing slot in the weekly planning meeting. The second is the authority to escalate any preventive item that slips its date twice.

If your org is still sorting out where SRE ends and platform begins, the split is covered in platform engineering vs DevOps vs SRE. The follow-up owner can exist before that debate ends. I would not wait for it.

Blamelessness matters here too. Google’s SRE book says a blameless postmortem assumes everyone acted with good intentions on the information they had. If action items read like punishment, people will negotiate them down to nothing. Write them against the system, never against a person.

How do you prioritize postmortem action items against feature work?

Rank them by the cost of the next occurrence, and decide in the postmortem review with the roadmap owner in the room. If the person who sets feature priority never sees the action items, the items lose by default.

Not every item deserves to be built. Google’s SRE book notes that postmortems carry a real cost in time and effort, so the team chooses deliberately when to write one. Apply the same discipline to the items. Each one should leave the review with one of two outcomes. Either it gets an owner, a date and a place in the plan, or it gets closed as accepted risk with a written reason. Both are honest. An item left open for two quarters is neither.

I would sort candidates with one question per item. If this contributing factor fires again next month, what does it cost in user impact and on-call time? Items that prevent data loss or user-visible downtime go first. Items that only shorten detection come next. Google lists data loss of any kind and user-visible downtime among its standard postmortem triggers. That is a decent hint about where to put the weight.

Then protect a small, fixed slice of each planning cycle for the items that survive. Agree on its size before the next incident, while nobody is emotional about it. A slice negotiated during an outage week gets traded away the week after.

How can you tell if your postmortems are preventing repeats?

Check whether new incidents share a contributing factor with an older postmortem that still has open items. If they do, you have your answer for leadership.

AWS names the failure mode directly. Without a mechanism to keep lessons learned, teams end up with repeat incidents that share the same or similar root cause.

A few plain numbers will tell you more than a dashboard redesign. Count last quarter’s incidents that match a prior postmortem’s contributing factor. Note the age of the oldest open preventive item. Compare items closed as done against items closed as accepted risk. If accepted risk is winning, say so out loud in the review.

If you already track DORA metrics, keep them. They show that something is wrong. They do not tell you which postmortem you ignored.

Also widen the net. AWS points out that an incident does not require an outage. A near miss counts. Near misses are cheap repeats, so write them up briefly and check them against open items the same way.

Finally, share the results beyond the team that got paged. AWS notes that sharing findings lets other workloads mitigate the risk when they carry the same contributing factors. A fix that lives in one team’s backlog protects one team.

Start with the last repeat incident

Take the incident leadership asked about most recently. Open the postmortems from the past year and search for the same contributing factor. Give it 30 minutes. If you find an earlier postmortem with an open preventive item, you have the honest answer to why it happened again. Bring that item to your next planning meeting with one owner’s name and a date next to it. Then ask the roadmap owner to place it before the meeting ends.

Sources

  • postmortems
  • incident management
  • SRE
  • reliability

Read this post in SpanishRead this post in Portuguese

New posts by email

Notes on rates, platform engineering, DevOps, SRE, QA and AI engineering, with the prices and figures from our own work.

DX Clouditive LLC (7901 4th St N, Ste 300, St Petersburg, FL 33702, USA, info@dxclouditive.com) will email you new posts and a weekly digest once you confirm. You can withdraw consent anytime with the unsubscribe link in any email. Read our privacy policy. Unsubscribe anytime from any email.

Keep reading

All posts

Tell us your case.

We reply to every request within 1 business day. We sign an NDA before the call if you ask.