The Atlassian April 2022 Outage: How a Maintenance Script Deleted 883 Customer Sites

A misconfigured maintenance script permanently deleted 883 Atlassian cloud sites; restoring them took up to 14 days due to brittle restore tooling and manual processes.

Infrastructure · intermediate · 12 min read · 2084 words · By Sandip Das · Published

Public postmortem: based on what the organisation published. Sources are listed below.

What this covers

  1. Background
  2. What Happened
  3. Root Cause
  4. Detection and Response
  5. Impact
  6. The Fix
  7. Lessons for Your Team
  8. Apply It Yourself

Key numbers

MetricBeforeAfter
Customer sites deleted883 sites active883 sites permanently deleted by maintenance script
Time to full restorationExpected: hours (standard RTO)Actual: up to 14 days for some customers
Percentage of cloud customer base affected0%~0.18% of Atlassian cloud customers

Key components

  • Internal Maintenance CLI — The operator-facing tool that accepted App IDs but resolved and acted on Site IDs due to a type-handling bug, triggering the mass deletion.
  • ID Resolution Pipeline — The internal layer that translated input identifiers to entity references; the mismatch between App ID and Site ID namespaces was the proximate cause of the incident.
  • Tenant Database — Stored the authoritative record of which customer sites existed; once records were deleted here, the sites were inaccessible even if underlying data remained in backups.
  • Backup and Restore Tooling — Backups existed for all 883 sites, but the restore tooling was designed for individual restores and had never been exercised at bulk scale, making recovery take up to 14 days.
  • Dry-Run Safeguard (absent) — The maintenance script lacked a mandatory dry-run step that would have surfaced the ID mismatch before any destructive action was taken.
  • Deletion Rate Alerting (absent) — No alert fired when 883 sites were deleted in a single operation because the deletion was technically successful; anomaly-based alerting on deletion rates was not in place.

Background

Atlassian provides cloud-hosted versions of Jira Software, Jira Service Management, Confluence, Jira Work Management, Opsgenie, and other products to hundreds of thousands of organisations worldwide. By April 2022 the company had migrated a significant portion of its customer base to a unified cloud platform, meaning a single tenant-management layer controls access to multiple products for each customer site.

The affected customers ranged from small teams to large enterprises. Many of them depended on Jira and Confluence as the operational backbone for engineering, IT, and support workflows — making even a few hours of downtime painful, and days of downtime a serious business risk. The Atlassian Post-Incident Review notes that approximately 883 sites were permanently deleted, representing roughly 0.18% of the total Atlassian cloud customer base.

---

The rest of this case study — the trade-offs, the scaling numbers and the build prompt — is available to LearnXops members. Create an account or see membership.

Sources

Topics: incident-response, infrastructure, data-loss, postmortem, cloud, atlassian

Related case studies

Browse all case studies