Incident with Issues, API Requests and Git Operations

This incident has been resolved.

Issues is operating normally.

We experienced an incident causing missing Project item data. We have restored all data from before the incident, up to 19 Sep 20:36 UTC. Project data modified between 20:36 UTC and 20 Sep 00:06 UTC was partially restored.

We will publish a root cause analysis for this issue by Sep 29.

Projects data has been restored to the state prior to the incident (19 Sep 20:57 UTC). We are continuing to restore the last of Project metadata updated since 20:57 UTC.

Projects data has been restored to the state prior to the incident (19 Sep 20:57 UTC). We are continuing to restore the last of Project metadata updated since 20:57 UTC.

While rolling out an update to Projects, Project data was affected and needed to be restored. While restoring data, we experienced a replication interruption which affected Git Operations, Issues and APIs. We restarted replication, and restored Projects data to its state prior to the rollout, and continue working to restore the last of Project metadata updated since 19 Sep 20:57 UTC.

Issues is experiencing degraded performance. We are continuing to investigate.

We've restored access to a majority of Project data. A small number of customers are still affected, and we continue to work on full restoration.

We continue restoring access to Project data. We do not have an estimated time to full restoration at this time, and are working to provide that.

We continue working on restoring access to Projects data, along with an estimated time to full restoration.

API Requests is operating normally.

Git Operations is operating normally.

Git operations, API requests, and Issues are recovering after restoring data replication, and we continue working on restoring access to Projects data.

Git Operations is experiencing degraded performance. We are continuing to investigate.

* We are actively working on restoring access to Projects data.
* Issues and APIs are experiencing degraded performance. We are working on mitigating the issue.

API Requests is experiencing degraded performance. We are continuing to investigate.

We have identified the issue and are actively working to restore missing project items

Issues is experiencing degraded availability. We are continuing to investigate.

We are currently investigating an issue with project items not loading

We are currently investigating this issue.