Disruption in new project creation and project configuration updates

This incident has been resolved.

Things are continuing to look healthy, but out of an abundance of caution, we're going to keep this incident open and continue to monitor.

A fix has been implemented and we are monitoring the results.

We're continuing to scale up load across multiple regions. Managment API responsiveness should be improving as we continue this process.

New project creation, Configuration updates, and certain dashboard actions should now be working, but you may still see occasional slowness.

All existing projects, APIs, and databases continue to operate normally.

The underlying issues are improving, and we are spreading load across globally distributed services. This should result in lower latencies as we continue to spread the load again.

You may still see occasional issues with New project creation, Configuration updates, and Certain dashboard actions until this is completed. All existing projects, APIs, and databases continue to operate normally.

The team is rolling out another change which we hope to reduce error rates, but will result in longer latencies for responses on affected activities.

This will be done gradually to not cause any further disruption.

The team is continuing to work on recovery.

New project creation, Configuration updates, and certain dashboard actions are still affected.

All existing projects, APIs, and databases continue to operate normally.

We have identified the source of the issue and a fix has already been pushed. We are seeing much lower load on the management layer, and are keeping an eye on its effects on recovery.

We are currently investigating an issue affecting some management-layer services, which may impact: New project creation, Configuration updates, and Certain dashboard actions

All existing projects, APIs, and databases continue to operate normally.

Our engineering team is actively working to resolve this. We will provide updates as soon as more information is available.