On Demand Sandbox Performance Degradation

We continue to investigate slow performance when using On-Demand Sandboxes (ODS). A number of actions have been taken, and the focus is now on increasing the CPU limit as a workaround.
This issue is now believed to also affect customers on US01 and AP01. We have updated the trust posting to reflect this.

We have identified a recent release as a potential trigger, and a rollback has been initiated. Once the rollback is complete, a restart will be initiated. If we have more information, we’ll update you in 30 minutes or sooner.

We are exploring the options to increase the CPU limit as a workaround and continue to investigate the underlying performance issues.

We’ll update you in 30 minutes or sooner if we have more information.

We have completed the Redis cluster restart; however, this did not resolve the issue. We are investigating why restarting the Redis cluster was unsuccessful and why the connection between the application and Redis continues to cause performance issues.

We’ll update you in 30 minutes or sooner if we have more information.

The Redis cluster restart is currently in progress, and we are actively monitoring the service to see if performance improves.

We’ll update you in 30 minutes or sooner if we have more information.

We are preparing to restart the Redis cluster to see if this helps resolve the ongoing issue.

We’ll provide you with an update in 30 minutes or sooner if we have more information.

The investigation remains focused on the Redis service, where error patterns continue to be observed. We are working with the relevant teams to determine the next steps and are working to bring affected customers out of impact as quickly as possible.

We’ll provide you with an update in 30 minutes or sooner if we have more information.

We are investigating an issue where customers may experience slowness using On-Demand Sandboxes (ODS). We have identified connectivity issues with the Redis service and are investigating further to determine the trigger for this issue. We have engaged the relevant team to assist with the investigation.

We also determined that the start time of impact is different from what we initially understood, and have now revised the start time of the Trust post.

We’ll provide you with an update in 30 minutes or sooner if we have more information.