Performance Degradation

After extensive testing, the Technology team now plans to deploy the release on September 19. We will provide the next update when the timing of the release is confirmed.

The Salesforce Technology team continues to run tests for the emergency release that is scheduled for tomorrow, September 18, 2023.

We will provide an update at approximately 22:30 UTC on September 18, 2023, or sooner if additional information becomes available.

The Salesforce Technology team's focus remains on triaging and exploring multiple potential resolution paths. Further investigation determines an issue towards the older version of the Application Program Interface (API) release. The Technology team is working on a permanent fix and it will be deployed through an Emergency Release after validation and testing. This process may take several hours to complete.
We’ll provide an update at approximately 22:30 UTC on September 17, 2023 or sooner if additional information becomes available.

The Salesforce Technology team identified the potential cause and believes that an issue with Application Program Interface (API) is the likely trigger of the incident. The Technology team continues to validate the findings and explore paths to resolve the issue.
We continue to reach out to the affected customers.
We’ll provide an update at approximately 02:00 UTC on September 17, 2023 or sooner if additional information becomes available.

The team continues to analyze the logs and explore multiple paths to mitigate the issue. All the steps taken so far have not yielded the desired results. We have been working with affected customers to compile diagnostic information in real time. That outreach will continue with additional customers.
We will provide an update at approximately 18:00 UTC or sooner if additional information becomes available.

In an effort to resolve the issue, the team restarted the Cache Cluster to rule out any potential Cache issues however that action also proved unsuccessful in mitigating the issue.
We continue to explore alternative resolution paths at this time.
We will provide an update in 4 hours or sooner if additional information becomes available.

The team remains engaged and continues to assess additional paths to resolution. After further investigation, the Technology team confirmed that a subset of customers continue to experience intermittent page load communication errors. The intermittent nature of the issue is resulting in additional time required to capture and analyze the logs.

We will provide an update in 4 hours or sooner if additional information becomes available.

The Technology team performed a database failover, but that action did not resolve the issue. They continue exploring other paths to restore services.

We will provide another update in two hours, or sooner if additional information becomes available.

The Technology team is no longer observing any errors and is currently engaged in monitoring the health of the affected database. Additional investigation is currently in progress. We will provide an update in 2 hours or sooner if additional information becomes available.

The Technology team continues to segregate the triggers of the issue and analyze the captured logs. The intermittent nature of the issue is resulting in additional time required to capture and analyze the logs.

We will provide an update in 2 hours or sooner if additional information becomes available.

The Technology team continues to investigate the issue. Due to the intermittent nature of the issue, additional time is required to capture the logs and analyze them.
We’ll provide an update in 4 hours, or sooner if additional information becomes available.

The Technology team is exploring multiple paths to isolate the underlying cause of the issue and implement a fix to restore the services. Additional logs relating to network connectivity are being captured as part of the investigation.

We will provide an update in 60 minutes or sooner if additional information becomes available.

The team continues to debug the issue and analyze additional diagnostic and error logs to determine the best path to resolution.

We’ll provide an update in 60 minutes or sooner if additional information becomes available.

We remain engaged and continue to investigate the issue, its triggers, and a path to resolution.
We’ll provide an update in 60 minutes or sooner if additional information becomes available.

At 02:55 UTC, on September 14, 2023, the Salesforce Technology team became aware of an intermittent performance degradation affecting a subset of customers on this instance. During this performance degradation, a subset of customers will observe intermittent page load communication errors and be unable to complete requests when using the Lightning and Classic functionality of the application.
In an effort to resolve the issue, the team redeployed a recent Release however that did not help in resolving the issue. The team continues to investigate the issue and explore multiple paths to resolution.
We’ll provide an update in 30 minutes or sooner if additional information becomes available.

We’re investigating the cause of the issue. We will provide an update in 30 minutes or sooner if we have more information.