Spreedly API Errors

After a period of monitoring, we have not seen any recurrence of this issue. We have confirmed that the lagging data has been resolved and all systems are stabilized and functioning normally with current data.
The incident is being considered resolved.

We are still investigating to understand the specific causes of the incident and any residual impact. A post incident review will be published.

We apologize for the impact this caused to affected customers.

After a period of monitoring we have confirmed that our database environment remains stabilized and there are no errors impacting customer transactions.

We are working on addressing the lagging data issue with Dashboard and secondary services. We estimate transaction data to be caught up and current in about 3 hours from now.

We will continue to update with any changes.

We experienced a database capacity limit that resulted in intermittent 500s and lagging data in secondary systems. Once identified, we immediately began working with our service provider on a resolution. We’ve been implementing that fix and stabilized the database such that customers should no longer be experiencing 500s.

However, Dashboard and secondary systems are still lagging as we continue to reduce indexes and ensure the system is fully stabilized. We’re prioritizing system stability to support many year-end customer processes and will address the lagging data issue once we’re confident there are no risks to transaction processing.

Until then, we’re continuing to treat this as a SEV-1 incident with the highest priority as we monitor the continued resolution.

We continue to make progress on restoring the system to normal operations in a safe and efficient manner.

As we continue to implement the required changes to reach a resolution, users may continue to experience intermittent 500 errors across multiple endpoints and access to Dashboard and ID may be limited. Some functionalities will be starting to work again but this is not a full fix yet.

We will continue to provide regular updates as we work towards a full resolution. We apologize for the impacts this incident has caused.

We are continuing to make progress on restoring the system to normal operations in a safe and efficient manner.

As we continue to implement the resolution, users may continue to see intermittent 500 errors across multiple endpoints and access to Dashboard and ID may be limited.

We will continue to provide regular updates as we work toward full resolution.

We are continuing to make progress on restoring the system to normal operations in a safe and efficient manner.

As we continue to implement the resolution, users may continue to see intermittent 500 errors across multiple endpoints and access to Dashboard and ID may be limited.

We will continue to provide regular updates as we work toward full resolution.

We are having progress on making our system up and running in a safe and efficient manner.

Access to Dashboard and ID may still be limited.

We will provide another update within the next two hours.

We are having progress on making our system up and running in a safe and efficient manner.

Access to Dashboard and ID may still be limited.

We will provide another update within the next two hours.

We are having progress on making our system up and running in a safe and efficient manner.

Access to Dashboard and ID may still be limited.

We will provide another update within the next two hours.

We continue working on restoring service in a safe and efficient manner.

Access to Dashboard and ID may still be limited.

We will provide another update within the next two hours.

We have identified the cause for the service disruption.

We are currently working to restore service in a safe and efficient manner.

We will provide another update within the next two hours.

We are continuing to investigate this issue with assistance of our service provider. Some information in Dashboard and ID may continue to be unavailable.

Updates will be provided within 2 hours.

Our engineering teams are continuing to work toward a resolution for this issue, pursuing an investigation related to underlying internal connections between some of our components. We have escalated this to our service provider for additional assistance.

Viewing some information in Dashboard and ID may not be possible due to internal dependencies related to this incident.

We are continuing to investigate this issue.

We have also identified that Spreedly Dashboard may be inaccessible to some users.

We are investigating an issue causing intermittent HTTP 500 errors to GET endpoints on Spreedly's Core API.

Updates will be provided as they become available.