Possible issues loading assessment content in all regions

As of 19:40 UTC, we've concluded one hour of active monitoring without additional issues. We will continue to keep an eye on everything but are now ready to call this incident resolved.

Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required.

As of 18:36 UTC, we've now recovered fully and all stacks are operational. We will continue to monitor for a period before resolving the incident.

Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

As of 18:20 UTC, we've verified a cause of the current incident and recovery is already partially complete. We have restored part of the assessment stack and are working on full restoration now, with the remaining stacks to follow. We will provide additional updates soon, including a possible ETA as soon as a reasonably accurate estimate has been determined.

Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

We've now identified a possible cause for the current incident and are working to resolve the issue. We will add additional information as soon as it becomes available.

We've also added EMEA authoring and APAC authoring to affected regions for users of the hosted author site, which operates in the AMER region.

Learnosity Support and Systems Engineering teams are continuing to actively investigate the current issue, and will follow on with an update and resolution as soon as possible.

Correction: The authoring and analytics stacks are also begin impacted intermittently in workflows where the assessment stack is in play. (E.g. editing views where the assessment stack is used for previews.) Errors in the AMER region continue to reduce. 502 errors have cleared and 504 timeout errors are now being investigated. Elevating classifications to include all stacks and partial outage.

Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

As of 14:15 UTC, the assessment stack returned to normal operating metrics in the EMEA and APAC regions. As of 14:30 and 14:45 UTC, the NA region error count dropped by approximately 15% each period, and error counts are continuing to reduce as of 15:00 UTC. Intermittent errors continue to occur in the assessment stack in AMER. Authoring and Analytics APIs remain unaffected.

Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

As of 14:15 UTC, we've identified that the AWS Auto Scaling Groups failed to create additional EC2 instances quickly enough to accommodate sudden atypical scale increases, causing an elevated number of 502 errors in the assessment stack. Additional manual scaling is reducing the strain and availability of assessment content is starting to return to normal. We are actively continuing to manually adjust settings and monitor the situation.

Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

As of 14:00 UTC, We are seeing intermittent issues with loading demo assessment content and are investigating out of an abundance of caution.

Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.