SimpleMDM experienced intermittent degraded performance July 26 - July 28

SimpleMDM experienced intermittent degraded performance starting around midday Saturday the 26th spanning into the early morning of Monday the 28th. During this time, certain queued commands were very slow to be processed, impacting device configuration changes and app installation. This was due to queue workers running out of memory. Our team was not immediately notified of the issue due to a misconfiguration in the alerting system covering the service. Once the team was alerted, the workers were restarted which allowed the queue to be cleared out. The root cause of the memory issue took longer to be resolved, leading to additional intermittent slowness for a period of time afterwards. Our team has resolved the cause of the memory issue as well as addressed monitoring and alerting gaps to ensure we are notified if a similar issue were to occur.