Investigating GoFormz Service Disruption

This incident has been resolved. Please read on for a full post mortem of the incident.

GoFormz Service Disruption on April 16, 2026

On April 16, 2026, GoFormz experienced a service disruption that made our application slow or unreachable for many users over a roughly 35-minute window. We know reliability is critical to your business, and we apologize for the disruption.

Root Cause

On April 15, 2026, GoFormz rolled out a maintenance process designed to clean up expired transient data in our database and keep the system running efficiently. After approximately 24 hours, this process began conflicting with normal application activity under real-world usage, causing database deadlocks. These deadlocks blocked access to our core database, leading to significant application latency and user-facing timeouts. The issue was mitigated by disabling the cleanup service and increasing database capacity.

Impact to Customers: Application latency and user-facing timeouts.

- Start of impact: 8:23 AM PDT
- Mitigation began: 8:55 AM PDT
- Service restored: 8:59 AM PDT

During this window, you may have experienced very slow response times, login attempts that timed out in your browser, or been unable to access the application altogether.

Why We Didn't Catch It Sooner

GoFormz runs automated tests around the clock that simulate user logins to confirm the application is working. These tests only validated success, not performance.

What We're Doing About It

1. We will be redesigning this database maintenance process so it cannot interfere with normal application activity, regardless of load.
2. Better monitoring. We're adjusting speed thresholds to our automated tests and adding a new layer of alerts so we're alerted the moment the application becomes too slow under such circumstances. This will address the gap in login performance testing.

Again, we apologize for the disruption. If you have any questions, please reach out to our support team.

A fix has been implemented and all services are back to normal operation. We will follow up with a post mortem ASAP.

We are currently experiencing a major outage impacting all Web GoFormz services.