Need to monitor WiscNet outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including WiscNet, and never miss an outage again.
Start Free Trial
This incident has been resolved.
WiscNet's monitoring indicates that the last TEACH circuit was fixed at approximately 11:52AM. If you are still experiencing connectivity issues please reach out to WiscNet support. WiscNet will continue to work with TEACH to provide an RFO.
Further analysis on the collateral damage to non-TEACH services on the WiscNet network. The TEACH outage this morning was causing a large number of circuits to flap on two WiscNet core routers. Our initial analysis is pointing to these two core routers not being able to keep up with the protocol churn from circuit flaps and packet loss on this many circuits. A large number of members transit these two routers to get to the internet as these routers are members of WiscNet's 2x100Gbps core backbone ring. These routers should be using their ASICs for forward most traffic not destined to the control plane. We'll be talking about mitigations internally to limit impact in the future.
The related TEACH NOC GLS ticket number for this event is 10685278.
The TEACH configuration rollback is complete. The majority of TEACH circuits are back online. Four remain, and WiscNet and TEACH are actively troubleshooting.
WiscNet has received multiple reports from non-TEACH connected members of performance issues and outages. WiscNet is investigating high CPU utilization on core routers that land TEACH circuits in both Madison and Milwaukee as protocol churn from circuits flapping appear to be causing performance issues.
WiscNet is aware of and monitoring widespread circuit outages and packet loss remaining after a TEACH maintenance that occurred early in the morning on October 25th. WiscNet has been notified that TEACH is working on rolling back the changes at this time. WiscNet will update this Status Page incident with further information when we receive it.
With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.
Start free trialNo credit card required · Cancel anytime · 3278 services available
Integrations with
How much time you'll save your team, by having the outages information close to them?
14-day free trial · No credit card required · Cancel anytime