Outage in AgResearch eRI
compute nodes in Slurm are draining
Resolved MajorIncident Report
compute-[0-4],gpu-0,vgpu-2 nodes are all in a draining state due to "Kill task failures". The nodes will be investigated and resumed in Slurm shortly
Trusted by 1,000+ teams
Never miss outages in third-party dependencies
Stop finding out about outages from your users. Monitor 6,320+ cloud services and get alerted the second something breaks.
Latest Updates ( sorted recent to last )
RESOLVED
9 months ago - at 08/11/2025 02:35AM
This incident has been resolved.
MONITORING
9 months ago - at 08/10/2025 11:07PM
We are continuing to monitor for any further issues.
MONITORING
9 months ago - at 08/08/2025 12:32AM
The compute nodes have once again been resumed in Slurm.
INVESTIGATING
9 months ago - at 08/08/2025 12:02AM
Compute nodes are draining again. Investigation is underway
MONITORING
9 months ago - at 08/07/2025 11:19PM
We found numerous long waiters and defunct java processes across all nodes. Restarting GPFS on gpu-0, which had drained, cleared all the deadlocks, waiters, and defunct processes.
All compute nodes have been checked and now resumed in Slurm
INVESTIGATING
9 months ago - at 08/07/2025 10:46PM
compute-[0-4],gpu-0,vgpu-2 nodes are all in a draining state due to "Kill task failures". The nodes will be investigated and resumed in Slurm shortly
Latest AgResearch eRI outages
Slow Access to Data on S: Datasets on ERI for several Users - 2 months ago
Windows clients hanging when accessing dataseta - 3 months ago
compute-[1-4] nodes affected by a GPFS long waiter - 4 months ago
No Internet connectivity from compute nodes - 4 months ago
Research Developer Cloud Dashboard and API Issue - 4 months ago
Never miss outages in third-party dependencies
With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.
Start free trialNo credit card required · Cancel anytime · 6320 services available
Integrations with
