Outage in Farm HPC cluster
Farm's slurmdbd having intermittent issues
Resolved MajorIncident Report
Farm's slurmdbd is having intermittent issues. If you see an error like below, it means the problem has occurred again, and we will restart slurmdbd to bring it back into service.
"""sacctmgr: error: _open_persist_conn: failed to open persistent connection to host:monitoring-ib:6819: Connection timed out
sacctmgr: error: Sending PersistInit msg: Connection timed out"""
We have a support case open with SchedMD and will update this issue as we learn more.
Trusted by 1,000+ teams
The Status Page Aggregator with Early Outage Detection
Stop finding out about outages from your users. Monitor 6,320+ cloud services and get alerted the second something breaks.
Start Free Trial
- No credit card
- 14-day trial
- 2-minute setup
Latest Updates ( sorted recent to last )
RESOLVED
7 months ago - at 09/23/2025 12:42AM
This incident has been resolved.
INVESTIGATING
12 months ago - at 04/24/2025 12:22AM
Farm's slurmdbd is having intermittent issues. If you see an error like below, it means the problem has occurred again, and we will restart slurmdbd to bring it back into service.
"""sacctmgr: error: _open_persist_conn: failed to open persistent connection to host:monitoring-ib:6819: Connection timed out
sacctmgr: error: Sending PersistInit msg: Connection timed out"""
We have a support case open with SchedMD and will update this issue as we learn more.
Latest Farm HPC cluster outages
Partial degradation of logins to Farm - about 1 month ago
CSE subdomain offline. Farm available at farm.hpc.ucdavis.edu. - about 2 months ago
Slurm not available - 3 months ago
nas-6-1 still not functioning - 3 months ago
Quobyte Storage is Unavailable - 6 months ago
The Status Page Aggregator with Early Outage Detection
With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.
Start free trialNo credit card required · Cancel anytime · 6320 services available
Integrations with
