Outage in Coveralls

[Re-Opened] Github status updates failing intermittently

Resolved Minor
April 15, 2022 - Started about 2 years ago - Lasted 3 days
Official incident page

Need to monitor Coveralls outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including Coveralls, and never miss an outage again.
Start Free Trial

Outage Details

Re-opening this incident from yesterday, THU, APR 14, since we are receiving more reports this morning: > We are getting reports of Github status updates not arriving at Github for repos that have been fine in this regard for a long time. We are noticing an uptick in background job failures responsible for these status updates. In most cases, these failures resolve immediately by re-trying them. But some fail a few times before succeeding. We suspect there's an issue with the Github API (though not according to https://www.githubstatus.com/), or an isolated issue between coveralls.io and github.com, that's resulting in behavior like rate limiting. There is no known basis for actual rate limiting, but this is the effect. > > Please know that any failing status updates will be retried in exponential back-off fashion (so they should eventually succeed) and that we are monitoring closely and will take any action we can to improve things.
Components affected
Coveralls GitHub
Latest Updates ( sorted recent to last )
INVESTIGATING about 2 years ago - at 04/18/2022 04:32PM

We are keeping this issue open on reports of intermittent (but ongoing) failures to receive Github status updates. We have not identified a root cause but are addressing customer issues individually as we continue to monitor and investigate.

INVESTIGATING about 2 years ago - at 04/15/2022 05:20PM

Re-opening this incident from yesterday, THU, APR 14, since we are receiving more reports this morning:

> We are getting reports of Github status updates not arriving at Github for repos that have been fine in this regard for a long time. We are noticing an uptick in background job failures responsible for these status updates. In most cases, these failures resolve immediately by re-trying them. But some fail a few times before succeeding. We suspect there's an issue with the Github API (though not according to https://www.githubstatus.com/), or an isolated issue between coveralls.io and github.com, that's resulting in behavior like rate limiting. There is no known basis for actual rate limiting, but this is the effect.
>
> Please know that any failing status updates will be retried in exponential back-off fashion (so they should eventually succeed) and that we are monitoring closely and will take any action we can to improve things.

Start monitoring Coveralls and all your cloud vendors in minutes

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 3153 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook

Setup in 5 minutes or less

How much time you'll save your team, by having the outages information close to them?

14-day free trial · No credit card required · Cancel anytime