Outage in CodeTwo

[Global] Auto replies incorrectly considered as spam and not delivered to recipients

Resolved Minor
July 26, 2023 - Started 9 months ago - Lasted about 21 hours
Official incident page

Need to monitor CodeTwo outages?
Stay on top of outages with IsDown. Monitor the official status pages of all your vendors, SaaS, and tools, including CodeTwo, and never miss an outage again.
Start Free Trial

Outage Details

Our monitoring team have noticed that within the last 24 hours a large number of auto replies sent from Microsoft 365 tenants (everywhere in the world) has been considered as spam by Microsoft and thus not delivered to CodeTwo for final processing. Therefore, these messages have not been delivered to final recipients. It looks like Microsoft, for some reason, started to classify auto replies (and maybe also other types of automated messages, but we have not been able to confirm this yet) as spam or phishing and send them through a new high-risk delivery pool with IP range 40.95.0.0/16 (which is not a part of official "spf.protection.outlook.com"). Since Microsoft native mechanisms indicate that these emails are high risk messages, our software security mechanisms reject them to protect our IPs reputation. We have started working on this problem with Microsoft Premier Support now, but we recommend that you contact Microsoft support for assistance as well. To apply a temporary fix, you can exclude auto replies from going through CodeTwo transport rule as described in this article (https://www.codetwo.com/kb/automatic-replies-should-not-be-processed/), but note that these emails may still get blocked/rejected on the recipient’s end, as most of mail servers do not accept emails sent through the high-risk delivery pool (https://learn.microsoft.com/en-us/microsoft-365/security/office-365-security/outbound-spam-high-risk-delivery-pool-about?view=o365-worldwide#high-risk-delivery-pool). All standard emails sent by users are not affected by this problem. The next update will be provided within the next 4 hours.
Components affected
CodeTwo Signature preview CodeTwo Signature adding (rules processing) CodeTwo Sent Items Update CodeTwo Signature adding (rules processing) CodeTwo Sent Items Update CodeTwo Signature preview CodeTwo Onboarding CodeTwo Onboarding CodeTwo Mail flow CodeTwo Signature adding (rules processing) CodeTwo Signature preview CodeTwo Mail flow CodeTwo Signature adding (rules processing) CodeTwo Signature preview CodeTwo Manage Signatures App CodeTwo Signature adding (rules processing) CodeTwo Sent Items Update CodeTwo Manage Signatures App CodeTwo Mail flow CodeTwo Onboarding CodeTwo Mail flow CodeTwo Signature preview CodeTwo Signature adding (rules processing) CodeTwo Manage Signatures App CodeTwo Admin Panel CodeTwo Onboarding CodeTwo Admin Panel CodeTwo Sent Items Update CodeTwo Mail flow CodeTwo Manage Signatures App CodeTwo Admin Panel CodeTwo Sent Items Update CodeTwo Mail flow CodeTwo Sent Items Update CodeTwo Signature adding (rules processing) CodeTwo Azure AD data caching CodeTwo Azure AD data caching CodeTwo Azure AD data caching CodeTwo Azure AD data caching CodeTwo Azure AD data caching CodeTwo Onboarding CodeTwo Sent Items Update CodeTwo Onboarding CodeTwo Mail flow CodeTwo Manage Signatures App CodeTwo Signature preview CodeTwo Sent Items Update CodeTwo Signature adding (rules processing) CodeTwo Manage Signatures App CodeTwo Manage Signatures App CodeTwo Mail flow CodeTwo Onboarding CodeTwo Admin Panel CodeTwo Admin Panel CodeTwo Signature preview CodeTwo Manage Signatures App CodeTwo Signature preview CodeTwo Onboarding CodeTwo Azure AD data caching CodeTwo Admin Panel CodeTwo Admin Panel CodeTwo Admin Panel CodeTwo Azure AD data caching CodeTwo Azure AD data caching
Latest Updates ( sorted recent to last )
RESOLVED 9 months ago - at 07/27/2023 07:57AM

We have confirmed this is a solely Microsoft issue (not related to CodeTwo in any way) and that it most likely concerns a subset of Microsoft 365 tenants. All CodeTwo services have been operating normally during the time of the incident. If you believe your tenant is affected by this Microsoft issue, please contact Microsoft support.

IDENTIFIED 9 months ago - at 07/26/2023 02:25PM

Microsoft is looking into this issue. From what we have gathered, the problem concerns Outlook/Exchange native auto replies only. Even if CodeTwo is switched off (bypassed), auto replies from affected tenants usually land in junk folders due to them being sent from the high-risk delivery pool with IP range 40.95.0.0/16, which is not a part of official "spf.protection.outlook.com".

Only auto replies are affected. Other emails are processed and delivered normally. Signatures are added normally. All CodeTwo services in all regions operate as normal.

IDENTIFIED 9 months ago - at 07/26/2023 10:33AM

Our monitoring team have noticed that within the last 24 hours a large number of auto replies sent from Microsoft 365 tenants (everywhere in the world) has been considered as spam by Microsoft and thus not delivered to CodeTwo for final processing. Therefore, these messages have not been delivered to final recipients.

It looks like Microsoft, for some reason, started to classify auto replies (and maybe also other types of automated messages, but we have not been able to confirm this yet) as spam or phishing and send them through a new high-risk delivery pool with IP range 40.95.0.0/16 (which is not a part of official "spf.protection.outlook.com"). Since Microsoft native mechanisms indicate that these emails are high risk messages, our software security mechanisms reject them to protect our IPs reputation.

We have started working on this problem with Microsoft Premier Support now, but we recommend that you contact Microsoft support for assistance as well.

To apply a temporary fix, you can exclude auto replies from going through CodeTwo transport rule as described in this article (https://www.codetwo.com/kb/automatic-replies-should-not-be-processed/), but note that these emails may still get blocked/rejected on the recipient’s end, as most of mail servers do not accept emails sent through the high-risk delivery pool (https://learn.microsoft.com/en-us/microsoft-365/security/office-365-security/outbound-spam-high-risk-delivery-pool-about?view=o365-worldwide#high-risk-delivery-pool).

All standard emails sent by users are not affected by this problem. The next update will be provided within the next 4 hours.

Stay informed of vendor status updates

With IsDown, you can monitor all your critical services' official status pages from one centralized dashboard and receive instant alerts the moment an outage is detected. Say goodbye to constantly checking multiple sites for updates and stay ahead of outages with IsDown.

Start free trial

No credit card required · Cancel anytime · 3153 services available

Integrations with Slack Microsoft Teams Google Chat Datadog PagerDuty Zapier Discord Webhook

Setup in 5 minutes or less

How much time you'll save your team, by having the outages information close to them?

14-day free trial · No credit card required · Cancel anytime