Failed delivery of notification emails for very large query submissions and some forward-link alerts

This incident has been resolved. -IF

A fix has been implemented and we are monitoring the results.

We have confirmed that all notifications for registrations are being delivered and have been since 22 July. A small fraction of query submission email notifications (for very large queries) and forward-link alerts are going undelivered. Our technical team continues to work on a fix. -IF

We are continuing to work on a fix for this issue.

We are continuing to work on a fix for this issue.

We are continuing to work on a fix for this issue.

We are continuing to work on a fix for this issue.

We have resolved the problems for the vast majority of members and those submissions, but we know that a fraction of notifications are failing for very large submissions. We are continuing to work on a fix for this issue. -IF

We are continuing to work on a fix for this issue.

Yesterday's release did fix delivery of notification emails for many submissions, but we still have a small minority of notification emails that are going undelivered. Our technical team continues to work on a fix. -IF

We'll have an emergency, unplanned release (https://status.crossref.org/incidents/c5nyp41k6vx5) in ~20 minutes to address the ongoing errors with delivery of notification emails after registrations.

During that release window - from 1500 to 1700 UTC today - our members may experience delays in deposit system admin tool (doi.crossref.org) processing. Deposits submitted during this time will be stored and processed after the emergency maintenance has been completed.

We are continuing to work on a fix for this issue.

Delivery of email notifications of submission logs following a registration (for Simple Text Query form, web deposit form, and XML submissions) has been restored for many members and their submissions. We are working to remedy the other outstanding errors that are causing some email notifications to go undelivered. -IF

Delivery of email notifications of submission logs following a registration (for Simple Text Query form, web deposit form, and XML submissions) began failing again late yesterday. Our technical team continues to work on a fix. -IF

Submissions from Metadata Manager and the Simple Text Query form have been restored. Unfortunately, delivery of email notifications of submission logs following a registration (for Simple Text Query form, web deposit form, and XML submissions) do appear to be delayed. We believe this is due to the volume of notification emails in the system, and the result of quite a backlog there. We are monitoring. -IF

A fix has been implemented and we are monitoring the results.

We are continuing to work on a fix for this issue.

We'll have an emergency, unplanned release (https://status.crossref.org/incidents/1c42q5j29ds2) in ~30 minutes to address the ongoing issues with Metadata Manager and newly reported errors with delivery of notification emails after registrations.

During that release window - from 1430 to 1630 today - our members may experience delays in deposit system admin tool (doi.crossref.org) processing. Deposits submitted during this time will be stored and processed after the emergency maintenance has been completed.

Errors to Crossmark dialog boxes have been fixed, as you can see with this example: https://crossmark.crossref.org/dialog/?doi=10.36676/girt.v12.i1.109&domain=pdf

We're continuing to work on Metadata Manager and the Simple Text Query form. -IF

We are continuing to work on a fix for the errors present with Metadata Manager registrations. We have also identified errors with reference registrations via the Simple Text Query form (https://apps.crossref.org/SimpleTextQuery/) and errors in the display of Crossmark dialog boxes (e.g., https://crossmark.crossref.org/dialog/?doi=10.3233/JIFS-179309&domain=pdf). -IF

The issue has been identified and we continue to work on a fix for Metadata Manager registrations that are failing with the 'TypeError' error messages. We have deployed a fix for the depositor reports. That issue is resolved (e.g., https://data.crossref.org/depositorreport?pubid=J458872). -IF

As a result of our recent AWS migration (https://status.crossref.org/incidents/s92nm4xn1w84), Metadata Manager submissions are currently returning 'TypeError' error messages and our depositor reports are returning a 404 'Not Found' status code error message. Our technical team is working on a fix. -IF