Duo Agentic Chat intermittently fails to respond

Agentic chat on GitLab.com intermittently fails to return responses to user messages; the root cause is under investigation. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.

No material updates to share yet, please follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for any updates.

We are continuing to investigate this incident. The team is currently reviewing the stability of the cloud connector as a potential contributing factor, but this has not yet been confirmed. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.

We are observing transient authentication issues in our cloud connector service and are investigating this as a potential cause of the incident. Please follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for updates.

We have identified a recently deployed merge request as a potential cause of this issue and have begun rolling out the previous version to observe if behavior improves. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.

GitLab Duo Agentic Chat is showing promising improvement after rolling back to a previous version, with responses returning more consistently. We are continuing to review the identified merge request and its changes to confirm whether it is the root cause of the issue. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.

After further testing, the deployment without the suspected merge request has started exhibiting the errors again. We have identified an authentication failure as the cause of this issue and continue to troubleshoot it. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.

We have merged a patch that we believe will help restore GitLab Duo to a usable state, we are waiting for this change to be deployed to observe its impact. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.

After deploying the patch, we are no longer seeing related errors and the service appears stable. We will continue to monitor for a period before considering this incident resolved. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.