Connectivity Issues Post-Maintenance (SP-489) : Prisma Access China

Update: Status changed to Resolved.

Status: Resolved
Severity: Degraded
Affected Locations: Beijing, Ningxia
Customer Impact: Update Details
We have successfully identified the root cause of the connectivity issue impacting the tunnels from the affected nodes following the recent infrastructure upgrade.

A fix has been applied across the environment, and connectivity between the affected nodes has been restored for all impacted customers. We are seeing tunnels establish and stabilize post addition of the fix.

Current Status: Monitoring
The incident is now in the Monitoring stage. Our engineering team is closely observing the service for any other signs of degradation

Service Impact: Resolved

Action Taken: Fix deployed to stabilize the tunnel connectivity issues.

Next Update: We will provide a final "Resolved" status once the monitoring period is successfully completed.
Workaround: No work around at the moment.

Thank you for your patience.

Update: Status changed to Monitoring.

Status: Monitoring
Severity: Degraded
Affected Locations: Beijing, Ningxia
Customer Impact: Update Details
We have successfully identified the root cause of the connectivity issue impacting the tunnels from the affected nodes following the recent infrastructure upgrade.

A fix has been applied across the environment, and connectivity between the affected nodes has been restored for all impacted customers. We are seeing tunnels establish and stabilize post addition of the fix.

Current Status: Monitoring
The incident is now in the Monitoring stage. Our engineering team is closely observing the service for any other signs of degradation

Service Impact: Resolved

Action Taken: Fix deployed to stabilize the tunnel connectivity issues.

Next Update: We will provide a final "Resolved" status once the monitoring period is successfully completed.
Workaround: No work around at the moment.

We are continuing to monitor the situation.

Update: Changes to Customer Impact.

Status: Investigating
Severity: Degraded
Affected Locations: Beijing, Ningxia
Customer Impact: Update Details
We have successfully identified the root cause of the connectivity issue impacting the tunnels from the affected nodes following the recent infrastructure upgrade.

A fix has been applied across the environment, and connectivity between the affected nodes has been restored for all impacted customers. We are seeing tunnels establish and stabilize post addition of the fix.

Current Status: Monitoring
The incident is now in the Monitoring stage. Our engineering team is closely observing the service for any other signs of degradation

Service Impact: Resolved

Action Taken: Fix deployed to stabilize the tunnel connectivity issues.

Next Update: We will provide a final "Resolved" status once the monitoring period is successfully completed.
Workaround: No work around at the moment.

We will provide updates as we learn more.

Update: Changes to Customer Impact.

Status: Investigating
Severity: Degraded
Affected Locations: Beijing, Ningxia
Customer Impact: Progress Update
We are continuing to work on the connectivity issues affecting tunnels originating from Explicit Proxy (EP) nodes following today’s infrastructure upgrade.

We have successfully mitigated the impact for a subset of affected customers, who should now see stabilized connectivity between EP nodes and Service Connections. The SRE and the engineering team have also identified an issue that is affecting tunnel connectivity from Remote Networks and Service connections to the branch and the Data center sites respectively for a subset of customers. Our engineering teams are now focused on remediating these tunnel issues for the remaining customer environments.

Current Impact
Mitigated Customers: Connectivity has been restored; however, we recommend monitoring for any further intermittent behavior.

Pending Remediation: Some customers may still experience intermittent tunnel drops or packet loss.

Next Steps
Our technical teams are working through the remaining service instances to apply the fix. We are treating this as a high-priority post-upgrade event and will provide a final update once all customer environments have been confirmed stable.
Workaround: No work around at the moment.

We will provide updates as we learn more.

We are currently investigating an issue affecting Configuration

Status: Investigating
Severity: Degraded
Affected Locations: Beijing, Ningxia
Customer Impact: Incident Overview
While the scheduled infrastructure maintenance for the Beijing and Ningxia regions was successfully completed, we have identified a new connectivity issue that has emerged post-upgrade.

Our monitoring systems indicate that all tunnels originating from Explicit Proxy (EP) nodes are experiencing intermittent connectivity. This is specifically impacting the stability of tunnel traffic from the EP nodes.

Technical Details
Status of Maintenance: The infrastructure upgrade itself is finished.

The Issue: A post-upgrade issue is causing intermittent tunnel flapping or drops for all tunnels from EP nodes.

Impact: Users may experience inconsistent access to resources behind the tunnels terminating from the affected EP nodes.

Current Action Plan
Engineering teams have been re-engaged to investigate the root cause of this post-upgrade behavior. We are currently analyzing the tunnel configurations and routing tables to restore full stability. We will provide updates as soon as more information becomes available.
Workaround: No work around at the moment.

We will provide updates as we learn more.