OCPP disconnections/restarts

This incident has been resolved, postmortem will be posted when ready.

We are still monitoring the situation.

There is still full functionality. You may experience some minor delays, but all operations should work as expected. We are still monitoring the situation.

Full functionality have been enabled to OCPP Bridge, and systems are running fine. You may experience some minor delays, but all operations should work as expected. We are still monitoring the situation.

Thank you for your patience.

After the rollback at 10:00, we do see improvements, but we are still operating with limited functionality. With this, all charging stations that are connected, will work as expected. But, if there are done any changes on installation level - updates from installations, circuits, and chargers, OCPP messages will not be received. This includes changes such as change to authentication mode, and the addition or removal of installations, circuits, or chargers from OCPP.
We are still working on a solution to this incident.

After enabling full functionality at 09:15, we do see some issues. We will roll back to limited functionality at 10:00 and keep investigating the issue.

As our systems have been running as expected since last update, we are going to enable full functionality to OCPP at 09.15.

We are seeing improvements to OCPP bridge, and our systems are stable again after our latest deployment. However, OCPP have been deployed with limited functionality. With this, all charging stations that are connected, will work as expected. But, if there are done any changes on installation level - updates from installations, circuits, and chargers, OCPP messages will not be received. This includes changes such as change to authentication mode, and the addition or removal of installations, circuits, or chargers from OCPP.
We are actively monitoring the situation, working on a solution, and expect full functionality to be restored by tomorrow morning, with further improvements ongoing.

We still don't know the root cause, and will continue the investigation through the evening. Thank you for your patience.

Our development team is fully focused on identifying the root cause and working towards a resolution. While we are still investigating and do not yet have a definitive explanation, please rest assured that we have our best people on the case.
We will provide an update as soon as we have more clarity on the situation. Thank you for your patience and understanding.

It is still unclear what's causing the issues. We are continuing to investigate.

At this stage, we are still investigating the root cause of the issue and don’t have a definitive explanation yet. We will provide an update as soon as we have more clarity.

We will continue to investigate this issue today.

We will keep investigating this issue over the night.

We are continuing to investigate this issue.

We can see improvements on our systems as of now. But we are still investigating the issue.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

More logging have been activated to track down the issue. We will continue to investigate.

We are still on top of this issue, and we will continue to investigate. As of now we have no ETA on when we expect the problem to be fixed, but we will update frequently.

We are still investigating this issue.

We are continuing to investigate this issue.

At 13:00 we will do a reboot of the OCPP Proxy. You will see chargers disconnect from the OCPP backend, but should recover shortly after.

We are continuing to investigate this issue.

We are still investigating this issue.

We are continuing to investigate this issue.

We are seeing an increase of BootNotifications being sent from our charging stations without any clear reason. We are currently investigating this issue.