Transformations jobs takes long time in az-pnq-gp-001 cluster

The engineering team have investigated the reports from customers and determined that the slowness was related to an overload situation for the RAW bakend. The Transformations service has been running with a lot better performance the last few days, and we have not had any new complaints from users. Cognite is still monitoring the servcie and investigating if there are steps that can be taken to improve the user experience, but overall, the performance is now at a level that should be OK for our customers.

The engineering team is still working on an investigation that can explain why the execution times of transformations are different and some times quite a lot longer than one would expect. The service is meeting the SLOs, but we are still investing time on gaining the understanding needed to explain the observed execution times.

The Cognite Engineering team is currently working on improving the processing times for transformations jobs in the az-pnq-gp-001 cluster. More resources have been added, and we are currently monitoring the effect to asess whether to take additional actions.