Intermittent slowness processing chip and pin transactions - UK/IE Opayo

Following the fix at around 12:25 UTC, we've seen the error rates reduce to expected levels for sending payments to chip and pin devices using the Opayo payment provider. After monitoring for some time, we're confident that this has now been resolved.

Thank you for your patience and understanding while we worked to investigate and solve this issue.

If you experience any further issues, or have any questions, please do get in touch with our team via support@spektrix.com or give us a call on +44 (0)20 7183 3586.

We have implemented a change which has had a positive effect on the number of errors we are seeing and the delays experienced in sending payments to C&P devices. We will continue to monitor the situation to determine whether further action is needed.

Please continue to report any issues to us with details of the length of delay you experience in processing payments.

Our updates this morning have not made a significant improvement to the delays experienced with C&P devices, but having identified the cause we are continuing to investigate solutions which can mitigate and resolve the impact.

Thank you for your patience. We will update again this afternoon.

We are continuing to work to fix issues with slowness sending C&P payments to devices.

This morning we deployed a new version of the boxoffice API but this did not mitigate the issue. We are currently working on another release which we hope will have some impact and we will be able to give another update in an hour and a half.

Thank you for your patience and for providing logs. We no longer need new logs as we can now monitor impact on our side.

We've identified the cause of the slowness. There is a queuing delay when Opayo Chip & Pin transactions are sent to the Spektrix Box Office App. They will complete and your terminals will display the payment, but this can take several minutes and is intermittent. We are now investigating if this is because of a Microsoft Platform issue. Thank you for your patience and thank you if you sent through logs and conducted testing to help us narrow this down.

We're still seeing slowness when sending chip & pin payments to the Opayo terminal. Thank you for your patience as we continue to narrow down the cause of this issue. Please continue to send us your Box Office Logs and timestamps of exactly when you're experiencing the slowness. If you need support in sending these to us, please contact our Support Team.

We've increased resource to the database that receives box office app messages. This may relieve symptoms. Please continue to test transactions and feedback to our support teams if you are seeing improvements or continued slowness. We are continuing to investigate and will update here again in an hour.

We've received reports that there is still significant slowness between starting a chip and pin transaction, and that transaction then appearing on the pinpad terminal screen.

We are renewing our investigation to isolate the cause now that we've verified last week's fix hasn't had impact.

Please continue to send us screen recordings of the issue happening, with timestamps, along with the Sagepay and Box Office App Logs to help with our investigation. You can find these at the filepath C:\sagepay\guardian\logfiles and C:\Program Files (x86)\Spektrix Ltd\LocalApp > BoxOfficeApp.{number}.log If you need help retrieving those, please call the Spektrix Support team for help.

We will update here in the next hour.

We have been able to isolate a particular aspect of our infrastructure that may have been causing slowness when attempting a chip and pin transaction.

We haven't been able to completely verify this has fixed the issue but we have seen some positive signs from those we have managed to test with.

To anyone affected, we encourage you to try again and try multiple payment attempts to see if speeds are consistently faster. Restarting your PC may also help with any slowness.

We will continue to work on investigating this and will eagerly receive any further reports either of continued issues or improvements.

We're continuing to investigate intermittent slowness issues when starting to process a chip and pin payment using an Opayo pin pad.
We are in dialogue with Opayo as well as checking all other infrastructure to establish a cause.

If you have an issue and you are able to send a screen recording whilst you are waiting for the payment message to appear on your pin pad, that shows the time on the PC, this would help us track more detailed logging. We can use this in combination with a specific log file ("C:\sagepay\guardian\logfiles\GuardianServer.log") to cross reference the timings.

If you need any support with this let us know and we can set up a screenshare session with you to record your screen and get the log file directly.

Thanks for your patience as we continue to investigate this issue.

We are continuing to investigate these reports of slowness processing C&P payments and are in dialogue with Opayo. If you are experiencing issues please contact support. It will help to have log files and where possible a video recording of the slowness.

We are remaining in dialogue with Opayo and reviewing all possibilities that could be causing these issues. If you are experiencing problems we will likely need to request some further information from you including log files from the pin pad, if this is the case we will contact affected users directly.

We'll continue to update here with any progress on the issue.

We are investigating reports that there is intermittent slowness affecting chip and pin transactions on the Opayo payment provider in the UK and Ireland.

The slowness is happening after the user selects to pay by chip and pin and causes a delay in the payment information loading on the card reader. Other aspects of payment processing across all payment providers are functioning normally.

We are in dialogue with all relevant parties to establish any potential causes of the problem and will update when we hear more.