The Cint platform is 100% operational and restored to our normal high standards.
A backend component responsible for aggregating and calculating prices was overloaded and stopped responding to requests. This in turn caused other services to fail.
The number of requests made to the component was outside it's design parameters. At 12:41 CET automated monitoring systems made engineers aware of a problem with the component. The primary bottleneck for the component was identified to be a database. Additional capacity was added. By this time the users started to see the impact of the issue on Access, Access Pro, Buyer API.
Engineers also identified the source of the increased requests to be a misconfigured reporting function. The reporting function was disabled. With the increased capacity and the misconfigured reporting function disabled the component recovered at 14:20 CET. Teams further monitored the system to ensure there were no more issues, and confirmed the same at 16:17 CET.
We are reviewing the issue to avoid similar issues in the future.
This issue has been fully resolved.
Thank you.
Team Cint
Posted Jan 25, 2021 - 12:04 UTC
Monitoring
Good news!
Performance is back to normal for creating orders and saving drafts.
Our team has addressed and fixed the cause of the problem and is closely monitoring the platform to make sure your work can continue without any obstacles.
We're sorry again to the group of users who were affected by this challenge and thank you once more for your understanding.
Team Cint
Posted Jan 22, 2021 - 15:17 UTC
Identified
Hi again,
Our team is already making the changes needed to bring performance back to our usual high standards.
We're aiming to have orders and saving drafts back to full strength again shortly.
We will continue to update you as we keep making progress.
Thank you again for your continued patience.
Team Cint
Posted Jan 22, 2021 - 13:42 UTC
Investigating
Hi there,
While the Cint platform is online, we wanted to let you know that we are aware of an issue that is preventing several key aspects of the platform from functioning properly.
Users may experience issues with creating orders and saving drafts.
We firstly want to apologize to any users who have had their work impacted in any way by this occurrence.
Rest assured, several teams are looking into this as a matter of urgency and we expect the platform to be back to full strength again shortly.
We will of course keep you updated as work progresses.
Thank you for your understanding.
Team Cint
Posted Jan 22, 2021 - 13:07 UTC
This incident affected: Demand Exchange Services (Access, Access Pro, Buyer API) and Insight Exchange - Project Management Admin.