On 23 October 2022 starting around 22:50 (CET) we were notified that some of our systems were not operating normally. We immediately started investigating and took some corrective actions. The problems were solved around 23:21 (CET).
We have started to investigate the cause of the issue in the morning. We found out that our service provider AWS had restarted one of our services for reasons we can't identify at this time. As a result, some measurements in some monitors were lost.
To make our systems more resilient to these kinds of unforeseeable effects we have started to work on implementing some preventive measures in our code base. They are currently under test and expected to be deployed next week.
Meanwhile, we started a probe to investigate the unexpected restart of the service by AWS. We are working with them to find the root cause of this. Once we know more we will update you.