We have discovered a service disruption in the alerting system of Ymonitor. We would like to inform you about what happened.
Today (31-03-2022) between 17:55 and 19:00, because of a network communication problem, the alerting system did not process some alerts. While measurement data kept coming in, some alerts for some monitors were created later than they happened. Furthermore, notifications could not be sent for some of them (via all media, e.g. SMS, WhatsApp, Slack, ServiceNow, email, etc.). All measurement data and alert data are saved in the databases.
The impact was limited as the network problem was not general, but partial. The possible effects that might have been observed by the end users are as follows.
For some monitors and services (not all): 1. An alert notification for an open alert is not received. 2. An alert notification for a closed alert is received without a previous notification of an open alert. 3. An alert record is seen on Yviewer and/or dashboards later than it happened.
No measurement data was lost during this time. Since 19:00 the alerting notification services are working normally. We apologise for the inconvenience that this might have caused.
Should you have any questions, please feel free to reach out to your service delivery contact.
Posted Mar 31, 2022 - 22:03 CEST
This incident affected: Ymonitor Dashboards, ymonitor.nl, API, Measurement Data Storage, Alerting, and YGate API.