Data Processing Delays
Incident Report for Batch
Resolved
Everything is back to normal since 15h UTC
Posted Mar 17, 2020 - 17:38 UTC
Monitoring
Identification :
We replaced a node on one of our Cassandra database cluster. This new node had a different hardware setup and was not working correctly with Cassandra. We decommissioned that node and replaced it with another to restore the service.

During those manipulations, no data has been lost, but :
- the indexation of inbox messages are late for about 3h : your user can see only the messages for 3 hours ago
- the indexation of custom attributes are late for about 1h30 : your queries can use stale attributes
- some hits for inapp-messaging and inbox serving may have terminated in 500

We're still monitoring the indexation drifts.
Posted Mar 17, 2020 - 14:02 UTC
Investigating
We are currently experiencing data processing delays, impacting the custom data indexation, the Inbox and the retargeting features. No data is being lost. We will post updates shortly.
Posted Mar 17, 2020 - 11:53 UTC
This incident affected: Webservices (Inbox, Custom attributes, In-app messaging).