Push delivery issues in certain conditions

Incident Report for Batch

Resolved

We identified a bug in our push scheduler affecting delivery in certain conditions, starting on the 25/10/2020 and which was resolved on the 07/11/2020.

The bug only affected campaigns which were scheduled to be sent at the user's local time and only affected users in some particular timezones. Furthermore, the bug only affected applications where a special index had been enabled.
In this subset, new users or users who changed timezone were not properly indexed, resulting in the push scheduler not being able to target them.

The root cause is a dependency upgrade which introduced a bug in its timezone data, which we use in our push scheduler.
After upgrading the dependency again the problem was fixed for new users.

A fix for existing users was written and applied over the next few days, ending on the 30/11/2020.

Posted Oct 24, 2020 - 22:00 UTC