[2019-02-13] Resolved: Dashboard campaign listing issue
[Update 07:30 GMT] The issue has been fixed and push campaigns can now be listed.
[05:00 GMT] We are aware of a issue affecting our dashboards where campaigns cannot be listed. Notifications sent from the dashboard, Campaign API and Transactional API are not impacted.
13th Feb, 2019
[2019-02-12] Resolved: Push campaigns issue
[Update 18:22 GMT] The issue has been fixed and push campaigns are now being sent as expected.
[15:00 GMT] We are aware of a major issue affecting all campaigns sent with an image and scheduled from Batch's dashboard or the Campaigns API. Notifications sent from the Transactional API are not impacted. We have disabled images while the issue is being resolved. As a consequence, all the campaigns including an image will be delivered without any images.
12th Feb, 2019
[2019-02-01] Resolved: Android Transactional Notifications Delay
[14:00 GMT] We are aware of an issue causing delivery delays on a small number of Android transactional notifications
[14:27 GMT] The issue has been resolved.
1st Feb, 2019
[2019-01-24] Completed: Scheduled maintenance
[Update 11:17 GMT] The maintenance is completed.
[09:00 GMT] We will be performing a maintenance today at 11:15 GMT. Creating or updating new campaigns won't be possible for a couple of seconds.
24th Jan, 2019
[2019-01-23] Resolved: Push campaigns delivery delay
[Update 09:39 GMT] The issue has been fixed.
[08:23 GMT] We are aware of an issue causing delivery delays on iOS and Android. This shouldn't impact clients on the Enterprise plan. We are currently working on a fix.
23rd Jan, 2019
[2019-01-21] Resolved: Push campaigns delivery issues
[Update 12:20 GMT] The issue has been fixed. Campaigns created less than 30 minutes ago have been resent automatically.
[10:40 GMT] We are currently investigating an issue with campaign delivery. Campaigns created from the dashboard or the Campaigns API since 10:20 are not being sent.
21st Jan, 2019
[2018-12-12] Resolved: iOS push delivery issues
[Update 17:02 GMT] No issues have been detected since 14h37 GMT and iOS push delivery is back to normal.
[Update 14:37 GMT] Push delivery should be back to normal. Our team is monitoring our services closely.
[14:07 GMT] We are currently investigating an issue impacting iOS notifications. The impacted campaigns are sent with a significant delay or canceled before our security timeout.
12th Dec, 2018
[2018-11-19] Resolved: Login and delivery issues
[Update 9:40 GMT] The delivery issue has been fixed. Campaigns scheduled to be sent during the downtime were sent with some delay.
[Update 8:47 GMT] The login issue has been fixed. Our team is still working on the push delivery issue.
[8:40 GMT] We are aware of an issue affecting push delivery and preventing our clients from logging in in some cases. Some campaigns may not be sent correctly. We are working on a fix.
19th Nov, 2018
[2018-11-13] Resolved: Label-based frequency capping issue
[Update 19:12 GMT] The issue has been fixed.
[9:50 GMT] We have detected an issue impacting exclusively the
label-based frequency capping. These rules haven't been taken into account from November 7th. Our team is working on a fix.
13th Nov, 2018
[2018-10-30] Resolved: Batch webservices downtime
[Update 14:50 GMT] The issue has been fixed and our webservices are back online.
[14:48 GMT] An issue is currently causing a downtime of our webservices (dashboard + push delivery). We are working on a fix.
30th Oct, 2018
[2018-10-18] Resolved: Batch APIs downtime
[Update 15:39 GMT] The issue has been fixed. Batch's APIs were not available for 12 minutes.
[15:28 GMT] We have detected an issue impacting our public APIs. We will post an update here when it's fixed.
17th Oct, 2018
[2018-10-16] Resolved: Recurring campaigns did not respect cappings
[Update 2018-10-17 8:40 GMT] All the events have been replayed. The issue is now resolved.
[Update 14:50 GMT] Our team found the issue and deployed a fix. Additionally we have started replaying old events to correct the capping data. We expect this process to take at most a day.
[13:15 GMT] We have identified a problem with recurring campaigns that only impacted customers who are not using the retargeting feature. Due to a bug introduced on the 11th of October (2018/10/11) the capping data was not updated correctly for those customers and the direct result was that users might have been pushed multiple times for a campaign if they matched the targeting.
16th Oct, 2018
[2018-10-15] Resolved: Push campaigns delivery issues
[Update 19:49 GMT] The issue is now fixed, we'll continue to closely monitor our infrastructure tonight. We identified that a specific campaign badly impacted some others starting 17:00 GMT. It reached a peak before 19:00 GMT where the system could not deliver all campaigns in time. We cleaned the system and all late notifications started were released after 19:40 GMT.
[19:17 GMT] We are currently investigating an issue with campaign delivery. The impacted campaigns are sent with a significant delay or canceled before our security timeout.
15th Oct, 2018
[2018-09-30] Resolved (+ Post-mortem): Push campaigns delivery issues
[Update 14:10 GMT] Push delivery is back to normal. Campaigns that would have been delayed too much have been cancelled. The dashboard push campaign analytics accurately show what has been sent, and what has not been.
[Update 10:51 GMT] Push delivery is progressively going back to normal for recent or new campaigns.
[Update 10:42 GMT] Our team has identified the issue and is currently working on a fix.
[8:58 GMT] We are currently investigating an issue with campaign delivery. Some campaigns might have a significant send delay
This is an unusually rare and important issue that our team has been taking very seriously and will continue to investigate this week to understand how it could happen and how to make sure it can never occur again.
What happened:
At 07:55 (GMT) on Sunday morning our technical team spotted a production issue in one of our applications that had been going since approximately 01:30 (GMT) that day. Because it initially only impacted a very small number of campaigns, the issue went unnoticed for a few hours before it started spreading and that’s when our team was able to spot it. What happened from a technical standpoint is that due to an unexpected estimated value returned by the Java Time library on duration computations, our scheduling mechanism entered in an infinite loop resulting in an important number of tasks (a task in this context is responsible for sending a portion of all notifications normally sent in a campaign) that our scheduler was not able to handle before the task was no longer valid. We consider a task invalid once its creation date is older than 30 minutes. Around 08:20 (GMT) we spotted the lines of codes that were responsible for this outage. Our team took the following steps to get things back to normal:
- 08:30 (GMT) : We isolated the application on a single computing node to ease-up the debug
- 10:27 (GMT) : We deployed a cleanup mechanism on the impacted campaigns.
- 10:40 (GMT) : Then we deployed a new release of our scheduling application to enable our customer to send campaigns.The newly created campaigns were then properly sent.
- 13:36 (GMT) : Lastly, after the impacted campaigns were cleaned-up, our engineering team released the fully fixed version of the application responsible for this outage. Scheduling and sending of the campaigns were working as expected.
Our team spent the rest of the day monitoring our infrastructure closely and attending to customers feedback. Today our Projects team is currently reaching out to customer accounts that may have been affected and our Tech team is having an internal meeting to investigate what happened even further, and make sure this incident can never occur again.
[2018-09-19] Resolved: Batch webservices downtime
[Update 21:11 GMT] Our webservices are back online.
[21:04 GMT] We have detected an issue impacting our webservices (dashboard + push delivery).
We will post an update here when it's fixed.
26th Sep, 2018
[2018-09-15] Resolved: GCM push delivery delays
[Update 14:40 GMT] The issue has been fixed.
[12:14 GMT] Due to recurring errors from Google Cloud Messaging, some notifications may be sent with a small delay.
15th Sep, 2018
[2018-09-06] Resolved: Push delivery delay
[Update 9:57 GMT] The issue has been fixed and caused a 5 minutes delay for all the notifications sent from 9:49 GMT.
[9:49 GMT] We have detected an issue causing delivery delays on iOS and Android. The issue is impacting notifications sent from the dashboard and our APIs.
6th Sep, 2018
[2018-09-04] Resolved: Push delivery delay
[Update 14:05 GMT] The issue has been fixed.
[13:49 GMT] We are aware of network issues that may impact push delivery on iOS and Android. Our team is monitoring our services closely.
4th Sep, 2018
[2018-08-29] Resolved: Batch dashboard/public APIs intermittent issues
[Update 6:45 GMT] Issue has been resolved.
[00:40 GMT] We have detected a network issue on servers handling homepage, dashboard and public APIs traffic. While still accessible, a high number of requests could timeout.
29th Aug, 2018
[2018-07-24] Resolved: Notifications delivery delay
[Update 14:45 GMT] The issue has been fixed. ~10% of all the scheduled campaigns and ~10% of campaigns sent without scheduling were impacted.
[13:45 GMT] We have detected an issue causing delivery delays on iOS and Android. The issue is exclusively impacting campaigns with targetings based on custom user data.
24th Jul, 2018
[2018-07-10] Resolved: GCM notifications delivery delay
[21:13 GMT] The situation has been improving since 21:03 GMT and we are currently not experiencing any issues.
[20:50 GMT] We are experiencing delivery issues for GCM notifications which in turn incurs some delays in delivery. Both campaigns and transactional notifications are impacted.
10th Jul, 2018
[2018-06-27] Resolved: Notifications delivery delay
[Update 17:24 GMT] The issue has been fixed.
[17:13 GMT] We are aware of an issue causing delivery delays on iOS and Android. We are currently working on a fix.
27th Jun, 2018
[2018-06-27] Resolved: Push delivery and data processing delay
[Update 14:53 GMT] The issue has been fixed.
[Update 14:08 GMT] Push delivery and data processing are progressively going back to normal. We are monitoring our systems closely.
[13:18 GMT] Due to a database issue on our end, most of the notifications sent from the dashboard or the Transactional API are delayed. Data processing is also affected. No data is lost and our team is working on a fix.
27th Jun, 2018
[2018-06-20] Resolved: Batch dashboard downtime
[Update 15:46 GMT] The issue has been fixed, the web dashboard is back online.
[15:35 GMT] We are aware of an issue affecting our web dashboard. We are working on a fix.
20th Jun, 2018
[2018-06-14] Resolved: Push delivery issues
[Update 17:21 GMT] The incident has been resolved. The delivery of some notifications may have been delayed.
[16:58 GMT] An issue is currently affecting the delivery of push notifications sent from the dashboard or the Campaigns API.
14th Jun, 2018
[2018-06-06] Resolved: Campaign creation issue
[Update 15:24 GMT] The issue is now fully resolved. We are monitoring our systems closely.
[Update 15:05 GMT] The campaign API and campaign creation/edition from the dashboard are down again for a few minutes.
[Update 14:57 GMT] The incident has been temporarily resolved, we are still working on a permanent fix.
[14:37 GMT] We are aware of an issue preventing our clients from creating new campaigns on the dashboard or from the API.
6th Jun, 2018
[2018-05-28] Resolved: Push delivery issue
[Update 21:04] The issue has been fixed.
[14:47 GMT] We have detected an issue causing duplicated push notifications on rare occasions. Our team is currently working on a fix.
28th May, 2018
[2018-05-17] Resolved: Campaigns delivery issue
[Update 14:50 GMT] The issue has been fixed.
[Update 14:28 GMT] The service is progressively going back to normal.
[13:39 GMT] An issue is currently impacting our campaigns schedulers. Some campaigns scheduled from the dashboard or the Campaigns API since May 17th, at 13:35 GMT, may not be delivered correctly.
17th May, 2018
[2018-05-09] Resolved: Notifications delivery delay
[Update 17:44 GMT] The issue has been fixed.
[17:39 GMT] We are aware of an issue causing delivery delays on iOS and Android. We are currently working on a fix.
9th May, 2018
[2018-04-19] Resolved: Transactional push delivery issue
[Update 16:42 GMT] The issue has been fixed. Some notifications have been sent with a delay of 20 minutes.
[16:30 GMT] An issue is currently affecting notifications sent through the Transactional API on iOS and Android. We are currently investigating the issue.
19th Apr, 2018
[2018-02-26] Resolved: Android notifications with TTL delivery issue
[22:37 GMT] Since around 16:20 GMT today we experienced problems delivering Android notifications which used a Time To Live (TTL). This was due to a misconfiguration while deploying a newer version of one of our microservices. Android notifications without a TTL were not affected. The incident has been fully resolved since 22:01 GMT.
26th Feb, 2018
[2017-12-07] Completed: Live chat maintenance
[Update 23:55 GMT] The maintenance is completed and the live chat module should be back on your web dashboard.
[08:22 GMT] We are upgrading our live chat software, the module will be removed from the web dashboard pages. The maintenance is expected to be completed around 19:00 GMT. You can still reach our support teams through email at support@batch.com today.
7th Dec, 2017
[2017-12-06] Resolved: iOS notifications delivery issue
[Update 23:19 GMT] The network latency to Apple's APNS servers is now back to normal. The incident is resolved but we'll continue to closely monitor the network in the coming days.
[Update 22:50 GMT] We are still suffering high latency between our servers and Apple's APNS API servers. We are still waiting for a full resolution from this hosting provider. Most iOS notifications are either sent on time or delayed, few might be still canceled.[Update 20:36 GMT] We are still experiencing difficulties with iOS push notifications. Contrary to Google's FCM/GCM APIs which have front-end servers across the globe, Apple's APNS API is only delivered through US-based infrastructures. The network link between our main infrastructures in France and Apple's servers in the US through London is causing very high latency. iOS notifications are delivered slowly or sometimes canceled because of network timeouts.[Update 19:57 GMT] The network issue has been identified to be an incident from one of our hosting providers. The network is back online after a 7 minutes downtime. Android push notifications are now delivered as expected but we are still experiencing timeouts and a very high latency with iOS push notifications. We are waiting for more informations from this provider.
[19:54 GMT] A network issue is apparently blocking our servers from communicating with APNS (Apple's Push API) and FCM/GCM (Google's Push API) servers. We are investigating the root cause.
6th Dec, 2017
[2017-11-28] Resolved: iOS notifications delays
[Update 20:09 GMT] The problem has been resolved and everything should be back to normal.
[20:01 GMT] iOS notifications through APNS are delayed because of a software error. We are working on a fix right now.
28th Nov, 2017
[2017-11-27] Resolved: Services downtime
[Update 08:36 GMT] The network incident has been fixed by this provider, all services including web dashboard should be restored. Delayed notifications will be send quickly during the next minutes if they were accepted by the system during downtime.
[08:18 GMT] Some Batch services are unavailable because of a network problem of our main hosting provider. This impacts the dashboard and push notifications delivery. We'll keep you informed on this page.
27th Nov, 2017
[2017-11-09] Resolved: Services downtime
[Update 09:46 GMT] Batch services are now back online. We are monitoring our services closely.
[Update 09:42 GMT] The service is progressively going back to normal.
[Update 09:25 GMT] Our main hosting provider just announced a resolution in 30 min. In the meantime, we are monitoring the network closely to see if our services go back to normal.
[Update 08:27 GMT] Batch dasbhoard and push deliery are still impacted by the downtime. We are still waiting for a full resolution ETA from our main hosting provider. Last update on their side identified a major network outage across Europe. Please note that data currently collected by the SDK are still being collected.
[07:09 GMT] Batch services are unavailable because of a major network problem of our main hosting provider. This impacts the dashboard and the push notifications delivery. We are monitoring closely the situation and waiting for a fix on their end.
9th Nov, 2017
[2017-10-25] Resolved: Transactional push delivery issue
[Update 11:25 GMT] The database issue is now fully resolved.
[Update 10:57 GMT] We identified a major database issue that affects Transactional push notifications delivery. We are working on a fix.
[10:37 GMT] We are aware of an issue affecting the delivery of notifications sent through the Transactional API, the incident started at 10:26 GMT. We are currently investigating the issue.
25th Oct, 2017
[2017-10-13] Resolved: Transactional push delivery issue
[Update 10:40] The issue has been fixed.
[10:12 GMT] An issue is currently affecting the delivery of some push notifications sent using the Transactional API. Our team is working on a fix.
13th Oct, 2017
[2017-10-02] Resolved: Transactional push delivery issue
[Update 10:16 GMT] The issue has been fixed and some notifications may have been delayed.
[9:49 GMT] We're aware of an issue affecting the delivery of transactional push notifications. Campaigns are still delivered without issue.
2nd Oct, 2017
[2017-09-30] Resolved: Transactional push delivery issue
[Update 11:40 GMT] The issue has been fixed. Some notifications sent with the transactional API may have been delayed or, in some cases, not delivered.
[10:29 GMT] We're currently experiencing an issue with our database impacting delivery of transactional push notifications. Campaigns are still delivered without issue.
30th Sep, 2017
[2017-09-20] Resolved: Push delivery delay
[Update 22h42 GMT] The problem has been resolved and everything should be back to normal.
[14:45 GMT] We are aware of an issue affecting the push delivery on iOS and Android. Some scheduled campaigns may be sent with a delay of ~5 minutes. We are currently working on a fix.
20th Sep, 2017
[2017-09-20] Resolved: Campaigns delivery issue
[Update 07:30 GMT] The campaigns delivery is going back to normal. Campaigns that weren't sent during the downtime will need to be rescheduled manually.
[06:13 GMT] We have detected an issue impacting our campaigns schedulers. Some campaigns scheduled from the dashboard or the Campaigns API since September 19th, at 16:00 GMT, may not be delivered correctly. Notifications sent from the Transactional API are not impacted. We will post updates here.
20th Sep, 2017
[2017-09-14] Resolved: User targeting issue
[Update 10:30 GMT] We rolled back the maintenance update and the issue has been fixed. It impacted up to 30% of the campaigns scheduled since September 12th.
[10:00 GMT] We are aware of an issue impacting the user targeting due to our last maintenance update. Some targeted users are not receiving push notifications. Our team is working on a fix.
14th Sep, 2017
[2017-09-11] Resolved: Push delivery delay
[Update 16:28 GMT] The issue has been resolved.
[Update 16:23 GMT] The service is progressively going back to normal.
[16:04 GMT] We are aware of an issue affecting the push delivery on iOS and Android. Some notifications may be sent with a delay of ~15 minutes. We are waiting for a fix on our hosting provider's side.
11th Sep, 2017
[2017-09-02] Resolved: Android Push Campaigns Notification delivery issue
[Update 14:00 GMT] The issue has been resolved.
[12:20 GMT] We noticed a problem regarding the processing of Android Push Campaigns since approx. 10:45 GMT: their error rates are unusually high. Transactional notifications and campaigns on other platforms are unaffected.
2nd Sep, 2017
[2017-08-11] Resolved: Push delivery and data processing delay
[Update 19:32 GMT] The issue is now fully fixed and everything should be working as expected.
[Update 18:35 GMT] The push delivery delay has been fixed, however we are still experiencing slowdowns regarding the processing of data sent by the SDKs.
[18:16 GMT] ~10% of the push notifications are currently being sent with an average delay of 20 minutes. Data processing is also affected. No data is lost, but changes such as push token registrations or custom user identifier changes might take some time to be reflected in our systems. We are working on a fix.
11th Aug, 2017
[2017-08-11] Resolved: Transactional API downtime
[Update 16:17 GMT] The problem has been resolved and everything should be back to normal.
[15:19 GMT] Several customers are reporting delivery issues with notifications sent from the Transactional API. The Transactional API is momentarily down for diagnostics.
11th Aug, 2017
[2017-08-11] Resolved: Transactional push delivery issue
[Update 12:50 GMT] The issue has been fixed. We are monitoring our services closely and some push notifications may be delayed.
[11:56 GMT] We are aware of an issue affecting push notifications sent from the Transactional API. Our team is investigating the issue.
11th Aug, 2017
[2017-07-19] Resolved: Transactional push delivery issue
[Update 23:10 GMT] The issue has been resolved. Transactional pushes created during the issue have been sent.
[22:05 GMT] We are experiencing an issue regarding the delivery of pushes sent using the Transactional API. Other APIs are unaffected. The team is currently working on a fix.
19th Jul, 2017
[2017-07-19] Resolved: Data processing delays
[Update 9:20 GMT] The issue has been resolved, and all data has been processed.
[6:10 GMT] We're currently experiencing slowdowns regarding the processing of data sent by the SDKs. No data is lost, but changes such as push token registrations or custom user identifier changes might take some time to be reflected in our systems.
19th Jul, 2017
[2017-07-10] Resolved: Push delivery delay (Transactional API)
[Update 11:54 GMT] The issue has been fixed. Some push notifications were sent with a 50 min delay.
[11:34 GMT] We are aware of an issue delaying the delivery of push notifications sent with the Transactional API. We are working on a fix.
10th Jul, 2017
[2017-06-27] Resolved: API and push delivery downtime
[Update 14:38 GMT] The problem has been resolved and everything should be back to normal.
[13:55 GMT] We are experiencing a downtime on the HTTP API and in our push delivery downtime. The team is currently working on a fix.
27th Jun, 2017
[2017-06-15] Resolved: Batch webservices downtime
[Update 11:22 GMT] Both the dashboard and the Campaigns API are back online.
[Update 10:54 GMT] The Transactional API is now back online. The dashboard and the Campaigns API should be back soon too.
[10:21 GMT] We are aware of an issue affecting our webservices (dashboard + API). iOS and Android push delivery is affected. An update will be posted here when we have more information.
15th Jun, 2017
[2017-06-14] Resolved: Maintenance rollback
[Update 08:10 GMT] The dashboard is now back online.
[08:00 GMT] We have identified an issue with our last routine maintenance causing a dashboard downtime. We are currently rolling back the maintenance.
14th Jun, 2017
[2017-06-13] Resolved: Dashboard campaigns / Campaigns API delivery issue
[Update 16:55 GMT] The issue is now fully resolved.
[Update 16:49 GMT] The service is progressively going back to normal. We are monitoring our services closely and some push notifications may be delayed.
[Update 16:19 GMT] We have identified the issue and we are working on a fix.
[16:06 GMT] Due to an issue causing duplicated push notifications, we are momentarily interrupting push delivery for campaigns sent from the dashboard or the Campaigns API. The Transactional API is not impacted. We will post updates here.
13th Jun, 2017
[2017-06-07] Resolved: Transactional API downtime (Android)
[Update 18:32 GMT] The issue has been fixed and the transactional API is now back online.
[14:30 GMT] An issue is currently causing a downtime of the Transactional API for Android apps. We are currently investigating the issue.
7th Jun, 2017
[2017-06-05] Resolved: API downtime
[Update 07:45 GMT] The API is now back online.
[07:37 GMT] We detected an issue with the API's, our team is investigating it. We will post an update here when it's fixed.
5th Jun, 2017
[2017-05-30] Resolved: API downtime
[Update 10:22 GMT] The API is now back online.
[10:19 GMT] We are aware of a downtime affecting our APIs. We will post an update here when it's fixed.
30th May, 2017
[2017-05-17] Resolved: Major dashboard and API downtime
[Update 18:45 GMT] This database incident is now fully fixed and everything should be working as expected.
[Update 17:42 GMT] On-call engineers will now leave our senior back-end teams handling this major database incident. We'll provide more informations here as soon as possible.
[Update 15:36 GMT] The issue is now affecting all our services including the web dashboard. We are still trying to understand what went wrong and we'll provide a fix as soon as we find the cause.
18th May, 2017
[2017-05-12] Dashboard downtime
[Update 15:32 GMT] The dashboard is back online.
[15:21 GMT] We have identified an issue that is causing causing downtimes of the dashboard. We are currently working on a fix.
18th May, 2017
[2017-05-09] Resolved: Dashboard and API downtime
[Update 12:53 GMT] The dashboard and the APIs are back online.
[12:40 GMT] We have identified an issue that is causing causing downtimes of the dashboard and our APIs. We are currently working on a fix.
9th May, 2017
[2017-05-04] Resolved: iOS push delivery issues
[Update 21:08 GMT] The issue looks to be fully resolved, we are monitoring closely.
[10:02 GMT] We are aware of an issue (increased timeouts between Batch and APNS) affecting our iOS push notifications delivery. Some ISPs are having problems right now with global interconnection between Europe (where our servers are based) and US (where Apple Push Notification Service endpoints are located). We are waiting for more informations from our datacenters providers.
5th May, 2017
[2017-04-14] Resolved: API downtime
[Update 4:56 GMT] The API is now back online.
[3:03 GMT] We are aware of a downtime affecting our APIs. This is a side effect of tests we run early this morning to be able to switch to a secondary infrastructure in case of a complete failure of our hosting provider. We will post an update here when it's fixed.
14th Apr, 2017
[2017-04-10] Resolved: API downtime
[Update 13:13 GMT] The API is now back online.
[12:49 GMT] A database issue is currently causing a downtime of our push APIs. We will post an update here when it's fixed.
10th Apr, 2017
[2017-03-16] Resolved: API downtime
[Update 1:45 GMT] The API is now back online.
[1:14 GMT] We are aware of connectivity problems to our Push API. We are working to resolve the issue.
16th Mar, 2017
[2017-02-28] Resolved: Dashboard and push delivery downtime
[Update 08:59 GMT] The Batch dashboard is back online. Push notifications scheduled during the downtime were delayed and no data has been lost.
[08:35 GMT] We are aware of an issue causing causing downtimes of the Batch dashboard and impacting the push delivery. We are currently working on a fix.
28th Feb, 2017
[2017-02-24] Resolved: Analytics/push delivery issues
[Update 2017-02-25 15:00 GMT] This database incident is now fully fixed and everything should be working as expected.
[Update 23:05 GMT] We are still working to resolve all the side effects of a major database incident. If you encounter any error with push delivery, please let us know by contacting the support team.
[Update 16:20 GMT] Most issues has been fixed, we are monitoring our services closely.
[Update 15:40 GMT] All our services are progressively going back to normal. We are still experiencing delays of 4-5 minutes. We are monitoring our systems to make sure the issue is fully resolved.
[Update 15:32 GMT] Our team is still working on a fix. Push notifications are currently being sent with an average delay of 30 minutes.
[Update 14:30 GMT] Our team has identified a major database incident and is currently working on a fix. Analytics and push delivery are directly affected. We will keep you updated here.
[13:29 GMT] We are aware of an issue affecting analytics and the push delivery. Our team is currently investigating the issue.
24th Feb, 2017
[2017-02-09] Completed: Scheduled dashboard maintenance
[Update 13:16 GMT] The scheduled maintenance was completed nominally.
[Update 13:10 GMT] The maintenance has now started.
[09:00 GMT] We will be performing a maintenance today at 13:10 GMT. The dashboard may be unavailable for a short period of time. Push APIs and senders shouldn't be affected.
[2017-02-08] Resolved: Dashboard downtime
[Update 18:12 GMT] The Batch dashboard is back online.
[18:00 GMT] A hardware error of our hosting provider is causing downtimes of the Batch dashboard, we are working to replace the impacted servers. APIs, push senders and SDK web-services are not impacted.
[2017-02-08] Resolved: Dashboard downtime
[Update 8:26 GMT] All services running smoothly. Our team is keeping a close eye on everything. The dashboard was down for 5 minutes.
[8:21 GMT] An error of our hosting provider is causing downtimes of Batch dashboard. We are waiting for a fix on their end.
[2017-01-06] Resolved: Services downtime
[Update 07:38 GMT] Batch services are back online, we are monitoring our softwares closely.
[07:21 GMT] Batch services are unavailable because of a major network problem of our main hosting provider. We are currently waiting for a fix on their end.
6th Jan, 2017
[2016-11-29] Completed: Scheduled maintenance
[Update 15:55 GMT] The scheduled maintenance was completed nominally.
[Update 15:46 GMT] The maintenance has now started.
[09:29 GMT] We will be performing a maintenance today at 15:40 GMT. The dashboard may be unavailable for a short period of time.
29th Nov, 2016
[2016-11-28] Resolved: Dashboard downtime
[Update 7:30 GMT] We have identified and fixed a web server issue that caused the downtime. Batch web-services, SDK and push notification senders were not affected.
[4:15 GMT] Batch dashboard is temporarily unavailable. Our team is investigating the issue.
28th Nov, 2016
[2016-11-21] Resolved: Batch webservices downtime
[Update 18:10 GMT] The network issue has now been resolved by our hosting provider. All systems are functioning normally and notifications are sent correctly. We are currently experiencing a delay in processing new custom data.
[Update 18:04 GMT] All our services are progressively going back to normal. We are monitoring our systems to make sure the issue is fully resolved.
[Update 17:40 GMT] Our team has identified the issue. A network problem of our hosting provider is affecting our webservices, APIs and the iOS/Android push delivery. We are waiting for a fix on their end and will keep you updated.
[17:28 GMT] We are aware of an issue affecting our webservices. iOS and Android push delivery may be affected. An update will be posted here when we have more information.
21st Nov, 2016
[2017-11-21] Completed: Scheduled maintenance
[Update 15:32 GMT] The scheduled maintenance was completed nominally.
[Update 15:16 GMT] The maintenance has now started.
[09:29 GMT] We will be performing a maintenance today at 15:15 GMT. Transactional push delivery might be delayed for some users. An update will be posted here when it is complete.
21st Nov, 2016
[2016-11-07] Completed: Maintenance rollback
[Update 11:36 GMT] Maintenance rollback is complete.
[11:05 GMT] We have identified an issue with our last routine maintenance. Some push notifications were delayed or not delivered on Android.
7th Nov, 2016
[2016-10-27] Completed: Scheduled maintenance
[Update 12:52 GMT] The scheduled maintenance was completed nominally.
[Update 2016-10-27 12:30 GMT] The maintenance has now started. We will post an update here once it's complete.
[15:30 GMT] We will be performing a maintenance of the dashboard on 28/10/2016 at 11:00 GMT. The dashboard may be unavailable for a short time. An update will be posted here when it is complete.
27th Oct, 2016
[2016-10-21] Resolved: Notifications analytics display & debug page downtime
[Update 15:45 GMT] The issue has been fixe. You can now use notifications analytics and the Debug page. No data were lost but we have a ~2 hours delay on analytics compute. Everything will be live again during the next hour.
[Update 15:09 GMT] The debug view in Settings > Debug is affected as well. We are working on a fix right now.
[13:35 GMT] We are aware of a display issue affecting campaigns & transactional notifications analytics since 13:04 GMT. No data has been lost and our team is working on a fix.
21st Oct, 2016
[2016-09-13] Resolved: Dashboard downtime
[12:11 GMT] One of our databases encountered an error due to an issue on our hosting provider's side and has now recovered. Dashboard and push were down for 7 minutes. Campaigns scheduled to be sent during the downtime have been delayed.
[12:04 GMT] Batch dashboard is temporarily unavailable. We are currently investigating the issue.
13th Sep, 2016
[2016-09-03] Resolved: Campaigns push delivery delay
[18:54 GMT] The issue has been fixed since 13:30 GMT approximately. Most affected campaigns were delayed between three to four hours at most. We monitored our systems since then and have reasons to believe the situation is back to normal.
[12:04 GMT] We are currently experiencing a delay with campaigns push delivery on iOS and Android. Our team is investigating the issue.
3rd Sep, 2016
[2016-08-30] Completed: Scheduled maintenance
[21:08 GMT] The scheduled maintenance was completed nominally.
[7:32 GMT] We will be performing a maintenance of the Transactional API today, at
21:00 GMT. The T
ransactional API will be unavailable for a few minutes. An update will be posted here when it is complete.
30th Aug, 2016
[2016-08-25] Resolved: API / Elevated error rates
[9:09 GMT] Our team managed to reproduce the issue and deployed a fix. We will be monitoring our systems closely during the next hours.
[9:00 GMT] Several customers are reporting sporadic API issues causing elevated error rates. We are currently investigating the issue.
25th Aug, 2016
[2016-08-24] Resolved: Analytics display issue
[13:45 GMT] The campaigns statistics have been recalculated and systems are running nominally.
[8:39 GMT] App statistics (DAU, installs, userbase stats) are now displayed correctly. Push campaigns statistics are beeing recalculated and should be available shortly.
[7:41 GMT] We are aware of a display issue affecting app and campaigns analytics since 00:30 GMT. No data has been lost and our team is working on a fix.
24th Aug, 2016
[2016-07-20] Resolved: Transactional push delivery delay
[10:10 GMT] The issue has been fixed. We observed a delay of 15 minutes for some transactional notifications sent during the incident.
[9:50 GMT] An issue on our hosting provider side may delay the delivery of iOS and Android transactional notifications. We will post an update here when it's fixed.
20th Jul, 2016
[2016-07-18] Completed: Scheduled maintenance
[2016-07-19 09:35] The maintenance is completed and our new website is now online.
[2016-07-19 09:00] The maintenance has started. We will post an update here when it's completed.
[14:57 GMT] Batch's website is getting a long overdue overhaul and as a consequence we’ve scheduled a Dashboard downtime of 20 minutes tomorrow,
from 09:00 GMT to 09:20 GMT. The push senders and delivery won't be affected. We will post an update here when the maintenance is completed.
18th Jul, 2016
[2016-07-06] Completed: Scheduled maintenance
[15:35 GMT] The scheduled maintenance was completed nominally.
[10:47 GMT] We will be performing a maintenance of our database on 06/07/2016 at 15:00 GMT. Transactional push delivery might be delayed for some users. An update will be posted here when it is complete.
6th Jul, 2016
[2016-07-04] Completed: Scheduled maintenance
[23:57 GMT] All our services are back to normal. Some push notifications were delayed for new users during the incident.
[18:49 GMT] Database indexing of new installs push tokens and new custom identifiers might be delayed for some users. Our team is investigating the issue. Other services (API, senders etc.) are running smoothly.
[18:39 GMT] The scheduled maintenance was completed nominally. We will be monitoring our systems closely during the next hours.
[8:17 GMT] We will be performing a maintenance of our database on 04/07/2016 at 14:50 GMT. Transactional push delivery might be delayed for some users. An update will be posted here when it is complete.
4th Jul, 2016
[2016-07-01] Solved: Delivery delays with the transactional API
[15:56 GMT] The issue has been fixed and no data has been lost.
[15:36 GMT] We are aware of delay issues affecting notifications sent with the transactional API. Our team is investigating the issue.
1st Jul, 2016
[2016-06-30] Resolved: Push delivery delay
[16:15 GMT] All our services are back to normal. Some push notifications were delayed during the incident.
[15:00 GMT] We are currently experiencing a delay with push delivery on iOS and Android. Our team is investigating the issue.
30th Jun, 2016
[2016-06-21] Resolved: Batch webservices downtime
[13:15 GMT] All services running smoothly. Our team is keeping a close eye on everything.
[12:13 GMT] All our services are progressively going back to normal. We are monitoring our systems to make sure the issue is fully resolved.
[11:42 GMT] An error of our hosting provider is causing downtimes of Batch webservices. We are waiting for a fix on their end.
21st Jun, 2016
[2016-06-09] Resolved: Delay in data processing
[14:16 GMT]
The issue has been fixed. Custom IDs and user's data are now processed in real time.
[2016-06-10 13:44 GMT] The cause of the issue has been identified and a fix will be released in a few hours.
[2016-06-09 10:34 GMT] We are aware of a ~2 hours delay in processing users' data and campaign analytics. Our team is currently investigating the issue. No data has been lost and push delivery has not been affected. We will post updates here.
9th Jun, 2016
[2016-05-31] Resolved: Batch webservices downtime
[08:13 GMT] All our services are progressively going back to normal. We are monitoring our systems to make sure the issue is fully resolved.
[08:03 GMT] An error of our hosting provider is causing downtimes of Batch webservices. We are waiting for a fix on their end.
31st May, 2016
[2016-05-20] Completed: Scheduled maintenance
[23/05/2016 14:12 GMT] The scheduled maintenance was completed nominally.
[10:26 GMT] We will be performing a maintenance of our database on 23/05/2016 at 14:00 GMT. The push delivery will be interrupted for a few minutes. An update will be posted here when it is complete.
20th May, 2016
[2016-05-18] Completed: Scheduled maintenance
[10:05 GMT] The scheduled maintenance was completed nominally.
[7:30 GMT] We will be performing a maintenance of our database at 9:51 GMT. The push testing will be unavailable during the maintenance. An update will be posted here when it is complete.
18th May, 2016
[2016-05-10] Completed: Scheduled maintenance
[16:54 GMT] The scheduled maintenance was completed nominally.
[14:30 GMT] We will be performing a maintenance of our database at
16:45 GMT. The push delivery and our webservices may be unavailable for a few minutes. An update will be posted here when it is complete.
10th May, 2016
[2016-05-04] Resolved: Batch webservices downtime
[8:10 GMT] All our services are now up and running. Network issues at our hosting provider impacted the delivery of notifications. All notifications sent during the downtime were delayed and some have been lost.
[7:51 GMT] All our services are progressively going back to normal. We are monitoring our systems to make sure the issue is fully resolved.
[7:08 GMT] We're experiencing a service issue and are currently investigating. Our team is investigating the issue.
4th May, 2016
[2016-04-28] Resolved: Push delivery issues
[17:39 GMT] The issue has been fixed.
We have detected sporadic push delivery issues resulting in an increased rate of errors. We are currently investigating.
28th Apr, 2016
[2016-04-19] Completed: Scheduled maintenance
[15:13 GMT] The scheduled maintenance was completed nominally.
[14:51 GMT] We will be performing a maintenance of our database. An update will be posted here when it is complete.
19th Apr, 2016
[2016-04-12] Resolved: Analytics & audience estimate issue
[15:10 GMT] The issue has been fixed and no data has been lost.
[14:58 GMT] We’ve identified an issue that is affecting the Analytics and audience estimate. We are working on a fix.
12th Apr, 2016
[2016-04-11] Resolved: Test push delivery issue (iOS)
[09:00 GMT] Apple has fixed the issue on their side. You can now send test normally. Let is know if you are still seeing errors.
[14:00 GMT] We are aware of an issue affecting test notifications sent to iOS devices. Our team is investigating the issue.
12th Apr, 2016
[2016-04-02] Resolved: Transactional push delivery errors
[10:04 GMT] The issue has been fixed and all new transactional push notifications are delivered. All notifications sent through the Transactional API from yesterday 16:00 GMT to 10:00 GMT this morning might have been lost. We will keep monitoring the system.
[09:00 GMT] We are aware of a major issue with transactional push notifications. We are currently investigating the issue.
2nd Apr, 2016
[2016-04-01] Resolved: Push delivery delay
[16:32 GMT] The issue has been fixed and our systems are now functioning as normal.
[16:10 GMT] We are aware of an issue that is causing delayed delivery of push notifications. We are currently investigating the issue.
1st Apr, 2016
[2016-03-25] Resolved: Campaigns data calculation error
[9:30 GMT] Due to a maintenance issue, direct opens stats are not available for iOS campaigns sent between 15:30 GMT on 24/03/2016 and 9:30 GMT on 25/03/2016. The issue has been fixed and systems are now running nominally.
25th Mar, 2016
[2016-03-13] Resolved: Push delivery delay
[15:00 GMT] All our services are progressively going back to normal. We are monitoring our systems to make sure the issue is fully resolved.
[10:20 GMT] We are currently experiencing a delay with push delivery on iOS and Android. We are working on a fix.
[2016-03-07] Resolved: Batch webservices downtime
[16:16 GMT] All our services are progressively going back to normal. We will keep monitoring the system.
[15:12 GMT] An error of our hosting provider is causing sporadic downtimes of Batch webservices. We are waiting for a fix on their end.
7th Mar, 2016
[2016-02-11] Resolved: iOS delivery issue
[11:30 GMT] The issue has been fixed and our systems are now functioning as normal.
[10:45 GMT] An issue is currently delaying the delivery of 12% of the iOS notifications. We are working on a fix.
26th Feb, 2016
[2016-02-11] Resolved: Batch webservices downtime
[16:09 GMT] All our services are now up and running.
[15:59 GMT] An error of our hosting provider is causing sporadic downtimes of Batch webservices. We are waiting for a fix on their end.
11th Feb, 2016
[2016-02-10] Completed: Maintenance rollback
[9:44 GMT] Maintenance rollback is complete, we are monitoring our systems to make sure the issue is fully resolved.
[9:20 GMT] We have identified an issue with our last routine maintenance. Some push notifications were delayed or not delivered on iOS. A rollback is in progress.
10th Feb, 2016
[2016-02-04] Completed: Scheduled maintenance
[12:08 GMT] The scheduled maintenance was completed nominally. Batch APIs and webservices are now restored to normal function. The dashboard was down for 28 minutes and campaigns delivery was affected for 5 minutes. The transactional push delivery was not impacted.
[11:39 GMT] We will be performing a maintenance that will interrupt Batch APIs and webservices for a couple of minutes. An update will be posted here when the maintenance is complete.
4th Feb, 2016
[30-11-2015] Resolved: Dashboard downtime
[12:21 GMT] One of our databases encountered an error after a maintenance. Access to the dashboard was down during this time. Batch web-services and SDK were not affected.
[12:00 GMT] Batch home page and dashboard are temporarily unavailable. We are currently investigating the issue.
30th Nov, 2015
[2015-11-28] Resolved: Dashboard downtime
[5:50 GMT - 9:10 GMT] One of our databases encountered an error and has now recovered. Access to the dashboard was down during this time. Campaigns scheduled to be sent during the downtime have been delayed or cancelled if the delay was superior to two hours.
28th Nov, 2015
[2015-11-25] Resolved: Campaign analytics data calculation error
[12:44 GMT] The statistics have been recalculated and systems are running nominally.
[08:40 GMT] We are aware of an issue affecting Engagement Analytics and the native attributes push estimates. The processes have been reset and are currently generating the missing calculations from intact data.
25th Nov, 2015
[2015-10-22] Completed: Scheduled maintenance
[Update 10:40 GMT] The scheduled maintenance was completed nominally, the dashboard and the REST API restored to normal function.
[7:30 GMT] Batch's dashboard will be unavailable from 10:25 GMT to 10:40 GMT today due to a scheduled maintenance. An update will be posted here when the maintenance is complete.
22nd Oct, 2015
[2015-10-19] Resolved: Batch downtime
[7:53 GMT] An error of our hosting provider caused sporadic downtimes of Batch webservices, from 00:15 GMT to 2:20 GMT. All our services are now up and running and no data has been lost.
19th Oct, 2015
[2015-10-08] Completed: Scheduled maintenance
[Update 15:22 GMT] The scheduled maintenance was completed nominally.
[15:22 GMT] We will be performing a scheduled maintenance that will interrupt Batch webservices for a couple of minutes. An update will be posted here when the maintenance is complete.
8th Oct, 2015
[2015-09-10] Completed: Scheduled Maintenance
[Update 15:05 GMT] The scheduled maintenance was completed nominally, the REST API restored to normal function.
[15:00 GMT] We will be performing a scheduled REST API maintenance that will temporarily interrupt Batch API services. An update will be posted here when the maintenance is complete.
10th Sep, 2015
[2015-08-24] Completed: Scheduled Maintenance
[Update 18:40 GMT] The scheduled maintenance was completed nominally, Batch.com is restored to normal function.
[18:00 GMT] We will be performing a scheduled push tokens migration in 15 minutes (18:15 GMT 2015/08/25) that will temporarily interrupt Batch services. An update will be posted here when the maintenance is complete.
24th Aug, 2015
[2015-08-13] Resolved: Batch downtime
[13/08/2015 22:02 GMT] Due to an error of our hosting provider, Batch SDK and web-services were down from 22:02 GMT to 23:34 GMT. All our services are now up and running and no data has been lost.
13th Aug, 2015
[2015-08-12] Resolved: Campaign analytics data calculation error
[9:39 GMT] The push campaigns statistics have been recalculated and systems are running nominally.
[8:30 GMT] We are aware of an issue affecting the calculation of Batch Push statistics for Campaigns created on August 11th. The processes have been reset and are currently generating the missing calculations from intact data. Rest assured, all push notifications were sent successfully. Full restoration of push campaigns stats is expected by 12:00 GMT.
12th Aug, 2015
[2015-08-03] Resolved: Campaign edition issue
[15:00 GMT] The issue has been fixed and campaigns can now be edited.
[13:30 GMT] We’ve identified an issue that prevents users from editing Unlock and Push campaigns. We are currently investigating the issue.
3rd Aug, 2015
[2015-04-16] Completed: Scheduled Maintenance
[Update 09:06 GMT] The scheduled maintenance was completed nominally, Batch.com is restored to normal function. No Batch services were impacted during the maintenance.
[08:30 GMT] We will be performing front-end database maintenance in 30 minutes (09:00 GMT 2015/04/16) that will take the site down and display a temporary maintenance page for 10-15 minutes. No Batch processes will be affected and all apps using Batch will maintain full connection with the Batch system. An update will be posted here when the duration is complete.
16th Apr, 2015
[2015-04-10] Resolved: Analytics data calculation error
[Update 14:00 GMT] The Userbase and Engagement statistics have been recalculated and systems are running nominally.
[08:00 GMT] We were alerted to processes that were not accurately computing Userbase and Engagement statistics for Batch Analytics for the past 4 days. The processes have been reset and are currently generating the missing calculations from intact data. Full restoration of Analytics data is expected by 14:00 GMT. Before this time you may notice artificially higher than usual starts, DAUs, and engaged users reflected on the dashboard.
10th Apr, 2015
[2015-03-31] Resolved: Dashboard downtime
[Update 23:23 GMT] One of our databases encountered an error and has now recovered. Access to the dashboard was down during this time. Batch web-services and SDK were not affected.
[Update 22:40 GMT] We’ve identified an issue with one of our database servers that we’re working on resolving. SDK web-services are not impacted.
[21:45 GMT] We are aware of a major outage of the Batch.com dashboard. We are currently investigating the issue.
31st Mar, 2015
[2015-03-26] Resolved: Ads downtime on iOS
[Update 20:35 GMT] A very specific ad configuration was causing a crash on iOS when displayed because of a JSON format error. We fixed the problem server-side, which won’t require an SDK update. We apologize for any inconvenience.
[18:20 GMT] We are aware of a crash on the iOS SDK when displaying an ad with Batch. We are currently investigating the issue.
26th Mar, 2015