Cronofy Status Page

Jul 7, 2026 · resolved Jul 7

Between approximately 02:25 UTC on July 1st and 10:11 UTC on July 7th 2026, scheduled notifications in our US data center failed to send. The affected notifications were queued but never picked up, and were flushed once the underlying cause was cleared. The impact was limited to customers in the US data center using notification-driven API features. Approximately 9,000 Event Trigger notifications for calendar events starting and ending were delivered late. A small number of Scheduler Workflows lifecycle notifications were also delivered late, and a small number of scheduled Meeting Agents did not arrive to their meetings. The root cause was a regression introduced on April 22nd, in an internal service responsible for sending scheduled notifications. The service uses leader election so that a single process handles distributing the imminent notifications at any one time, coordinated through a lock. The regression made that configuration fragile. If a leader did not release the lock cleanly, no other process would override it. When the leader stopped in the early hours of July 1st, it left the lock held, and with no process able to take over, notifications quietly backed up from that point. Nothing alerted us to this. The failure mode was an absence of activity rather than an error, and we had no alerting on notification throughput that would have caught the drop-off. During the incident, we manually cleared the stuck lock, which allowed the service to resume and flush the backlog within four minutes. ## Timeline All times are in UTC. * **April 22nd, 10:27** A change is merged that makes the leader election's lock-holding fragile. The fault is latent and has no immediate effect. * **July 1st, 02:25** The leader process in the US stops in a way that invokes the regression, leaving the lock held. No notifications are picked up from this point. * **July 2nd, 06:42** A support ticket reports an issue with Event Triggers. * **July 6th, 14:53** A second support ticket reports an issue with Event Triggers. * **July 7th, 09:38** The pattern across the two tickets is identified internally and raised. * **July 7th, 09:50** Telemetry is reviewed, showing a clear drop-off in notification sends. * **July 7th, 09:57** An internal incident channel is opened. * **July 7th, 10:05** The held lock is raised as the probable cause. * **July 7th, 10:07** The lock is manually cleared. * **July 7th, 10:07-10:11** The backlog of late notifications is flushed by the now-working service. * **July 7th, 10:12** The public incident is opened. * **July 7th, 10:40** Impact is reviewed and confirmed, including the other affected paths \(Scheduler Workflows, Meeting Agents\). * **July 7th, 11:12** The public incident is resolved. ## Retrospective We always ask the same three questions. ### Could the issue have been resolved sooner? Once we were looking at the right thing, resolution was fast. The probable cause was raised within eight minutes of opening the channel, and the backlog was flushed four minutes after that. We are satisfied with how the resolution itself went. ### Could the issue have been identified sooner? This is where we fell short, and badly. The failure was live for almost six days. Two things let it run that long. First, we had no alerting on notification throughput, so a complete stop in sending produced no signal. It is harder to alert on things _not_ happening, but that is exactly the case this warranted. Second, the first support ticket arrived on July 2nd and did not surface the pattern. It took a second ticket, four days later, for the connection to be made. Six days is too long for a customer-facing feature to be silently failing. ### Could the issue have been prevented? The regression was introduced in April and sat latent until a leader failed to shut down cleanly for the first time. The fragility was in the lock's release path, the case where a leader stops without releasing cleanly, which is precisely the kind of edge that is easy to miss in review and under-covered by tests. We did not catch it at either stage. ## Actions We have added alerting on notification throughput, so that a drop-off in sends is positively and quickly identifiable rather than depending on customer reports. We have fixed the code so that a lock held by a stopped leader is reliably released and picked up by another process, rather than leaving the work stuck. We have added tests around the configuration to ensure we cannot fall foul of the misconfiguration by accident, and integration tests to show that an orphaned lock can be obtained by another process. We are reviewing how we handle support tickets that describe an absence of expected behavior, so that a single ticket of this kind prompts a check of the relevant telemetry sooner.

Jun 16, 2026 · resolved Jun 16

The mitigation has been completed. Increased API errors were observed from 09:30:30 to 09:31:30 UTC.

Jun 3, 2026 · resolved Jun 4

Google Meet Meeting Agents are attempting meeting joins successfully again after our further fix earlier today. We shall continue to monitor but consider this incident to be closed.

Jun 2, 2026 · resolved Jun 2

Events continue to sync events into Google Calendars as normal. We are still seeing an elevated number of errors between Cronofy and Google Calendar, but are confident that our system are keeping events in sync correctly. The vast majority of events are still syncing almost immediately, while a minority of updates sync after a few seconds. This should have no impact on customers applications, so we are resolving the incident. We will continue to monitor the situation and will raise a new incident if performance declines to a noticeable level.

May 14, 2026 · resolved May 14

Between 03:08 UTC and 03:10 UTC we observed increased API error rates in our US data center. During this time, customers may have received 500 Internal Server Error responses. Since that time, error rates have returned to normal levels. These errors were caused by a read-only database server going through an automated recovery process.

Is Cronofy down right now?

No — Cronofy is up. All systems operational as of Jul 13, 11:44 PM UTC.

What is Cronofy's current status?

Cronofy: All Systems Operational. Alert24 checks Cronofy's status page continuously and can notify you the moment it changes.

How do I get alerted when Cronofy goes down?

Alert24 monitors Cronofy and 3,700+ other cloud and SaaS providers. When an outage is detected it updates your status page automatically and pages your on-call team — no manual checks. Start free at alert24.net.