Honeycomb Status Page

Jul 2, 2026 · resolved Jul 3

This incident has been resolved.

Jun 29, 2026 · resolved Jun 30

Backfill of the Activity Log events has completed. All events during the affected period should be present.

Jun 25, 2026 · resolved Jun 25

This incident has been resolved.

Jun 24, 2026 · resolved Jun 24

On June 24, we experienced approximately 30 minutes of severe data ingestion degradation in Honeycomb’s US and EU instances. During this degradation, Honeycomb rejected a significant percentage of inbound telemetry, and customers would have seen up to a half-hour gap in ingested telemetry. Additionally, during this same time window, customers would have experienced data processing degradation and service disruption within Anomaly Detection and Service Maps. A deployment containing a change to how services retrieve dataset schemas into local caches caused a cascading set of failures, resulting in failed remote cache retrievals, and subsequently, a MySQL stampede due to several services falling back to retrieving schemas from the database. This stampede resulted in significant memory utilization growth, causing some services to crash loop with out-of-memory exceptions while retrieving schemas. Customers would have seen 5xx errors from our ingestion APIs when this occurred, because ingestion services were among those that were crash looping. This also included crash looping from services responsible for Anomaly Detection and Service Maps. Right after these service crash loops began, alerts fired for the crashing services, and procedures were run to pin all services back to the last known good build before the breaking change landed. Once services were running the previous deploy’s build, the 5xx error responses from ingestion APIs returned to nominal baseline levels, and services that were crash looping recovered and resumed normal operation. The underlying issue centered on how dataset schema cache payloads and metadata were serialized to memcached for remote cache retrieval, and how memcached clients deserialize that data before writing it to a local cache. The code change contained a feature flag to toggle the serialization behavior, and backwards-compatibility safeguards for clients deserializing different schema versions from memcached. However, services’ schema deserialization prior to the feature flag flip on did not behave as intended, treating each backwards-compatible schema read as a hard error rather than as a fallback, which subsequently caused every remote schema cache read to fall through to the database. Subsequent analysis and review of the code identified the issue and we have re-deployed this change with a fix along with additional tests. Services now correctly and successfully deserialize schemas from memcached in a safe, backwards-compatible manner.

Jun 19, 2026 · resolved Jun 19

MCP Write Tools have been restored. MCP users may need to reconnect to see all available tools.

Is Honeycomb down right now?

No — Honeycomb is up. All systems operational as of Jul 13, 11:46 PM UTC.

What is Honeycomb's current status?

Honeycomb: All Systems Operational. Alert24 checks Honeycomb's status page continuously and can notify you the moment it changes.

How do I get alerted when Honeycomb goes down?

Alert24 monitors Honeycomb and 3,700+ other cloud and SaaS providers. When an outage is detected it updates your status page automatically and pages your on-call team — no manual checks. Start free at alert24.net.