unico Status Page

Aug 30, 2026 · resolved Aug 30

**Summary** In the early hours of August 29–30, 2026, between 23:00 and 23:47 \(Brasília time\), process creation on the platform became unavailable, resulting in 503 errors for all users attempting to access the service. The incident lasted approximately 47 minutes and affected 91 user groups, with a direct impact on more than 63,000 requests. **Impact** The success rate for the process creation flow dropped to 67.86% against a 99% availability target. Of the 196,884 requests during the period, 63,273 resulted in failure. The highest volume of errors was recorded among users of sports betting platforms, which operate with high traffic volumes especially during live matches. In addition, other user groups — including financial and healthcare services — were affected to a lesser extent. **Root Cause** The incident was caused by a design flaw in the interaction between two automated database maintenance processes, compounded by high traffic volume at the time of the event. The database was running a mandatory internal maintenance operation on the main process table. This type of maintenance is non-interruptible and holds an exclusive lock on the table. At the same time, a scheduled partition creation job on another table — which has a foreign key relationship with the process table — attempted to acquire a lock on the same main table. Because the database uses a lock queue, the partition job waited indefinitely \(with no timeout configured\), causing all subsequent write operations to also queue up. This accumulation exhausted the service's available connection limit, causing new connections to be rejected. Since application pods perform a database connectivity check on startup and fail immediately when unable to connect — with no retry mechanism — 99 out of 100 pods entered a continuous restart loop, rendering the service completely unavailable at the edge layer. **Resolution** The mitigation was precise and surgical: only the blocked partition job session was terminated. Immediately after, the accumulated connections were released — dropping from ~1,500 to 19 within seconds — the pods returned to normal operation, and the availability indicator reached 100% at 23:48. **Lessons Learned** * The scheduled maintenance job had no lock acquisition timeout configured. A timeout of just a few seconds would have been enough for the job to fail gracefully and be rescheduled in the next cycle, preventing the cascading failure. * The foreign key relationship between the partition table and the main process table was identified as the architectural link that connected a low-impact maintenance job to an extremely high-write table. This relationship was removed as part of the fixes applied after the incident.

Aug 10, 2026 · resolved Aug 10

**Summary** On August 10, 2026, between 5:08 PM and 5:27 PM \(Brasília time\), the availability indicator for the 1:N facial recognition flow dropped below 95%, with a total impact of approximately 19 minutes. The incident was caused by saturation of the vector search database used by the biometric engine, resulting from unexpected behavior in the queue consumption mechanism, which generated a call volume 2.36 times higher than configured. The excess load reached the database's maximum scaling limit, causing cascading timeouts across multiple dependent services and affecting 55 client users with errors during the period. **Impact** Users of the facial recognition, authentication, and process creation flows from multiple clients experienced errors and timeouts during the period. The database saturation cascaded through the biometric orchestration, engine, and process creation services. Mitigation was carried out by disabling an internal service and increasing the database's maximum scaling limit, stabilizing the service at 5:27 PM. **Root Cause** The root cause was a combination of factors related to service behavior and the capacity of the infrastructure involved. The service was configured with a limit of 2 requests per second, but the queue consumption mechanism behaved unexpectedly: messages whose processing exceeded the acknowledgment deadline were redelivered by the queue without canceling the original execution — turning 938 work items into 2,217 effective calls \(~8.4 RPS\). This amplified traffic reached the database without passing through the edge protection layer, which would normally limit request volume, since the network path used in this flow did not go through those controls. The database, already operating close to its maximum scaling limit, was unable to absorb the additional load — its CPU reached 88% while scaling up to the configured ceiling of 100 nodes — and query latency jumped from ~3 seconds to ~18.75 seconds at p90, propagating timeouts across the entire chain of dependent services. **Resolution** The team identified the process in question as the source of the load within a few minutes of the alert. The process was disabled, and the database's maximum scaling limit was increased from 100 to 150 nodes. With these two actions, the service stabilized at 5:27 PM. **Lessons Learned** The incident showed that rate limiting applied only on the message publishing side is insufficient when the consumer can amplify volume through redelivery. Rate and concurrency controls need to exist on the consumption side as well. Additionally, internal services that access critical components through paths that bypass edge protections represent a structural risk — any internal client can generate uncontrolled load on shared infrastructure.

Jul 31, 2026 · resolved Jul 31

Prezados clientes, Informamos que o incidente relacionado à exibição de envios no painel de acompanhamento foi oficialmente encerrado. Após a aplicação da correção e o período de monitoramento, confirmamos que a listagem e o histórico de documentos foram totalmente restabelecidos, e a plataforma opera em completa estabilidade. Agradecemos pela compreensão e paciência de todos durante o processo. Atenciosamente,

Jul 29, 2026 · resolved Jul 29

**Summary** On July 29, 2026, between 8:32 AM and 10:05 AM \(Brasília time\), the webhook delivery service was degraded for approximately 1 hour and 33 minutes. The incident was caused by a planned infrastructure migration that created a transient incompatibility state: after internal routes were switched to the new cluster, the webhook service began generating authentication tokens from the new cluster, but deliveries were still being directed to the old cluster — which rejected the tokens with an authorization error. Lost events were reprocessed on demand. **Impact** Multiple products were affected simultaneously — including authentication, payments, and onboarding flows. Users of clients that relied exclusively on webhook notifications to continue their business flows experienced operational interruptions during the period. Clients that had implemented active polling as a fallback mechanism experienced reduced impact. **Root Cause** The root cause was identified in the migration planning: the procedure did not map the chained dependency of the webhook flow — where the internal route creates an asynchronous task that, in turn, calls the public route for delivery. By migrating the internal routes and the public routes in separate steps, an incompatibility window was created where tokens generated by the new cluster were rejected by the old cluster's authorization policy, which specifically validated the previous service account identifier. This scenario was not identified during the change's risk assessment, and none of the existing service quality indicators covered the asynchronous delivery layer where the failure materialized. **Resolution** The team identified the token incompatibility as the source of the failures and completed the migration of the public routes to the new cluster, eliminating the divergence. From that point on, the pending deliveries in the queue began processing normally. The impact ceased at 10:05 AM, and traffic on the old cluster reached zero at 10:32 AM. **Lessons Learned** The incident showed that migrations involving chained asynchronous delivery flows require all routes participating in the chain — internal and public — to be migrated in a coordinated manner, without incompatibility windows between clusters. The gradual approach used, while suitable for most migrations, created a prolonged failure period here that could have been avoided with an atomic transition or a much shorter transition window. Additionally, the lack of observability at the asynchronous delivery layer was the reason the incident was not detected internally — existing indicators measured only event acceptance, not actual delivery. Creating dedicated alerts for failures and backlog at this layer is the most urgent monitoring improvement identified.

Jul 24, 2026 · resolved Jul 24

**Summary** On July 24, 2026, between 6:05 PM and 6:29 PM \(Brasília time\), the webhook creation and delivery service was unavailable for approximately 24 minutes. The incident was caused by the loss of access to an infrastructure service account during a resource decommissioning process linked to a cluster migration carried out days earlier. The affected account was shared by multiple services without this dependency being documented, causing the webhook service to lose its authentication capability and stop creating and delivering events. **Impact** Webhook event creation and delivery was interrupted during the period, affecting 18 client users with more than 1,550 errors recorded. The impact cascaded to other services that depended on the same authentication flow, causing simultaneous degradation across different platform capabilities. **Root Cause** The root cause was identified in the service dependency configuration. This cross-service dependency was neither documented nor visible in any dependency mapping. When the decommissioning process removed the account as part of the original service's resource cleanup, there was no automated validation to check whether the account was still being used by other active services. The absence of this safeguard kept the dependency hidden until its removal caused widespread authentication failures. **Resolution** The team quickly identified the unstable service account as the source of the authentication failures. The account was manually adjusted and applied to the cluster, restoring the service at 6:29 PM. The total impact duration was approximately 24 minutes. **Lessons Learned** The incident exposed the risks of legacy configurations that create hidden dependencies between services: a single resource change during planned maintenance was enough to disrupt multiple services simultaneously. The infrastructure resource decommissioning process must include automated dependency validation before any change.

Is unico down right now?

No — unico is up. All systems operational as of Oct 12, 7:42 AM UTC.

What is unico's current status?

unico: All Systems Operational. Alert24 checks unico's status page continuously and can notify you the moment it changes.

How do I get alerted when unico goes down?

Alert24 monitors unico and 3,700+ other cloud and SaaS providers. When an outage is detected it updates your status page automatically and pages your on-call team — no manual checks. Start free at alert24.net.