How to Correlate Grafana Alerts with Recent Deployments

The Question That Derails Every War Room

An alert fires at 2 AM. Error rate is up, latency is climbing, and someone on the call asks the inevitable question: "Did anything deploy in the last hour?" Then the conversation stalls while someone opens GitHub, filters the deployment history, checks Slack for deploy notifications, and tries to reconstruct a timeline from three different sources.

That delay is not a people problem. It is an architecture problem. The deployment context that would answer the question in five seconds lives in a completely different system from the alert that surfaced the incident. This post shows you how to close that gap.

Why Grafana Alone Cannot Answer the Question

Grafana is excellent at telling you that something is wrong and approximately when it started. Panels, annotations, and alert rules give you metric-level visibility. What Grafana does not have is awareness of what changed in your environment at the infrastructure or application level — deployments, config pushes, feature flag flips, database migrations.

You can add annotations manually or via the Grafana API, and some teams do. The problem is that it requires discipline from every team that ships code, and it couples your alerting data model to your deployment process in a way that is fragile and hard to maintain across multiple repos or services.

The cleaner approach is to log deployment events to a dedicated change log that is also visible at the moment an incident opens. When your incident management layer can surface "three services deployed in the last 90 minutes" on the same screen as the Grafana alert that triggered the incident, you stop switching tabs and start debugging.

Setting Up Deployment Logging from CI/CD

Alert24's change log accepts events via a straightforward API call. You send a payload describing what changed, and that event becomes part of the timeline for any incidents that open within a configurable window after the change.

The payload structure is simple:

{
  "summary": "Deployed payments-service v2.4.1 to production",
  "environment": "production",
  "service": "payments-service",
  "version": "v2.4.1",
  "triggered_by": "github-actions",
  "commit_sha": "a3f9c12",
  "commit_url": "https://github.com/your-org/payments-service/commit/a3f9c12"
}

You POST this to https://app.alert24.io/api/v1/changes with your API key in the Authorization header. The event is timestamped server-side at receipt.

GitHub Actions Integration

Add a step at the end of your deployment job, after the deploy itself succeeds:

- name: Log deployment to Alert24
  if: success()
  run: |
    curl -s -X POST https://app.alert24.io/api/v1/changes \
      -H "Authorization: Bearer ${{ secrets.ALERT24_API_KEY }}" \
      -H "Content-Type: application/json" \
      -d "{
        \"summary\": \"Deployed ${{ github.repository }} to production\",
        \"environment\": \"production\",
        \"service\": \"${{ github.event.repository.name }}\",
        \"version\": \"${{ github.sha }}\",
        \"triggered_by\": \"github-actions\",
        \"commit_sha\": \"${{ github.sha }}\",
        \"commit_url\": \"${{ github.event.head_commit.url }}\"
      }"

Store ALERT24_API_KEY as a repository or organization secret. If you are deploying multiple environments from the same workflow, parameterize the environment field using the job matrix or environment name variable.

GitLab CI Integration

The equivalent in GitLab CI belongs in your deploy stage, after the deployment commands complete:

log-deployment:
  stage: deploy
  script:
    - |
      curl -s -X POST https://app.alert24.io/api/v1/changes \
        -H "Authorization: Bearer $ALERT24_API_KEY" \
        -H "Content-Type: application/json" \
        -d "{
          \"summary\": \"Deployed $CI_PROJECT_NAME to production\",
          \"environment\": \"production\",
          \"service\": \"$CI_PROJECT_NAME\",
          \"version\": \"$CI_COMMIT_SHA\",
          \"triggered_by\": \"gitlab-ci\",
          \"commit_sha\": \"$CI_COMMIT_SHA\",
          \"commit_url\": \"$CI_PROJECT_URL/-/commit/$CI_COMMIT_SHA\"
        }"
  when: on_success
  only:
    - main

GitLab's built-in CI variables give you everything you need. $ALERT24_API_KEY should be a protected CI/CD variable scoped to your production branch.

Connecting Grafana Alerts to Alert24 Incidents

With deployments flowing into the change log, the next step is routing Grafana alerts through Alert24 so that incidents carry the full context automatically.

Grafana supports contact points using a generic webhook. In your Grafana instance, navigate to Alerting > Contact Points and create a new contact point of type Webhook. Set the URL to your Alert24 Grafana ingest endpoint:

https://app.alert24.io/ingest/grafana/{your-integration-key}

Alert24 parses Grafana's native alert payload, extracts the alert name, severity labels, and firing time, and opens an incident. At incident creation time, the system queries the change log for events in the preceding window — by default, 60 minutes — and attaches them to the incident timeline.

The result is that the first thing your on-call engineer sees when they open the incident is a timeline that looks like this:

The deployment events appear before the alert because they happened before the alert. The timeline is built from real timestamps, not from memory or manual reconstruction.

What This Changes About How You Debug

When the change log and the incident timeline are unified, a few things happen that seem small but compound over time.

Your first question in a war room shifts from "did something deploy?" to "which of these deployments is most likely responsible?" That is a more useful question because it has a testable answer. You can look at what changed in each deployment and reason about which one could have caused a checkout error rate spike.

You also get a passive audit trail. Every incident record includes the deployment events that were active at the time. When you run a retrospective two weeks later, you do not need to reconstruct the deployment history — it is embedded in the incident.

Teams that correlate deployments with incidents this way also tend to notice patterns in their postmortems. If a particular service consistently has deployment events preceding incidents, that is a signal worth investigating at the process level.

Extending the Pattern

The same change log endpoint accepts events from sources beyond CI/CD. Infrastructure changes made via Terraform can POST a change event at apply time. Database migration scripts can log before and after a migration runs. Feature flag systems can record when flags are toggled in production.

The common thread is that any change to a production system that a human or automated process makes intentionally is worth logging. The more complete the change log, the more useful the incident timeline becomes.

If your team uses multiple environments and you want to scope change visibility to incidents in the matching environment, include the environment field consistently across all your logging calls. Alert24 will filter change events to those matching the environment of the alerting integration when building the incident timeline.

Next Steps

Start with the integration that covers the most deployment activity on your team, whether that is GitHub Actions or GitLab CI. Add the logging step to one service first, trigger a test deployment, and verify the change appears in Alert24 before rolling it out across repositories.

Then set up the Grafana contact point if you have not already. You can run it alongside an existing PagerDuty or Opsgenie integration while you evaluate — Alert24 incidents will open in parallel, and you can compare what the timeline looks like versus your current tooling.

Once deployments are flowing and Grafana alerts are routing, the correlation happens automatically. The question "did something deploy recently?" stops being a blocker and becomes something your team can answer before anyone has to ask.