Datadog Integration - GrowthBook Documentation

When a Datadog monitor alerts, a Datadog webhook can call the GrowthBook REST API to undo a release. There are two ways to set it up, depending on how the feature is released:

Roll back ramp-ups: for features released with ramp-ups. Any monitor you already have works, and Datadog needs no flag data.
Turn a flag off: for a flag without a ramp-up. Connect an existing service monitor to a specific flag, or send flag evaluations to Datadog to monitor only users who receive that feature.

Both need a GrowthBook API key whose role can publish the features in the affected environments. A key with Require confirmation turned on holds each request until someone confirms it, so use one without it (API Keys for Agents). The examples use https://api.growthbook.io. If you self-host, use your API host.

Roll back ramp-ups

A ramp-up adds traffic to a feature in steps. You can use a service error rate, an SLO, or a RUM error count as a signal to stop releases affecting that service. The webhook does not determine which release caused the alert; you choose which ramp-ups it can roll back through its filters. One webhook calls Roll back matching ramp schedules, which rolls back every running or paused ramp-up that matches its filters. That includes ramp-ups started after you set it up, so there is nothing to change for each release. Rollback restores the rule’s saved base state, not just the previous step or necessarily zero exposure. Edits made while paused can update that base state; see Editing a rule under a running schedule. Normally, a full rollback ends the schedule. A schedule requiring start approval instead returns to its pre-start hold with its rules disabled, awaiting fresh approval.

Completed schedules are not matched. If an alert arrives after a ramp finishes, this webhook will not undo that release. Allow time during the ramp for the monitor’s evaluation window and data ingestion delay, and plan how to disable the feature if a problem appears after completion.

1. Create the webhook

In Datadog, create a webhook, for example growthbook-rollback-checkout:

URL: https://api.growthbook.io/api/v1/ramp-schedules/actions/rollback?environment=production&tag=checkout
Custom headers: {"Authorization": "Bearer <GrowthBook API key>"}
Payload: {"reason": "Datadog: $ALERT_TITLE $LINK"}

The query string picks the ramp-ups. Each filter takes comma-separated values and matches any of them. Different filters must all match, and a rollback needs at least one.

Filter	Matches ramp-ups on features…
`environment`	whose rule runs in one of these environments
`project`	in one of these project ids
`tag`	with one of these tags
`feature`	with one of these feature ids
`customFields[<key>]`	whose custom field `<key>` has one of these values

For example, if three running checkout ramp-ups match environment=production&tag=checkout, the webhook rolls back all three. Paused matches are included too, even if they were paused before the incident. Filters select whole schedules; they do not identify the offending release. Adding a matching tag or custom field value to a feature puts its matching active and future ramp-ups under the webhook. Scope the Datadog monitor to the corresponding service and environment: its tags do not automatically change the filters in this URL. To pause instead, so someone decides whether to resume, use /api/v1/ramp-schedules/actions/pause. A pause leaves current exposure in place and stops further progression; it does not remove the feature from users already receiving it. A pause works with no filters too, which pauses every running ramp-up during a site-wide outage.

2. Add it to your monitors

Mention the webhook in each monitor that should roll the ramp-ups back, inside {{#is_alert}} so warnings and recoveries don’t call it:

{{#is_alert}} @webhook-growthbook-rollback-checkout {{/is_alert}}

Each rolled-back ramp-up records the reason, so its history links to the alert. The response lists every ramp-up the call matched, marked applied or skipped with the reason, for example because the API key can’t publish that feature.

Turn a flag off

An existing service error or latency monitor can disable a specific flag without flag-level telemetry. In that case, skip to Create the webhook. For a small rollout, a service-wide metric may hide problems affecting the enabled users. Follow the instrumentation and monitor steps below to watch that cohort. When the monitor alerts, the webhook toggles the flag off in one environment, and users get the fallback value in your code. It’s a kill switch rather than a rollback. For example, after the SDK receives the updated definitions, gb.getFeatureValue("new-checkout", false) returns the code fallback false when the flag is disabled in that environment. This is not the feature’s default value configured in GrowthBook. Check that your application’s in-code fallback selects the safe behavior, especially for string, number, or JSON flags. Both toggles and ramp rollbacks take effect in applications when their SDKs receive updated feature definitions. Verify your SDK’s streaming, polling, or refresh configuration and any caching in between; an API success does not mean every open browser or running server has updated yet.

1. Send flag evaluations to Datadog

Report each evaluation with the SDK’s onFeatureUsage callback. This also lets you filter and group errors, sessions, and traces by flag value while investigating. The snippets below extend an existing, initialized GrowthBook and Datadog setup. Keep your SDK connection settings when adding the callbacks. Browser (Datadog RUM): RUM records evaluations on the user’s views and errors as @feature_flags.<key>.

const gb = new GrowthBook({
  onFeatureUsage: (featureKey, result) => {
    datadogRum.addFeatureFlagEvaluation(featureKey, result.value);
  },
});

If your Datadog RUM browser SDK is older than v5.17.0, enable feature flags during RUM SDK initialization. Server (Datadog APM): gbClient below is your initialized GrowthBookClient. Evaluate flags through req.growthbook in your request handlers so the callback records that request’s values. Run the middleware after authentication if you use req.user.id for targeting. Tag the request span, which monitors aggregate on, rather than whichever span is active when a flag is evaluated. With Express, collect each request’s flag values in a scoped GrowthBook instance and write them in dd-trace’s request hook, which runs just before the request span finishes:

tracer.use("express", {
  hooks: {
    request: (span, req) => {
      for (const [key, value] of Object.entries(req.featureFlags ?? {})) {
        span?.setTag(`feature_flag.${key}`, value);
      }
    },
  },
});

app.use((req, res, next) => {
  req.featureFlags = {};
  req.growthbook = gbClient.createScopedInstance({
    attributes: { id: req.user?.id },
    onFeatureUsage: (featureKey, result) => {
      req.featureFlags[featureKey] = String(result.value);
    },
  });
  next();
});

APM monitors on span tags only evaluate spans kept by custom retention filters. Add a retention filter for the spans the monitor queries (Datadog bills them as indexed spans), or monitor RUM errors, which need no filter.

Before creating the monitor, evaluate the flag and generate a controlled error in a test environment. In RUM Explorer, confirm that the error has @feature_flags.new-checkout:true. For APM, confirm that the request span has @feature_flag.new-checkout:true, an error status, and the expected service and environment. Repeat with the flag evaluating to false to verify both cohorts.

2. Create a monitor for the flag

For a concrete starting point, create a RUM monitor that alerts on an error count:

Go to Monitors > New Monitor > Real User Monitoring.
Select error events and use the search @type:error env:production service:checkout-web @feature_flags.new-checkout:true. Replace the service, environment, and flag key with yours.
Select Count, with no grouping, to aggregate the matching errors.
Set the alert condition to above 10 over the last 5 minutes. These are example values, not recommended production thresholds: tune them to your normal error volume, traffic, and acceptable impact.
Add the webhook notification from step 4 below. Also notify your incident responders so someone can investigate the disabled feature.

This monitor limits error volume in the enabled cohort; it does not prove the flag caused the errors. A count threshold also becomes easier to reach as exposure grows. If you compare enabled and disabled cohorts, compare error rates with a consistent denominator, such as failed requests divided by all requests in each cohort, rather than raw error counts. Require enough traffic in each cohort before acting, and decide how missing data should behave. An error-only query cannot provide that denominator. See Datadog’s minimum request threshold guide for a metric-based rate example. For server errors, choose APM > Trace Analytics and count request spans matching env:production service:checkout-api status:error @feature_flag.new-checkout:true. Use the request operation or resource filter for your service to select the spans tagged by the hook. The status:error filter is necessary: the flag tag alone also matches successful requests. Verify that the custom retention filter directly retains these request spans before relying on the monitor.

3. Create the webhook

In Datadog, create a webhook, for example growthbook-off-new-checkout:

URL: https://api.growthbook.io/api/v2/features/new-checkout/toggle
Custom headers: {"Authorization": "Bearer <GrowthBook API key>"}
Payload: {"environments": {"production": false}, "comment": "Datadog: $ALERT_TITLE $LINK"}

The comment is recorded on the revision the toggle publishes. If the feature’s project requires approval to publish, the API key also has to be able to bypass approvals.

4. Add it to the monitor

Mention the webhook in the flag’s monitor, inside {{#is_alert}}:

{{#is_alert}} @webhook-growthbook-off-new-checkout {{/is_alert}}

Before you rely on it

Starting a release does not trigger a Datadog notification. An already-alerting monitor may send another webhook through renotification, depending on its settings and notification message, but do not assume it will immediately stop a new release. Check monitor health before starting or resuming a rollout.
Datadog’s Test Notifications sends a real request, which rolls back every matching ramp-up or turns the flag off.

Test the full path in staging

Use a disposable flag or ramp-up in a test environment. Point a separate webhook at that environment and, for ramp-ups, include a feature filter for the test feature so the test cannot match other releases. Use a key with the same relevant permissions as your production key.
Record the expected safe value or ramp base state. For a ramp-up, start it and advance to a step whose exposure differs from that state.
Use Test Notifications to check delivery and permissions. In GrowthBook, inspect the ramp’s event history or the flag’s published revision for the Datadog reason or comment, and verify the resulting rule or environment state.
Reset the test release, let the monitor recover, and generate enough matching errors to cross the test threshold. Confirm that a real alert triggers the same action. This also checks instrumentation, retention, query scope, and notification routing, which Test Notifications alone does not validate.
Verify the safe behavior in the application, including an already-open browser or running server, and measure the delay from the alert to the changed behavior. Confirm that monitor recovery does not re-enable the flag or resume the ramp; recovery is excluded by the notification template.

Diagnose missing actions

A delivered webhook is not proof that a release stopped. The bulk ramp endpoint can return a successful HTTP response with an empty match list or with skipped results. Inspect the response when testing the API directly: an empty rampSchedules list means no eligible schedules matched; a skipped entry includes a message explaining why the action was not applied. For live incidents, check GrowthBook’s history and current release state as well as Datadog’s notification delivery. Check the filters and schedule status for zero matches, and the API key’s project/environment publish permissions for skipped operations. For flag toggles, also check approval bypass permissions. If you need automated detection of skipped actions, add a webhook receiver that inspects the response and notifies your team; the direct webhook setup does not do this. Datadog’s webhook retry policy covers certain delivery failures, including server errors, but does not retry individual skipped entries inside a successful response. Arrange a human notification alongside the webhook so delivery failures or an unsuccessful rollback can be investigated. After fixing the incident, deliberately re-enable the flag or restart/resume the appropriate schedule; these examples do not restore a release automatically.

​Roll back ramp-ups

​1. Create the webhook

​2. Add it to your monitors

​Turn a flag off

​1. Send flag evaluations to Datadog

​2. Create a monitor for the flag

​3. Create the webhook

​4. Add it to the monitor

​Before you rely on it

​Test the full path in staging

​Diagnose missing actions

Roll back ramp-ups

1. Create the webhook

2. Add it to your monitors

Turn a flag off

1. Send flag evaluations to Datadog

2. Create a monitor for the flag

3. Create the webhook

4. Add it to the monitor

Before you rely on it

Test the full path in staging

Diagnose missing actions