Migration Guides

Migrating from Firebase A/B Testing

Move your A/B tests from Firebase A/B Testing to ConfigDirector experiments. How experiments, variants, goal metrics and the results page map, the Flutter and web code that replaces the Remote Config and Analytics calls, and a checklist.

This guide is for apps that run A/B tests with Firebase A/B Testing on Remote Config and want to run them in ConfigDirector. It maps Firebase's experiment concepts onto ConfigDirector's, shows the Flutter and web code that replaces the Remote Config and Analytics calls, compares the two results pages, and ends with a checklist.

The biggest change is where the results come from. Firebase measures an experiment on the events your app logs to Google Analytics. ConfigDirector measures it on events your app reports to ConfigDirector with track, and shows the results on the experiment's Experiment tab, so an experiment does not need Google Analytics. Your Analytics events stay as they are: keep logging them for everything else you use them for.

If your app also reads Remote Config parameters that are not part of an experiment, Migrating from Firebase Remote Config covers them, along with setting up the project and the SDK. The Experiments guide walks through each step of an experiment in the dashboard.

Before you start

Let the running experiments finish in Firebase. ConfigDirector assigns variations in its own way, so an experiment moved halfway would put some users in a different variation from the one they had, and its counts would start from zero. Start new experiments in ConfigDirector, and leave each running one in Firebase until you have read its result.
Decide how your app identifies users. Firebase assigns and counts each installation of your app. ConfigDirector assigns and counts each context with an id, so a signed-in user keeps the same variation on every device. A context without an id is still served a variation in Flutter and in the browser, but no experiment counts it. To count signed-out users, give each installation an id of your own, such as a random identifier you store on the device the first time the app starts, and leave anonymous off. ConfigDirector does not merge that context with the one a user gets by signing in: the context with their user id is another context, which can be served another variation and is counted on its own. Give every context a stable identifier has the details.
List the Analytics events your experiments measure. Each goal and metric of an experiment you plan to move becomes a metric on an event your app reports with track. Only those events need track: ConfigDirector stores up to 100,000 tracked events per environment per day, so report the outcomes your experiments compare rather than every Analytics event. See Limits.

How the concepts map

Experiments

Firebase A/B Testing	ConfigDirector
Experiment on Remote Config parameters	An experiment: a config with the `Experiment` role, of type String, Integer, Float or Boolean. Keep the parameter's key as the config's key, so only the SDK call changes in your code.
A variant that changes several parameters	One experiment whose variations stand for the whole change. Your code turns each variation into the values it used to read from the parameters, such as a String variation `compact` that selects a layout, its spacing and its copy.
Baseline	The control: the variation the default targeting rule serves when the run starts, so pick the baseline's variation as the Default variation when you create the experiment. See The control.
Variants	Variations: named values. Name each one, since the dashboard, the audit log, the CLI and OpenFeature refer to a variation by its name.
Variant weights	The shares of a percentage rollout. Saving different shares while the experiment runs ends the run and starts a new one, whose counts start from zero.
Targeting conditions	A conditional rule with the percentage rollout inside it, using the conditions or a segment that matches the same users. Contexts the rule does not match get the rules below it and are not part of the experiment.
Percentage of target users	Shares that add up to less than 100%, such as 5% for each of two variations. The other contexts go on to the rules below the percentage rollout, are served what those rules serve, and are not part of the experiment. The targeting rules editor warns you when an experiment's shares add up to less than 100%.
Activation event	A context joins the experiment when your code reads the experiment and the percentage rollout serves it a variation, and only events it reports from then on count. Read the experiment where the change shows, not when the app starts, so only the users who reach that point count.
Goal metric	The primary metric, with the metric kind it decides on.
Other metrics	Secondary metrics, which support the decision, and guardrails, metrics that must not get worse. An experiment can have up to 10 metrics, and a project up to 50.
Starting the experiment	Saving the targeting rules with the percentage rollout starts a run in that environment. Each environment runs its own split, so try the experiment in `Test` before `Production`.
Stopping the experiment	Stop run…: every percentage rollout is removed, the default targeting rule value keeps serving, and the run keeps a note on why you stopped it.
Rolling out the leader	Ship…: in one save, the variation you pick becomes the default targeting rule value and every percentage rollout is removed. It applies to the whole environment, not only to the users the experiment targeted; a conditional rule that serves a single value stays, so the contexts it matches keep that value. See Ship a variation.
A time limit on experiments	None: a run lasts until you ship a variation, stop it or change the split.
A limit on running experiments per project	No limit of its own. An experiment is a config, so it counts toward the configs limit: 100 on Free, unlimited on Pay as you go. Each environment has one run of an experiment at a time.
Test devices	A conditional rule above the percentage rollout that serves a single variation to your own test users, with `identifier is one of`. Contexts served by a conditional rule's single value are not part of the experiment, so add the rule before the run starts and testing does not change the results. The targeting rules tester shows which variation any context gets.
Google Analytics, required for A/B Testing	Not needed. The results come from the events your app reports with `track`.
Experiment membership in Analytics and BigQuery	Optional: report each exposure to your analytics tool from the SDK's evaluation event. See Send exposures to your analytics tool.

Metrics

A ConfigDirector metric is an event name your app reports with track, defined once per project on the Metrics page with a direction, Higher is better or Lower is better. Each metric kind turns a variation's events into one number:

Firebase goal or metric	ConfigDirector metric
An Analytics event, such as a completed checkout	A metric on the same event name, with Conversion: the share of the variation's contexts that reported the event at least once.
How often users trigger an event	Count per context: how many times each context reported the event, averaged over the variation's contexts.
Purchase revenue, or other revenue	Value per context: the sum of the event's `value` per context, averaged over the variation's contexts. Track the event with the amount as its `value`. The value is a plain number, so report every amount in the same currency.
An event you want less of	Any of the kinds, on a metric with Lower is better, such as `support_ticket_opened`. The results read fewer tickets than the control as better.
Retention	Not available. Count per context on an event your app tracks each time it opens compares how many times each variation's users open the app during the run.
Crash-free users	Not available: ConfigDirector does not receive crash reports. A guardrail on an event your app tracks when it shows an error, with Lower is better, catches the failures your code sees.

How ConfigDirector computes results explains each kind, and the Experiments guide shows how to define a metric and attach it with a role.

Replace the SDK calls

Set the client up as Migrating from Firebase Remote Config shows, with the id you chose for each context. Then two calls change: reading the experiment, and reporting the outcome.

Read the experiment

Read the experiment where Remote Config's value used to be read, with the control's value as the in-code default value. Your app gets the in-code default value when it cannot get a value from ConfigDirector. The Flutter and JavaScript SDK pages describe each way to read a config:

// Before: remoteConfig.getString('checkout_cta_copy')
final buttonLabel = client.getValue('checkout_cta_copy', 'Start free trial');

// Before: getValue(remoteConfig, "checkout_cta_copy").asString()
const buttonLabel = client.getValue("checkout_cta_copy", "Start free trial");

A context the percentage rollout serves is exposed once per variation, however often it reads the experiment.

Track the outcome

Keep your Analytics call, and report the same outcome to ConfigDirector with track, under the same event name, so the metric has the name your team already knows. Give the event a value when it carries an amount:

await FirebaseAnalytics.instance.logEvent(
  name: 'checkout_completed',
  parameters: {'value': 49.0, 'currency': 'USD'},
);

client.track('checkout_completed', const TrackOptions(value: 49.0));

logEvent(analytics, "checkout_completed", { value: 49, currency: "USD" });

client.track("checkout_completed", { value: 49 });

track reports the event for the client's current context, so the event counts for the variation that context was served. ConfigDirector does not read your Analytics events: an outcome counts only when your app reports it with track.

The Flutter and JavaScript SDK pages describe track, its options and its limits. The React, Vue, Next.js and Nuxt SDKs offer it too, and every other SDK is listed under Tracked events.

Send exposures to your analytics tool

If your team also wants to see experiments next to its other events in Google Analytics, as Firebase's experiment user properties allowed, report each exposure from the SDK's evaluation event: onConfigEvaluated in Flutter and the configEvaluated hook in the browser. The Google Analytics 4 page has a handler to copy for the browser, and Send exposures to your analytics tool lists the hook for every SDK.

Read the results

The Results panel on the experiment's Experiment tab compares each variation with the control on every metric the experiment is measured on. This is how it relates to Firebase's results page:

Firebase results page	ConfigDirector Results panel
Baseline	The control, first in each table.
Observed totals and rates	Each variation's value, such as `14.3%` with `672 of 4,702` beneath it for Conversion.
Improvement over the baseline	Lift: how much higher or lower the variation's value is than the control's.
Confidence interval	Likely range: where the real lift is likely to be. A range that spans zero means the variation could still turn out better or worse than the control.
p-value	Chance to beat control: the chance that the variation is better than the control, in the metric's direction. It reads directly as that chance, which a p-value does not. With three or more variations, Chance to be best gives each variation's chance of being the best of them all.
Leader	A verdict for each variation on each metric kind: Likely better, Likely worse or No clear difference. The verdict on the primary metric's kind is the one the Ship dialog shows. Nothing ships or stops on its own: your team decides.
Results updated once a day	Results up to 25 minutes behind what your app reports.
Cumulative chart	A daily chart of each variation's value for the run so far, with a point for each day in UTC, ending with the current numbers.

The panel also warns you when a guardrail is likely worse for a variation, and the Exposures card above it checks the split against the shares you configured: when they do not match, the run's results are not reliable. Until every variation has enough exposures, the panel shows each variation's value and says the run needs more exposures before the variations can be compared.

ConfigDirector computes these numbers with a Bayesian method, which How ConfigDirector computes results explains in full, along with what the method does not do. Read the results in the Experiments guide walks through the panel with screenshots.

Experiments, metrics and results are available on Free and Pay as you go, and tracked events are not billed. Pricing and billing lists what is.

Checklist

For each experiment you move:

The experiment exists in ConfigDirector with the parameter's key and a named variation for each Firebase variant, with the baseline's variation as the Default variation, so that it is the control.
Each Analytics event the experiment measured has a metric on the Metrics page, with its direction and the metric kinds you want.
The metrics are attached to the experiment, with the goal metric as the primary metric and the kind it decides on, and the hypothesis is written, before the run starts.
Your app gives each context the id you chose, signed-out users included, without anonymous.
Your app reads the experiment with the SDK's getValue where the change shows, with the control's value as the in-code default value.
Your app calls track for each measured event, beside its Analytics call, with the amount as the value for revenue.
The release with these calls reaches your users before you start the run. The SDK versions that report exposures are 1.8.0 for Flutter and 1.11.0 for JavaScript, or later.
In Test, the split is saved, the Exposures card counts contexts, and the Results panel shows each metric receiving events.
In Production, the split is saved with the shares you want, inside a conditional rule if the experiment targets part of your users.
You decided before the run how long it will go, and you read the primary metric's verdict then, checking the guardrails and the mismatch warning.
You shipped a variation or stopped the run, with a note, in each environment that ran it.
Once the experiment's config has settled, its value is in your code and the config is archived.
Once the app versions that read the Firebase parameter have faded from your user base, the parameter is removed from the Remote Config template.

From Firebase Remote Config

Move your remote configuration from Firebase Remote Config to ConfigDirector. How parameters, conditions, in-app defaults, and fetch and activate map, how to recreate the parameters, and how to switch the Flutter, Android, iOS, and web SDKs over.