Today we’re introducing Experiments in GrowthOS, to build confidence behind every big bet that gets you to your growth target. GrowthOS shows you which budget moves are worth testing and helps you design a geo test to check the move before you commit the budget.
Every result feeds straight back into your model, so Marketing and Finance build one shared body of evidence to make the big decisions together.
To learn more and get started, speak with our team.
Why standalone incrementality experiments are not enough
Incrementality experiments are often framed as the “causal ground truth” of marketing effectiveness. They are valuable because they give Finance a clear, easy-to-explain number behind a decision.
But on its own, an experiment is a snapshot: one answer, in one channel, at one point in time.
That’s why experiments can’t replace a marketing mix model. A single test cannot tell you what the same change would do at a different budget, how that channel works with the rest of your mix, or whether the effect holds next quarter. And because every test costs time and media, most teams only run a few a year. Your MMM covers every channel, market and week, so it’s what you plan the whole budget with.
Experiments and your MMM are strongest when they work together, in one tight loop. Each experiment provides evidence that feeds into your model and sharpens it over time. This gives Marketing and Finance the same set of numbers to read from, built on more independent evidence that points the same way, instead of reconciling numbers by hand or trusting any one number from multiple systems. This builds more confidence behind the big and impactful decisions and increases the likelihood that they get made instead of going stale in a deck.
This is how Experiments works in GrowthOS. Before you spend, you and your CFO agree on the smallest change in sales you’d act on. GrowthOS shows what your model predicts, then helps you design a geo test to check it.
What’s different in GrowthOS
Every experiment feeds your model automatically
Today, most teams run an experiment, read their MMM, then reconcile the two numbers by hand. They decide how much each one counts before asking their CFO to sign off.
Experiments in GrowthOS handles that step for you, with no manual calibration. The experiment and your model share one estimate of the channel’s effect. You get one number for that channel, not a test number and a model number to argue over.
See where your spend is working, market by market
Instead of a single national model, GrowthOS models your business region by region, at a hyper-granular level. That’s thousands of models, each learning from the others and getting sharper with every experiment you run. You see where spend is working, back the markets that respond, and get a readable answer even where data is thin.
Rigor is built in
A geo experiment is only useful if you can act on it. Every design has these checks built in:
- Checked before you spend. Every design is run on your own sales history. On past weeks when nothing changed, it must stay within a strict false-alarm limit. It must also hold up on 13 weeks of history it never saw.
- Read independently. Each result is read first with a standard method that doesn’t use your MMM. When the test and your model agree, you’re deciding on two independent pieces of evidence, not your model checking itself. The joint model then combines both into the one number you plan with.
- Evidence, not ground truth. A test has error bars, so it shouldn’t rewrite your plan on its own. In your MMM, a precise test moves the model more and a noisy one moves it less.
How Experiments in GrowthOS works
1. Set the hypothesis
See whether a move is worth testing before you design anything. Choose a channel, a spend change (switch off, cut or increase) and a measure (total revenue or units). Then set your threshold, which is the smallest change you’d act on. GrowthOS shows the effect your model predicts.
2. Choose the design
Know what a test costs and what it can detect before you spend. GrowthOS prices every candidate split of your regions, at every length from 4 to 12 weeks, and recommends four designs. A design can answer your question when the smallest change it can detect is at or below your threshold.
| Design | Length | What it’s for |
|---|---|---|
| Standard | 4 to 8 weeks | The cheapest design that can detect your threshold |
| Pilot | 4 weeks | A cheaper first look that only catches a large effect |
| Sensitive | 4 to 8 weeks | The most precise read, at up to 3× the cost of Standard |
| Extended | 10 to 12 weeks | Same as Standard, with a longer testing window |
3. Run the experiment
You keep control of your media. Your team or agency runs the test in your own ad platforms, because Mutinex doesn’t buy media. Before and during the test, your team:
- confirms the platform supports clean exclusion by region and your contracts allow it
- sets up one campaign and one budget per group
- runs one live week to confirm that 2% or less of spend leaks into test regions and 5% or less into control regions
- keeps flights, pricing and strategy the same in both groups
4. Review the results
Walk into the budget meeting ready to commit, or ready to move the money somewhere better. If the result doesn’t back the move, you’ve learned that in a few markets, not across the whole budget. Your Marketing Science Partner works through the result with you. Even an inconclusive result shows the direction of the effect, which helps you decide whether to act, run longer or test a bigger change.
Get started
Experiments is available now for Advanced and Enterprise customers in the US and Australia with regional modeling. In the US, that means 40 or more modeled geographies with revenue.
This is an early version, and your feedback will help shape what comes next. Speak with our team to get started, and start backing the big bets that get you to your growth target.