Today we’re introducing Experiments in GrowthOS, built for the big bets that get you to your growth target. GrowthOS shows you which budget moves are worth testing, like increasing Meta spend by 30%. Then it helps you design a geo test in a few markets to check the move before you commit the budget.
Every result feeds straight back into your model, so Marketing and Finance build one shared body of evidence and make decisions together with confidence.
To learn more and get started, speak with our team.
Why one experiment isn’t enough
Incrementality experiments are often called the “causal ground truth” of marketing effectiveness. A well-designed experiment is valuable on its own, because it gives you a number that can stand up to the CFO test. But a single experiment is a snapshot: one answer, in one channel, at one point in time.
That’s why experiments can’t replace a marketing mix model. A geo test measures one channel, at one spend level, in the regions and weeks it ran. It can’t tell you what the same change would do at a different budget, how that channel works with the rest of your mix, or whether the effect holds next quarter. And because every test costs time and media, you can only run so many. Your MMM covers every channel, market and week, so it’s what you plan the whole budget with.
An experiment becomes far more valuable when you treat its result as one strong piece of evidence and feed it into your MMM over time. Marketing and Finance then read the same number for each channel, backed by independent evidence that points the same way.
In GrowthOS, your experiments and your MMM form one joint model. Before you spend, you and your CFO agree on the smallest change in sales you’d act on. GrowthOS shows what your model predicts, then helps you design a geo test to check it.
What’s different in GrowthOS
Every experiment feeds your model automatically
Today, most teams run an experiment, read their MMM, then reconcile the two numbers by hand. They decide how much each one counts before asking their CFO to sign off.
Experiments in GrowthOS handles that step for you, with no manual calibration. The experiment and your model share one estimate of the channel’s effect. You get one number for that channel, not a test number and a model number to argue over.
See where your spend is working, market by market
Instead of a single national model, GrowthOS models your business down to the DMA level in the US and SA4 regions in Australia. That’s thousands of models, each learning from the others and getting sharper with every experiment you run. You see where spend is working, back the markets that respond, and get a readable answer even where data is thin.
Rigor is built in
A geo experiment is only useful if you can act on it. Every design has these checks built in:
- Checked before you spend. Every design is run on your own sales history. On past weeks when nothing changed, it must stay within a strict false-alarm limit. It must also hold up on 13 weeks of history it never saw.
- Read independently. Each result is read first with a standard method that doesn’t use your MMM. When the test and your model agree, you’re deciding on two independent pieces of evidence, not your model checking itself. The joint model then combines both into the one number you plan with.
- Evidence, not ground truth. A test has error bars, so it shouldn’t rewrite your plan on its own. In your MMM, a precise test moves the model more and a noisy one moves it less.
How Experiments in GrowthOS works
1. Set the hypothesis
See whether a move is worth testing before you design anything. Choose a channel, a spend change (switch off, cut or increase) and a measure (total revenue or units). Then set your threshold, which is the smallest change you’d act on. GrowthOS shows the effect your model predicts.
2. Choose the design
Know what a test costs and what it can detect before you spend. GrowthOS prices every candidate split of your regions, at every length from 4 to 12 weeks, and recommends four designs. A design can answer your question when the smallest change it can detect is at or below your threshold.
| Design | Length | What it’s for |
|---|---|---|
| Standard | 4 to 8 weeks | The cheapest design that can detect your threshold |
| Pilot | 4 weeks | A cheaper first look that only catches a large effect |
| Sensitive | 4 to 8 weeks | The most precise read, at up to 3× the cost of Standard |
| Extended | 10 to 12 weeks | Same as Standard, with a longer testing window |
3. Run the experiment
You keep control of your media. Your team or agency runs the test in your own ad platforms, because Mutinex doesn’t buy media. Before and during the test, your team:
- confirms the platform supports clean exclusion by region and your contracts allow it
- sets up one campaign and one budget per group
- runs one live week to confirm that 2% or less of spend leaks into test regions and 5% or less into control regions
- keeps flights, pricing and strategy the same in both groups
4. Review the results
Walk into the budget meeting ready to commit, or ready to move the money somewhere better. If the result doesn’t back the move, you’ve learned that in a few markets, not across the whole budget. Your Marketing Science Partner works through the result with you. Even an inconclusive result shows the direction of the effect, which helps you decide whether to act, run longer or test a bigger change.
Get started
Experiments is available now for Advanced and Enterprise customers in the US with 40 or more modeled geographies with revenue, and for customers in Australia with SA4 modeling.
This is an early version, and your feedback will help shape what comes next. Speak with our team to get started, and start backing the big bets that get you to your growth target.