The Function of Priors in Modern MMM
Bayesian methods are the standard for modern MMM because they effectively address common data challenges that are difficult for traditional regression models to handle.
- Sparse Data: For a new channel with a limited data history, a prior provides a reasonable starting value for its performance, preventing the model from making unstable estimates.
- Multicollinearity: Often, marketing spend for different channels increases at the same time, making it hard to isolate the impact of each one. Priors help the model differentiate between the effects of these correlated channels.
- Noisy Signals: Sales are influenced by many non-marketing factors such as seasonality, competitor actions, and economic trends. Priors help the model identify the marketing signal within this noise.
In summary, priors incorporate existing business knowledge to constrain the model’s outputs to a plausible range. This increases the stability and sensibility of the results.
How Bayesian Priors Work
The Bayesian framework combines existing beliefs with new data to arrive at an updated conclusion. This process has three components:
The Prior: An initial belief about a parameter’s value before analyzing the data. For example, an initial assumption about the expected Return on Investment (ROI) for a specific marketing channel.
- The Likelihood: The new evidence observed from the data. In MMM, this is the historical sales, spend, and other relevant data.
- The Posterior: The updated belief about the parameter’s value after combining the prior and the likelihood. This posterior is the model’s final, data-informed estimate.
The strength of the initial belief (the prior) determines how much it influences the final result.
Identifying Risk: Weak vs. Strong Priors
The risk associated with priors depends on how “informative” they are.
- Weakly Informative Prior: This acts as a gentle guide. It might set a wide, common-sense boundary, for example, stating that a channel’s ROI must be positive. This approach allows the observed data to be the primary driver of the final result.
- Strongly Informative Prior: This acts as a rigid constraint. It forces a parameter into a very narrow range. For example, setting a prior that a channel’s ROI must be between 2.5 and 2.6.
When a prior is overly strong, it is no longer a guide. It dictates the model’s output, regardless of what the data suggests. The model is simply confirming a pre-set assumption, not learning from the data.
How Strong Priors Degrade Model Quality
This problem becomes more severe when multiple strong priors are used in a single model. An MMM must attribute sales performance across all contributing variables. If strong priors are used to lock in the performance of major channels (e.g., Search, TV), they effectively pre-allocate a large portion of the sales results.
The model is then forced to explain any remaining sales variance using the few channels that were not constrained. This often leads to distorted and illogical results for these unconstrained variables, such as extremely high or negative ROIs.
The model may appear credible because the main channels show expected results. However, the overall model is unstable, fundamentally flawed, and will produce poor budget allocation recommendations. At this point, the priors are a source of systematic error.
Actionable Steps for Vetting Your MMM
Ensuring the integrity of your MMM entails verifying how your priors are being used. This requires complete transparency from your analytics vendor or internal team, and you should always ask them these questions:
- “Can you provide documentation for all priors used in the model?”
- “What is the justification for the level of informativeness (i.e., the tightness of the range) for each prior?”
- “Can you demonstrate how sensitive the model’s results are to changes in these priors?” (This is known as a sensitivity analysis).
Vague, dismissive, or incomplete answers are a major red flag.
A well-constructed MMM uses priors as justified and transparent guides; while a model driven by opaque or overly strong priors is not data-driven and presents a significant risk to your business. So keep asking demanding answers to the above three questions from your MMM vendor.
Before that red flag turns white.