More Than Measurement
In the marketing mix community, many use Bayesian regression to improve measurement. While there are benefits to measurement, focusing on measurement down-plays the very-profound and broader role of Bayesian approaches to making decisions.
In the end, marketing mix is all about managing marketing investment. Let’s start there, with how Bayesian marketing mix (MMM) approaches can help improve marketing investment decisions and conclude with a discussion about advantages to measurement.
Bayesian MMM Improves Decision-Making
Making decisions is fundamentally a knowledge problem. Since we cannot know the future, we’re left to reason about it based on what we know today. Our reasoning, using present knowledge, can be rigorously formulated using probabilities. Bayesian approaches naturally prevail as methods in this effort.
For example, the return on investment (ROI) of your next marketing investment is inherently uncertain. The uncertainty can be expressed as a probability distribution. Think about a bell curve over the range of ROIs. Some ROIs are likely in the center of the bell curve and even still others will fall in the tails, outside the most likely range.
Bayesian approaches allow us to combine information from multiple sources with evidence from observations. And sources include not only marketing mix models and various other forms of marketing analytics, but also the experience of the organization.
Organizations are rich with incumbent knowledge. There’s no doubt that very high or very low ROIs elicit a strong reaction. As an extreme example, an ROI that says an investment returned $100 for every $1 spent will almost certainly elicit a strong, “no way,” from many.
And further example, if required to make a marketing investment decision tomorrow, most professionals will have both common sense and institutional history to help make that decision. These cases provide evidence that marketers, and their organizations, have information that they will bring to bear on a decision.
Still without using Bayesian approaches, professionals lack the tools to properly combine their knowledge with the prevailing evidence to improve a decision. For example, imagine a $1.50 – 2.00 ROI is in the believable range to a decision maker, while a statistical analysis produces $15.00 ROI.
Without a Bayesian framework, a manager is left to accept what seems like an unbelievable positive result or reject it altogether. This leads to classic confirmation bias, where they are left to accept or reject the latest result. There is no way to rigorously learn over time. Without a method for rigorous learning, professionals have difficulty de-biasing their intuition, making consistent improvements over time, and there is no improvement in the decision.
Traditional vs Bayesian Framework: What’s the Difference?
With the Bayesian approach, there is a logical and formal means to integrate information and evidence. The logic is that the strongest source gets more weight. It’s like a fuzzy tug-of-war between the information and the evidence. Rather than declaring one the winner and the other the loser, the Bayesian approach formally adjusts the result so that the stronger source gets more weight.
Integrating information up front means that marketers are not left to accept or reject the latest study of their marketing performance, rather they can learn incrementally starting from what they know and build with each incremental source of evidence.
The Bayesian approach removes bias from the decision. It’s like in a cartoon when a snowball rolls downhill and gets larger as it goes. So goes the marketer’s knowledge using a Bayesian approach. It starts with what they know and they learn more as they pick up more evidence.
Therefore, a Bayesian approach improves decisions by providing a:
- Rigorous foundation for reasoning about the future,
- Means to capture incumbent organizational knowledge,
- Process to learn over time by weighing the latest evidence,
- Approach to remove bias from the decision-making process.
The Limitations of Traditional Marketing Mix Modeling
Marketers want to measure their results so they can make improvements and prove the effectiveness of their efforts. By far the best way to do so is to observe both what has been done and the outcomes of the business. While there are signals in these observations, there is also noise and limitations in the data.
The data limitations create confusion from the data. Traditional marketing mix models can yield nonsensical results where on the one hand a tactic will have an absurdly high positive ROI, while others will be measured to have negative ROI, and yet others are not measurable at all.
Most skilled marketing mix modelers will talk about the “art” and the “science” of their craft. “Art” means the host of tricks learned, which range from dropping variables to changing lag structures to constraining coefficients. All this “art” is performed without transparency to the marketer in order to deal with the limitations of the data. The problem is that the level of discretion is left inside the black box, and the decision-maker has no agency in the process.
The limitations of traditional marketing mix models fall into one of three main problem areas:
- Multiple correlated marketing tactics,
- Low variation in the data, and
- Sparsity in the signals.
All the things that marketers optimally do to propel their business lead to these data problems.
First, marketing is inherently a communication challenge
Communicating well requires marketers to use multiple means of communication. People learn most about any topic when they hear, see, read, feel, taste and generally experience the information from a variety of sources. So, for marketers to best communicate their messages, they use multiple tactics at the same time. This causes marketing tactics to be correlated with each other.
Second, marketers need to maintain their presence in the market
Marketers need to stay on and continue to employ the same tactics over time. Of course, they will optimize the effort and steer the ship, but it does not mean that they will experiment with the business. Without experimentation means marketing data suffers from low variability.
Note that in drafting experiments to maximize the information for statistical inferences, researchers create experiments where some of the actions may be sub-optimal to create contrasting information. Experimentation increases the amount of signal and clarifies the difference between signal and noise.
Third, marketers advertise around events
Event-based marketing tactics such as sponsorships, associations with particular sport seasons, or promotions on special shopping days like Black Friday or Prime Day result in marketing activities alternating between heavy up and zero. Some may pop up once a year, if executed irregularly once every few years.
Depending on the event there may be substantial sums of money spent in short periods of time. It’s natural for marketers to follow opportunities where based on the event people may be preconditioned to be more receptive. This causes marketing data to be sparse.
These three limitations cause the data to have less signal, more idiosyncrasies, and be of lower quality for statistical inference. When things are happening at the same time or being executed steadily or sparsely, it means that the qualities of the data that make it better for measurement are sacrificed.
It shows that operating the business is at odds with measuring drivers of the business.
It’s like when you’re driving a car, it’s impossible to at the same time be driving an alternate route just to see if that route could be better. Generally, it’s impossible to optimally do and optimally experiment at the same time.
Thankfully, using the right methods we can learn from the byproducts of our operations.
Bayesian Improvements to Measurement
Bayesian approaches advance the state of marketing mix modeling. Bayesian MMM allows us to substitute information where the data have limitations and idiosyncrasies. For example, when two tactics are happening at once and correlation creates a limitation, creating a sort of tie between where credit goes, the Bayesian techniques step in and weigh the information against our previous knowledge.
How Bayesian Methods Work
Without getting into the math, Bayesian methods use our previous knowledge to focus the analysis. It works like this. Imagine you have a handful of metal balls, like BBs, and you throw them on a hard polished floor. They will bounce and roll anywhere inertia will take them, including getting trapped by idiosyncratic holes and cracks in the floor. This is how traditional statistical methods work: they follow the paths of least resistance including into idiosyncrasies of the data.
With Bayesian marketing mix modeling methods, it’s like you laid a fine fabric on the floor before throwing the BBs. The fabric helps focus the energy of the BB’s keep them from settling in the holes and cracks.
Prior information: The Key Advantage of the Bayesian Approach
Previous knowledge, the fine fabric from above, is referred to as a “prior” in Bayesian terminology. It’s literally additional information that the statistical model treats like additional data.
It’s the ability of the Bayesian approach to provide this prior information that provides all the advantages of the approach. This prior, acting as additional data, is designed to smooth over the idiosyncrasies to reduce the limitations of the data, so the analytical methods can better separate signal from noise.
Since the prior acts like additional data, this opens the opportunity to transparently incorporate information from any number of sources. The analyst and decision maker can join forces to ensure that all the available information is included in the analysis.
The ability to use all the available information is particularly relevant where marketers are involved in more event-based or local activities where the measurement methods required to evaluate that activity are specific to the event. In this case, the marketer should require that the knowledge gained from these sources be incorporated into the marketing mix model via the prior.
By using the Bayesian approach, marketers can experiment on a more limited basis and incorporate that information into the analysis. This enables more experimentation. Findings across the organization including those from A/B tests or test markets can be incorporated. This also enables unified marketing measurement, helping to reduce conflict in the process. The Bayesian approach can be used to project the experimental information as priors into forecasting models.
When, as in the conceptual example above, the BBs roll into the holes and cracks in the floor, this is what is known in statistical modeling as overfitting. It occurs when there are so many variables that the model starts to make statements based on idiosyncrasies in the data. The tell-tale sign of this is when the model fits well in history but doesn’t forecast well. The ideal is that the model fits as well as it forecasts. This may well mean that using a simpler model that allows more noise and less precision is better. Bayesian priors, like in the example of the BBs, provide a means to shield the model from fitting idiosyncrasies.
Marketers Benefit from Bayesian Marketing Mix Modeling
Bayesian marketing mix modeling frees marketers to focus on running the business and get feedback on performance without concern for being led astray by idiosyncrasies in the data.
The Bayesian approach:
- Solves problems that stem from limitations of the data including multicollinearity, low variance, and sparsity.
- Improves transparency so the recipient of the results knows the information being used in the model.
- Provides a means to resolve conflict and source information across a wider domain.
- Avoids overfitting the data based on idiosyncrasies and improves forecasting.
Leverage Bayesian Marketing Mix Modeling with Keen
Bayesian approaches improve decision-making. They are a workhorse for marketers and marketing mix modelers alike.
By integrating previous knowledge with current evidence, Bayesian methods enable decision-makers to make marketing decisions with greater confidence, more information, and more transparency. Using a Bayesian MMM approach, marketers can avoid being led astray by idiosyncrasies and attend to their business based on the signals from the data.
Want to learn how Keen leverages a Bayesian marketing mix modeling approach? Start your 14 day free trial now!
