You can’t measure your way to a decision. If you’re building a house, you have to start cutting and hammering, at some point, even if after measuring twice. And so it is with making decisions, you might measure twice, but at some point you’re going to make a decision, develop and deploy the marketing budget.
What is Unified Marketing Measurement (UMM)?
Unified Marketing Measurement (UMM) is a comprehensive approach that integrates various marketing analytics techniques to evaluate and optimize performance across all channels. UMM provides a holistic view by consolidating data from online and offline sources, enabling marketers to understand the full impact of their campaigns. This method improves decision-making by offering insights into how different marketing efforts interact and contribute to business outcomes, thus ensuring a more accurate, data-driven strategy for maximizing return on investment (ROI).
Benefits of Unified Marketing Measurement
Tools for unified measurement offer several significant benefits. It provides a holistic view of marketing performance by integrating diverse data sources, leading to more accurate insights. This comprehensive perspective helps in understanding the interplay between various marketing channels, both online and offline. UMM enhances decision-making by delivering a clear picture of which strategies drive the most significant ROI. It also facilitates more efficient budget allocation, ensuring investment in the most effective tactics. Furthermore, UMM adapts to evolving market conditions, offering flexible and real-time analytics that improve responsiveness and optimize marketing effectiveness over time.
Making Decisions with Unified Marketing Measurement
Developing marketing budgets is difficult. Marketing and finance professionals are constantly challenged to find the right mix and level of investment to drive topline revenue growth and maximize profits. Budgeting is a constant negotiation that exists in every organization. Even when there’s an explicit annual planning cycle, there are invariably those critical moments when the situation changes for better or worse and marketing budgets along with revenue forecasts need to be revised.
Unified marketing measurement holds the promise of providing a structured approach to tackle these challenges, providing a framework that helps with making trade-offs between revenue targets and investment, across multiple marketing tactics. However, realizing that promise is challenging and far from guaranteed.
In this article, we will explore the factors that make marketing investment decisions difficult, how marketing mix models can help or hinder the process, and some tips for avoiding common pitfalls.
Navigating the Three C’s: Complexity, Conflict and Cost
Why is this so difficult?
At the heart of the challenges lie Three C’s – Complexity, Conflict, and Cost – each representing a hurdle that marketers must overcome to make marketing investment decisions that achieve results.
| Complexity | Conflict | Cost |
|---|---|---|
| Multiple products Multiple investment alternatives Multiple marketing tactics Multiple target/geos Non-linear effects Present vs future Uncertainty Ambiguity | Different people Different objectives Different theories/frames Different information Competition for resources Personal objectives Organizational misalignment | Data acquisition Knowledge management Access and retrieval Modeling and analysis Opportunity identification Storage/retention Misinformation |
Complexity means analytical and logical complexity. With a marketing mix it’s like untangling a big hairy ball. Think about the sheer breadth of variables in play – multiple distribution channels, diverse product segments, varying geographic markets, and a plethora of marketing tactics. And then you layer on the fact that there are diminishing marginal returns to investment, so that the incremental returns, while positive, decline when investment amounts increase. And there are different effects over time: short-term vs long-term.
All this complexity in marketing means that there are a lot of different ways to conceptualize this problem, logically and analytically. Marketers and analysts alike are immediately confronted with lots of ambiguity that stems from the multitude of ways to conceptualize and quantify the effect of marketing. Further, most marketing mix models are trying to quantify all this using historical data, which is imperfect. Imperfect data yield potentially biased and uncertain results.
Conflict often surfaces within the organizational structure itself. Marketing teams, armed with varying perspectives and vested interests, may have differing views on the optimal path forward. This conflict extends to strategic objectives, priorities, and the allocation of resources. Balancing these conflicting viewpoints requires not only strategic acumen but also a keen understanding of interpersonal dynamics.
Put yourself in the seat of the manager responsible for the decision. They have multiple different people coming to them, each of them responsible for different tactics, products, etc., and each having a different logical understanding of the problem, and a different information set and analysis. And of course, each responsible person makes the case for investment in their tactic.
Now the decision maker must evaluate all of that, and as they do, they also invariably are not just making trade-offs between tactics and investments, but also making trade-offs on information quality, and between the people themselves. This leads to a lot of conflict.
Costs are driven by the resources expended in analyzing data and modeling scenarios, underscoring the need for judicious and focused analytical efforts. Both Complexity and Conflict push managers and analysts to an endless pursuit of certainty out of fear for making the wrong decision. Decision-makers and analysts incorrectly assume that certainty exists, if only they had the right data. This means that managers and analysts can run up the tab analyzing the data and trying to find the right model to put it all together.
Together, these Three C’s challenge marketers to devise strategies that are analytically sound, organizationally aligned, and financially prudent.
The Role of Marketing Mix Models
Marketing mix models can play an important role in wrangling the Three C’s. To do so, marketers and analysts alike need to understand the role that the model plays in the decision process. The entire decision process can be imagined like a decision-maker sitting on a stool. The decision maker sits on the stool with furrowed brow and makes their decision. The stool is the model. It supports the decision-maker through the process.
When the model is properly built it supports the decision even under the weight of the heaviest of decisions. However, if not properly built the model crumbles and leaves the decision-maker to their own devices. And so, building the model well is fundamentally important.
To support the decision, the model must be built with the decision-maker in mind. In the end, the model should be an extension of the decision-maker’s considerations, objectives, values, alternatives and information.
Understanding the Frame in Marketing Decisions
It all starts with what we, at Keen, call the Frame. The Frame captures the presuppositions. It’s the ground on which the stool sits. We like to think about this in terms of what is in play. What tactics should be considered for the decision, and what factors are necessary to account for in the analysis.
In the end, we want to be able to think about the model as a representation of the drivers of sales. To achieve this goal, it is fundamentally important that the model account for the most important known causal factors in sales including but not limited to seasonality, holidays, price, distribution and supply, product changes, competition, economy, and of course marketing investment. At Keen, users in the Keen Platform start the process by naming all these factors and tag them as either Decision Factors or Environmental Factors.
A good Frame ensures that when it comes time to decide, all the factors that need to be expressed as drivers, outcomes, and alternatives are included in the decision. The Frame is the ground floor underneath the model.
The model itself must, therefore, translate all the factors into values that the decision-maker can compare and trade-off when comparing alternatives. A marketing mix model may provide sales forecasts, ROIs and forecasts of any other non-financial metrics like awareness. At Keen, we focus on the financial value of the marketing decisions because a dollar speaks like no other language. The model in the Keen Platform yields both a revenue forecast and contributions. Contributions are converted to cash flows from marketing investments and forecasted into the future. The marketing-driven cash flows are discounted based on how far into the future they occur and then totaled in a metric we call the Net Present Value of Marketing Investment (NPV). NPV captures both the short-term and long-term value of marketing in a single metric.
Last, but not least, it’s important to use the best information available and use the best methods for the available information. The operative word is available. As discussed above, in pursuit of certainty, too many analysts focus on using the “best” methods. Statistically speaking, the best method means a randomized controlled trial (RCT). Typically, large-scale RCTs are off limits due to the cost and operational requirements in their execution. The requirement to control the treatment, means that they are almost always limited to a single tactic, campaign and execution. This very restricted RCT is what people often call A/B testing.
Marketing mix models, because of their need to support the frame of the decision maker, need to be unified, meaning that they need to capture the range of the tactics influencing sales. This means that we need to have a common method that is appropriate for all the tactics under consideration in the decision.
In traditional approaches, the burden placed on the data to measure all the factors necessary to a unified decision is too high. The data are just not up for it. Even when the data are captured at the lowest levels of observation available, the burden to account for all the factors, measured and observed at different levels, and especially unobservable factors is too great. And so, we must abandon methods that focus only on the observational data.
Thankfully, organizations do conduct A/B tests, and they have direct observations from attribution studies, and they have a team of experienced professionals that include managers, agencies, and consultants. There is every reason for a decision-maker to rely on the information available from them. This is what we, at Keen, call the Information Estate.
To integrate the full Information Estate, users in the Keen Platform leverage a Bayesian estimation approach, which integrates the Information Estate with the observational data. This is effective in handling many of the problems that arise from observational data on marketing including sparsity, low variation and multiple correlation.
Nate Silver, in his book, The Signal and The Noise, does a great job of demonstrating the value of Bayesian methods for many different disciplines including applications to popular topics like politics and baseball. Every day you read your email your spam filter is using Bayesian methods to constantly revise its understanding of which emails go to your inbox versus junk. And Bayesian methods are guiding all sorts of autonomous things from vacuums to vehicles and drones to moon landers.
The Significance of Bayesian Estimation
The Bayesian approach, when done properly, also has the benefit of providing transparency to the information in the model so that all stakeholders understand and trust the inputs and outputs. At Keen, users in the Keen Platform, use Bayesian marketing mix modeling methods so that the model can learn from other sources of information about ROIs. Users can provide ranges of ROIs from multiple sources to be used in the model. ROIs are a user-friendly and transparent way of expressing information about marketing effectiveness in what may otherwise be a highly technical conversation about coefficients.
For the interested reader, credits to Howard Raiffa and Ronald Howard for their developments in the field of decision analysis. The framework used above is based on the Decision Quality framework used by practitioners of decision analysis.
Common Pitfalls in Marketing Mix Modeling
For all the benefits and promise, marketing mix models can crumble under the weight of the decision. When this happens, it causes the decision maker to resort back to using their own and less-informed mental model. It also leaves everyone involved in the process to wonder why they expended such effort, time and money to go through the process. We commonly see three reasons why marketing models fail: mismatched frame, improper use, and missed timelines.
Mismatched Frame occurs when the decision maker desires to make a decision about a tactic that is not in the model. This commonly occurs when there are new tactics under consideration. For example, imagine a manager who wants to invest in television for the first time after having previously invested in digital tactics such as paid search. It’s common for marketing mix analysts to restrict their model to only the historical data, rather than the frame for the future decision. In other words, the model is framed in service of the historical data rather than in service of the decision maker.
A closely related cause for mismatch is that the statistical analyst, desiring to follow statistical practices, will remove variables from the model that are not “statistically significant” according to the data. If there has not been any discussion with the decision maker about the frame, then removing this variable will limit the possible frame. In this case, the analyst relying on the data, is creating the decision frame rather than relying on the decision maker for the frame.
Further, it’s possible for estimates from the model to defy the logical expectations of the tactics. For example, it’s common for some marketing estimates to be negative when there are problems with multiple correlation or for other reasons for price to be estimated as positive. If left in the model, these results can oppose common knowledge and more importantly, oppose the prevailing mental model of the decision maker. If removed from the model, then this can lead to the frame mismatch problem.
Improper Use occurs when the model is used in a way that does not match the natural conclusions of the model. The most common cause of this is reliance on ROIs and Contributions as metrics to inform future decisions. It’s important to understand that Contributions from marketing mix models provide the decision maker with an estimate of the difference between investing at a given level and investing nothing. ROIs typically reflect a translation of Contributions and Investments into financial terms. Literally, ROIs provide an estimate of the value of investing at the simulated level versus investing nothing. This implies that the only two choices are investing at the simulated level and investment nothing, which does not express the idea that investments can be incrementally increased or decreased.
Expanding further, if the decision maker were to consider investments at lower or higher levels, they would be left without information from ROIs or Contributions alone. Unfortunately, many decisions are erroneously made assuming that the stated ROI holds up for any level of investment. If the ROI is low the tactic might be cut all together or, if high, seen as a blank-check endorsement for more investment at any level. Both are erroneous conclusions because they ignore the fact that marketing investments have continuous and diminishing marginal returns. ROI-based decision rules ignore the knowledge that returns increase by spending less and decrease by spending more, and that there exists a financially optimal investment for each tactic.
Missed timelines happen with marketing mix modeling projects when those involved in the effort fail to appreciate all the problems with Complexity, Conflict, Cost, and all the ways in which marketing mix models can fail. This leads to endless iteration and rework. Many marketing mix modeling projects are planned to be eight-week efforts only to turn out to be sixteen-week slogs.
One of the hidden costs to timelines, especially with those who focus on the statistical modeling aspects, is ambiguity. As a part of the upfront marketing data analysis and collection effort, there is typically a lot of ambiguity in the way that certain tactics are defined and monitored. For example, is Influencer a tactic unto itself or is it a social media tactic? What does it mean to invest in Influencer tactics? Just wrestling these questions to the ground can take a week by the time all the people are consulted. Further, these questions may not even be raised until after the first iteration of the model is completed.
How does traditional mix help?
| Complexity | Conflict | Cost |
|---|---|---|
Multiple marketing tactics Multiple target/geos Non-linear effects | Data acquisition Modeling and analysis |
Best Practices to Avoid Modeling Pitfalls
It’s clear that marketing mix modeling not only holds great promise as a unified approach, but also poses great challenges. The road marketers travel is littered with marketing mix models that have gone wrong. So, it’s important to understand some steps to take to avoid the pitfalls.
Start with establishing the frame. When establishing the frame, it’s important to embrace the frame of the decision maker. Identify the person or group in the organization that is responsible for making the marketing investment decisions and make sure the outcome measure, tactics to be decided, and all the other drivers of that outcome measure are included in the model. Ask the decision maker to think forward with you and play out some of the alternatives they might consider and ask questions to get behind the whys. As advocated by Judea Pearl in his book, The Book of Why, developing a causal graph with arrows pointing in the direction of the causal influence between the different factors expected to be in the model is helpful. This exercise will support a conversation about the frame and level-set the model early in the process.
Be a Bayesian. Embrace the Bayesian way of thinking about statistical analysis. These days most machine learning algorithms employ some kind of Bayesian estimation procedure, but there’s more to this than just the algorithm. Embrace the idea that the data are limited in their ability to fully inform the model. Spend time thinking about how the modeling process can leverage all the available information including previous studies, meta-analyses, and even the experience of the decision maker. A good marketing mix model will support a decision, rather than produce a metric. Decisions are ultimately made by people, and so it’s helpful to take the perspective of the decision maker and their organization when informing the model.
Use software. The Three C’s, the data, estimation processes, and pitfalls are just too intricate to leave this to a bespoke effort. Using software will help tame ambiguity and complexity. As discussed above, at Keen, users in the Keen Platform go through a process of enumerating and naming all the factors in their model. The Platform walks users through a process that provides definition and removes ambiguity. Further, users in the Keen Platform take a Bayesian approach. Where they have information in their knowledge estate, they can provide it as an explicit transparent input to the model. And finally, The Platform does the heavy lifting of estimating the model, calculating metrics, forecasting revenue, and calculating NPV. These are all complex calculations that involve simulation from the model, some calculus and financial math. All happens in seconds, keeping the user focused on the support decisions.
Keen Platform addresses pain points
| Complexity | Conflict | Cost |
|---|---|---|
| Multiple products (Portfolio) Multiple investment alternatives (Plans) Multiple marketing tactics (Factors) Multiple target/geos (Portfolio) Non-linear effects (Model) Present vs future (Long-term/NPV) Uncertainty (Risk/Monte Carlo) Ambiguity (Notes, Naming, System) | Different people (User roles) Different objectives (Plan optimization) Different theories/frames (Plans, Priors) Different information (ROIs-to-Priors) Competition for resources (Financial analysis) Personal objectives (UX) Organizational misalignment (ROIs-to-Priors, Plans, Roles, System) | Data acquisition (MEE, Connectors, Data Proc) Knowledge management (System) Access and retrieval (System) Modeling and analysis (Model, Plans, Reporting) Opportunity identification (Plans) Storage/retention (System) Misinformation (Forecast validation, Transparency) |
Conclusion
Unified Marketing Measurement stands at the forefront of revolutionizing marketing investment decisions, offering a beacon of clarity in the often-tumultuous sea of complexity, conflict, and cost. At Keen, the true power is unlocked not by merely estimating a model, but rather through a deep understanding of the strategic role the model plays in supporting the analytical decision-making process. By anchoring our models within a well-defined framework, embracing the richness offered by a Bayesian approach, and harnessing our advanced software tool, marketers can transcend the common pitfalls that beleaguer less informed strategies. In doing so, it becomes more than a model—it transforms into a compass that guides marketers toward making more informed and impactful decisions, thereby ensuring that every marketing dollar is an investment towards a more profitable and insightful future.
To learn more about how Keen is helping teams make better decisions with our tool and how we can help support your next marketing decision, start your model today.
