10 biggest misconceptions about marketing mix modeling
Apr 8, 2024
Introduction
It’s easy to see why marketing mix modeling (MMM) is more popular than ever before. MMM only relies on first-party data—a key advantage as privacy laws restrict third-party tracking—and predicts marketing’s sales impact in real time thanks to machine learning and cloud computing advances.
So why do so many marketers still assume they can’t use marketing mix modeling? The problem is MMM’s 40-year-old reputation: that it’s an exclusive, expensive, time-consuming methodology reserved for only the largest B2C enterprises.
Today, MMM couldn’t be further from that image. The methodology has skyrocketed past the 1980s (even if puffy sleeves and power suits are back), becoming an accessible way for any company to evaluate marketing channels’ ROI.
We wrote this post to show that MMM works for a wide variety of companies, from P&G to Doordash and Asana. Let’s break down the common misconceptions floating around MMM so you have the information you need to evaluate the methodology yourself.
1. MMM is only for B2C enterprises running large brand campaigns
Why do marketers think this?
Marketers believe this myth because it was true in the 1980s when businesses began using MMM. The methodology was expensive and time-consuming because teams could only generate data manually, and computers were much slower than today. Because MMM was so resource-intensive, it was only available to the Coca-Colas and McDonald’s of the world.
The assumption that MMM is only for brand marketing also stems from its history. In the eighties, large B2C companies just used offline brand marketing channels—like TV, radio, print, and billboards—so that’s what they measured with MMM. Marketers didn’t use MMM to track online channels in real time because they hadn’t been invented.
What’s the truth?
Marketing mix modeling today isn’t limited to large B2C businesses. Any company, whether Nabisco or a tech startup, can use MMM as long as it has enough data for its model. Check out this post for tips on evaluating whether your business has sufficient data to produce valid results and forecasts.
MMM also isn’t limited to large brand campaigns. Here’s our hot take: there’s nothing inherently different about brand versus performance advertising. All marketing can be tracked to sales—whether its impact is direct or indirect—so all marketing can and should fit into MMM.
Consider the work management software Asana. It’s a large B2B company, but it’s nowhere near the size of the B2C enterprises marketers typically associate with MMM. Yet Asana relies on MMM to assess the effectiveness of its online performance channels, like its YouTube, search, and Facebook ads. Read more about how Asana uses MMM here.
2. MMM insights are always out-of-date, so you can't use them to make decisions
Why do marketers think this?
Like the last myth, this misconception stems from how MMM operated in the eighties.
The large CPG brands using MMM then had large, complex datasets for their models, which all had to be manually collected, validated, and inputted into spreadsheets and statistical software. Likewise, the computers they used to run their MMM and calculate their marketing activity’s impact operated slowly. By the time their MMM produced insights, the forecasts were out-of-date.
What’s the truth?
MMM forecasts today are based on real-time data. Unlike legacy firms that took months to refresh models, today’s agile vendors can ensure their clients’ MMMs are always up-to-date with two technologies:
APIs allow the model to pull data from each marketing channel or from your warehouse instantly, rather than teams needing to enter information manually.
Machine learning enables the model to automatically interpret historical data, determine correlations, and calculate marketing’s impact.
At Paramark, we’re constantly refreshing models for clients so they can use their MMM to create scenarios and make decisions based on the latest information.
3. MMM results can't be validated
Why do marketers think this?
Some marketers don’t trust MMM’s correlational insights because of a classic principle from science class: correlation doesn’t equal causation. In other words, you can’t assume each channel’s impact from your model’s data because the MMM estimates each channel’s effect on your target metric.
What’s the truth?
MMM insights alone aren’t definitive, but you can validate the model’s predictions by assessing the quality of its analysis and running marketing experiments.
Because incrementality testing shows causation, you can use it to check the accuracy of your MMM results. Imagine your model shows that billboard advertising is associated with increased sales. Your team could check whether the marketing channel actually influences revenue with an experiment: picking two identical regions and placing a billboard in one. Once the test is complete, you compare sales in each area to determine the impact of billboard advertising.
Internal data engineers or MMM vendors can also validate models with statistical techniques. We recommend checking the model’s “fit”—how well it predicts outcomes—with mean absolute percentage error (MAPE). This metric reflects the difference between your model’s actual and expected values.
4. MMM can only use weekly or monthly data
Why do marketers think this?
They still imagine the MMM of the eighties, like myths one and two.
In the early days of MMM, marketers could only use monthly data —or weekly if they were lucky—because adding information to the model was cumbersome. Teams had to manually pull data from floppy discs and enter it into their MMM. This process was time-consuming, so marketers couldn’t pull daily data for their MMM.
What’s the truth?
Marketing mix modeling today can use daily data. In fact, we recommend companies only start using MMM if they have at least one year’s worth of daily data.
Unlike the offline channels of the eighties, online marketing channels today can send daily data to your MMM via their API. Meta, for example, offers an API for marketers to automatically retrieve their daily Facebook ad impressions and spend for their MMM.
Remember that almost every marketing channel, not just Meta, can provide daily data through an API. So if an MMM vendor says they can't build a model with a daily aggregation, you know that’s a red flag.
5. I can’t trust my MMM because it yields different results than touch-based attribution
Why do marketers think this?
Marketers have been using touch-based attribution since the early 2000s, so they see its data as a source of truth. MMM, on the other hand, only gained widespread popularity in recent years. Naturally, marketers who have been using touch-based attribution for a longer period will be skeptical of MMM’s conflicting results.
What’s the truth?
You should still trust your MMM even if its results contradict your touch-based attribution data. At Paramark, we expect this to happen because touch-based attribution is biased towards digital, direct-response channels.
Imagine an ecommerce brand that wants to know how its marketing channels impact sales. Its touch-based attribution data shows that Google Search and Facebook ads have a higher ROI than their MMM’s data for the same channels.
Why? The MMM also factors in channels that aren’t trackable by touch but contribute to sales over time. For example, the model would account for a subway ad the ecommerce brand ran that drove thousands of impressions every day.
Touch-based attribution’s limited view is why we don’t use it to measure a marketing channel’s ROI. It’s an excellent tool for understanding your digital buyer journey, but not for measuring incrementality. We explain the pitfalls of touch-based attribution in detail here.
6. Doing experiments to validate or improve MMM is too expensive and risky
Why do marketers think this?
Some marketers are wary of using experiments with their MMM because testing feels expensive. You spend money creating an ad, but then you only show it to a portion of your audience.
What’s the truth?
Incrementality testing may result in a short-term opportunity cost, but that risk is worthwhile considering the long-term costs of no experimentation.
Without testing, you can’t confirm the causality of each channel’s impact on sales. You save the short-term cost of an experiment, but you could be investing much more in a channel that isn’t leading to positive target metric results—and you don’t even realize it!
Experimentation shows how a channel affects your target metric, so you know your marketing budget is going where it counts.
7. Firms that have been doing MMM the longest are using the best methodologies
Why do marketers think this?
It’s natural to assume that the businesses building marketing mix models for the longest periods would be the most qualified. Typically, the company with the most experience has the most expertise.
What’s the truth?
Years of experience alone don’t signal MMM expertise because legacy firms often don’t use the latest tools and technologies to build models.
That’s a significant downside. Marketing mix modeling today produces real-time results and is more accessible thanks to machine learning and cloud computing advances. When firms stick to the methodologies they used 20 years ago, clients see the familiar problems of legacy MMM: it’s hard to implement and operationalize, and it’s too expensive.
Instead of focusing on whether a company is a legacy MMM firm, consider what’s important to you in a vendor. Our clients at Paramark typically prioritize agility and actionability—being able to refine the model and experiment—along with cutting-edge technology and simplicity.
8. MMM is complex to execute
Why do marketers think this?
MMM can seem like a resource-intensive way to measure marketing because, on the surface, it looks like it requires a lot of engineering. Statistical concepts, like multi-linear regression analysis and confidence intervals, can intimidate marketers unfamiliar with the domain.
What’s the truth?
Marketing mix modeling looks complex on the surface, but it actually involves minimal data engineering compared to other measurement tools.
Touch-based attribution, for example, requires teams to collect and stitch together user-level data—like personally identifiable information, device IDs, session IDs, and UTM codes. This process is especially difficult today with privacy regulations restricting individual tracking.
MMM, on the other hand, requires aggregating platform-level data across channels. Even better, reputable MMM vendors will handle the entire process of building and executing models, so there’s no lift for internal teams.
9. I should have our internal data scientist build our MMM
Why do marketers think this?
Some teams assume this because they know their internal data scientists can build an MMM. Models involve simple data engineering and science, so most data scientists have the skills to create and manage an MMM.
What’s the truth?
While internal data scientists can build MMMs, it’s typically not the best use of their niche skills. They’ll have to dedicate the majority of their time to creating and refining the model and staying on top of the latest MMM innovations, even though their experience is an entirely different part of engineering.
Imagine an engineer who’s worked at some of the biggest consumer mobile app companies—-Facebook, Snapchat, Twitter, and Pinterest. You wouldn’t replace this engineer with just anyone if you’re building an app because you know how well their experience matches the work.
The same is true for building an MMM. Any data scientist can build a model, but that doesn’t mean they can make the best possible MMM for your company.
On top of creating the model, the data scientist will have to interpret the MMM’s results for non-technical marketing team members, adjust the MMM forecasts based on the marketing team’s feedback, and interface with them on incrementality testing. In other words, it takes a lot of time for the marketing team—not just the data scientist—to maintain the MMM in-house.
A vendor provides relief by fully handling the MMM and bringing a breadth of experience. Paramark, for example, has analyzed over $100M in clients’ marketing spend using MMM. With our background, we ensure clients’ models produce valid results and assess the latest MMM processes and technologies.
10. Geo-holdout is the only type of lift test I need for MMM
Why do marketers think this?
Geo-holdout testing can seem like the only test you need to complement marketing mix modeling because it offers so many benefits:
It uses aggregated data, rather than user-level data, just like MMM.
It’s privacy-friendly because it doesn’t require user-level data.
It works with most channels.
What’s the truth?
Geo-holdout lift testing works well with MMM, but like any methodology, it has drawbacks. Regions can be challenging to isolate (like areas with commuters from multiple states), and marketers can’t always successfully scale campaigns based on geo tests since locations have unique properties.
Geo-holdout also isn’t the only type of lift testing available. Marketers can also assess their models with:
Native conversion lift testing: a channel showing your ad to one subset of users on its platform
On/off testing: turning off spending for one channel to see how that change impacts your target metric.
Pulse up/pulse down testing: increasing or decreasing spending on a channel while keeping spending the same for other channels.
Make MMM easy for your internal team by partnering with a vendor
There’s a lot of misinformation about MMM out there. We’ve cleared up the most common myths, but this post is just the tip of the MMM knowledge iceberg. MMM processes and technologies constantly improve, so staying up to date on the latest information is a full-time job.
Partner with a vendor like Paramark to handle every aspect of your marketing mix modeling. Our team has analyzed over $100M in marketing spend with MMM, and we spend every day living and breathing the latest MMM news. By building and maintaining models for our clients, we help them answer critical questions, like:
How much of marketing's contribution is incremental?
What's the return on marketing investment for each channel?
Where should we invest our marketing spend based on MMM forecasts?
Curious about partnering? Chat with our team today.
