Sign in with Google 
Join Google’s Ads Product Liaison Ginny Marvin as she sits down with Senior Director of Product Management John Chen to discuss the distinct roles of attribution, incrementality, and media mix modeling, bust the single North Star metric myth, and explore new tools like open-source Meridian and Qualified Future Conversions.
Episode overview
Navigating media measurement today can feel complex and overwhelming. However, learning how to calibrate attribution, incrementality, and MMMs powers both sides of your marketing: giving you a complete picture of performance while fueling your AI-powered campaigns.
And it’s possible even if you don’t have a dedicated data science team on hand.
In this deep dive:
- The distinct roles of attribution, incrementality, and media mix modeling (MMM)
- How and why to value the entire journey, including capturing longer purchase journeys with Qualified Future Conversions (QFCs)
- Creating cross-channel budget projections and scenario plans in Google Analytics
- Why it’s getting easier for marketers to tap an open-source MMM like Meridian without a big budget or technical load
Want Ginny’s key takeaways and tips from this conversation? Subscribe to the Ads Decoded newsletter.
Additional resources
Download the guide to ads measurement in the AI era
Learn what’s required to engineer growth in the AI era with this collection of measurement articles and insights.
Transcript
John: I use an analogy with airplanes. If you imagine you need to fly a plane, data is your fuel. If you don’t have fuel, that plane can’t even take off. So that’s like the foundation. You got to get that set up as real time, as much data that you can provide. You should provide it. Once you’re kind of in the plane, then you’ve got to look at all your gauges, right? That’s kind of how I think about all the different tools we have. So attributions are a gauge, incrementality is a gauge, MMMs are a gauge. You want to look at each one and say, okay, how am I doing? You know, am I at the right altitude? Am I flying in the right direction? Am I flying at the right speed? And then lastly, you got to figure out, okay, what’s the right combination of all of them to actually land that plane.
**
Ginny: Measurement is getting both easier and harder in this era, with more data and tools available than ever before. I’m Ginny Marvin, Ads Product Liaison at Google and this is Ads Decoded.
I had the opportunity to sit down with John Chen, Senior Director of Product Management for Ads Measurement, at Google Marketing Live. We dove deep into the tools to help businesses evaluate their media.
We had a wide-ranging, actionable discussion about the distinct roles of attribution, incrementality, and media mix modeling. And bust the myth of relying on a single ‘North Star’ measurement solution.
We also talk about new metrics like Qualified Future Conversions, why last-click attribution doesn’t tell the full story, and how to use Google Analytics for budgeting and planning on Google and beyond.
Here’s our conversation.
**
Ginny: John, so great to be with you today, and I am thrilled to be talking a lot about measurement and deep dive with you. I want to start with attribution, analytics and MMM. If you can give us a quick breakdown of their distinct purposes and where they should fit into a business’s measurement stack to inform their media decisions.
John: Well, first, thanks for having me on. Very happy to chat about that. So, yeah, I think each one of the ones you described definitely has a place in the stack of how you actually evaluate the effectiveness of all your media. I think first you start with attribution. You get that every day. You look in the system, you look in the tool every single day, that number kind of changes, it allows you to make more real time decisions, so it’s very effective for that. Second, then you start looking at incrementality. Incrementality you probably want to run every quarter, every year. It’s kind of period based, you see it once in a while.
Ginny: And those are the holdback.
John: Yeah, exactly. Those are the holdback studies. So you kind of get a sense of what the causality of the media is, but then you can’t really run it all the time, right? And then lastly, you got MMMs, which have been around for a very long time. They kind of run in the aggregate. People use them every year, every quarter, perhaps they kind of look at things at an aggregate level versus event level. So very useful. And the three of them, sometimes they say the same thing, sometimes they don’t. But it’s really effective to have all three to actually really help you understand the effectiveness of your media.
Ginny: You kind of get that full view of what’s happening now? What has happened in the past, and potentially some predictive indications of what could happen and what if I mix my media allocations differently?
John: That’s right. I mean, I think it’s best that, like most people say, it’s like the best indicator of the future is the past. Like having that data, understanding it, and then being able to kind of do more projections, and then planning for different scenarios, kind of tweaking different things. It’s like a bit of art that comes along with the science.
Ginny: Okay, I think we probably have some people who are using attribution, maybe have thought about incrementality studies, but not quite sure where to begin. And MMM Meridian feels a bit out of scope.
So I guess my question is: I want to talk a little bit more about who these are best fit for. But fundamentally, is measurement getting harder or easier with all these tools.
John: Yeah, it’s a little bit of both, right? In some ways, it’s getting a little bit easier because I think there are more tools out there. There’s more data than there ever has been, and so it helps you make better decisions in general.
However, sometimes when you have too much data, it’s also a lot harder, especially if the data doesn’t always kind of sing the same song, right? They kind of give you different signals. So, for example, your real-time attribution might pick up on something that your incrementality test that you ran three months ago didn’t pick up on. Sometimes the MMM is kind of like seeing things from a big picture, but it doesn’t help you allocate at the keyword level or the creative level, so each one of these tools, you know, you kind of need to use it at the right time at the right place in order to actually help you make the right decisions. But if you don’t, then it can actually cause a lot more noise and confusion.
Ginny: I talked to Eleanor Stribling last season and talking about the importance of Data Strength, so comprehensive tagging, sharing high quality first party data with Google via Data Manager or Data Manager API, and so just love to just hand like drill down onto why it’s so important for AI and performance with AI campaigns.
John: I use an analogy with airplanes. If you imagine you need to fly a plane, data is your fuel. If you don’t have fuel, that plane can’t even take off. You can’t do anything with it. So that’s like the foundation. You got to get that set up as real time, as much data that you can provide. You should provide it. Once you’re kind of in the plane, then you’ve got to look at all your gauges, right? That’s kind of how I think about all the different tools we have. So attributions are a gauge, incrementality is a gauge, MMMs are a gauge. You want to look at each one and say, okay, how am I doing? You know, am I at the right altitude? Am I flying in the right direction? Am I flying at the right speed? And then lastly, you got to figure out, okay, what’s the right combination of all of them to actually land that plane, right? If you just look at one gauge, or one or the other, and not the whole picture, you might not be able to land the plane as well as you could.
Ginny: What kind of data is important? Where we’re talking about conversion data, but not necessarily the closed sale of the conversion. Could be many other touchpoints along the way. What other data, from a conversion standpoint, other data sources should advertisers be prioritizing?
John: You want to look across your business. So, for example, let’s look at conversions for a second. If you’re a company that’s primarily web-based, and that’s where you do most of your transactions, that’s probably all you really need. But if you then have an app, and people are buying in your app, and that represents a large portion of your sales, you’ve got to include the app data as well. And then if you have an offline presence, if you have stores or you’re selling through partners, like if you’re a brand that sells at Macy’s, you’re gonna need to get that offline data as well, because each one is—or last one I didn’t mention Amazon, right? That’s another huge source.
If you’re missing any parts of that, you’re missing part of the equation of how effective was that media, because you might have been able to seen the ads, the user might have done something, but if you can’t see that final transaction where it happened, then we’re not going to be able to measure it. And if you can’t measure it, you can’t actually optimize against it either.
So I think like if I look at conversions, that’s kind of like the first thought. And then if you look at a little bit more upper funnel, sometimes people care about other actions that people take. For example, like adds to cart, website visits, those are really important.
If you start to look at even beyond that, if you have other ad events that might, you know, if you want to have a truly cross-channel kind of like evaluation of the media, then you need kind of all that data.
And click data is a little bit easier, but you probably want to have view data, so you got to bring that in. So once you start bringing all that in, you need to make sure you have all the right data. Not only on the conversion side, but also kind of like the media side too.
Ginny: Yeah. Okay. And for lead gen, for example, you’d also want to bring your qualified leads all along that customer journey.
John: Exactly.
Ginny: Great. At GML, we introduced Qualified Future Conversions (QFC) to capture conversions that occur outside the standard lookback window. So, what problem does this solve for advertisers?
John: If you look at lookback windows in general, right? That’s like one thing that’s interesting. You know, you have your 30-day lookback or your seven-day lookback. But what’s really the difference between a conversion that happens on day 29 compared to a conversion that happens on day 31? There’s not a lot of difference.
The only thing was kind of that artificial lookback window to say, okay, one I want to count, I want to make sure that I optimize towards it, and one I shouldn’t. We think that’s probably a little bit too arbitrary. We’d like to be a little bit more data oriented, so the problem we’re trying to solve is to say instead of having a lookback window dictate which conversions you should count or not, like a very zero to one. Yeah. How about I find a mid-funnel signal that actually helps inform me which conversions actually mattered more.
So that’s where the Qualified Future Conversion kind of comes in. So what we do is, okay, you see an ad today, and then you click on it. Within seven days, do you take another action that the ad influenced you? And that action could be many things. So, for example, one of them could be an attributed brand search. For example, if you saw a Nike ad, you clicked on it, and then within seven days, you went on Google.com and you searched for Nike, or you searched for Jordan Ones.
That’s a very strong user signal that that ad played a big role. And you can think of many other signals. For example, I visited Nike.com, or I added a Jordan One to the cart. All of those could be the signal. As long as something happened within that one week, then I will try to see how many conversions actually happened within the next 30, 90, 180 days, up to six months.
And so, for purchases that kind of take a little bit longer, kind of evaluation, you know, buying cycle, this is perfect because then it allows you to say, I found a really clear signal that happens within a week, and I’m going to count all the conversions that happens up until 180 days.
Ginny: Great, and so it’s whether you have short or longer conversion cycles. Obviously, if you have longer conversion cycles, this is going to be especially important. Yeah. But even for short conversion cycles, you’re going to be able to kind of pick up more of those signals that you didn’t have visibility into.
John: Absolutely.
Ginny: Media mix models have often historically been seen as only accessible to big companies with big data teams, and for a mid-market advertiser with fairly limited technical resources, where can they start with Meridian today?
John: I think the legacy of MMMs, kind of like what you described. You really need a large engineering and data science team to kind of get one up and running before you can even look at the data or look at the analysis. However, I think that’s why we did Meridian was because we wanted to really democratize this and allow any smaller company with just a handful of engineers or data science to really get running with the open source code that we have, but we thought that’s probably still not enough. We probably can do more.
So earlier this year, we released Meridian Studio, which is on Google Cloud Platform, where you just kind of go in there, you can run the Meridian module on top of the BigQuery data you already have, and that probably eases that a little bit more.
In addition, we also announced at GML, Meridian within Google Analytics. So really taking it even another step further to say, can we really make this turnkey? You just log into Google Analytics, you kind of set up, you follow a few steps, assuming you have all the data connected already with your analytics product, then you can start to run Meridian.
So instead of spending all of that time, kind of like setting up the model, you know, and analyzing it and ensuring that it’s actually working, you can actually spend the time looking at the data, looking at the analysis, and seeing how effective your media was.
Ginny: And just want to clarify, so it’s Google Analytics 360. But also earlier this year, I believe we introduced the scenario planner for Meridian, also built for marketers specifically to be able to get more out of their Meridian data, their models, in a way that feels much more intuitive for the marketers.
There does seem to be a real workstream to help bring Meridian to marketers versus marketers having to go through data teams to get to Meridian.
John: Yeah, that’s exactly right. Right, we rather have the kind of user be able to look at the data, look at the analysis, look at the outcomes, and that’s the nice thing about Meridian. It kind of provides you both the backward looking as well as the future looking.
So you can look at how did you do in the past three years, and then kind of create all these different types of scenarios where you can say, oh, if I what if I change my budget here? What if I change my budget there? You know, what would happen? How do I really optimize my spend for next year? That’s what we think the real value is, and that’s where we want our customers to spend that time instead of like spending the time to set up the model.
Ginny: Yeah, I was just talking to an agency CEO a couple weeks ago, talking about how they have a couple of technical people on their team, and they have built a model for clients and said it was almost immediate the value that it brought to be able to see where they were undervaluing channels, and automatically talked to the client, said, look, this is what we’re seeing. Yeah, redistributed their budgets and just had their best month ever.
John: So great to hear.
Ginny: Yeah, I thought it was so fascinating because it really was such a great indicator of how much more accessible this type of tool is.
John: Yeah, and it really kind of takes, I would like to think, the hard work and in some ways the innovation out of it because Meridian is going to be ever evolving, right? We’re going to have innovations like we did around reach and frequency. We’re going to have innovations around Google Query volume. That’s great, and we’re going to have many more things that we’ll kind of build into the Meridian platform. But you’d rather have the team not have to worry about getting the latest innovations into their MMM models, but instead being able to take advantage of those and make better business and optimization decisions.
Ginny: Great, and for people who don’t know where to start, I will give a shout out. There’s a great video series that the team put together in terms of understanding what an MMM is and getting started with Meridian in a way that felt really accessible.
John: So great to hear.
Ginny: Yes, so I would highly recommend checking out those videos. We often see a data tug of war where the incrementality test said one thing, but the platform attribution says another. And when those signals conflict, which one should a practitioner treat as kind of the north star for their optimizations.
John: North Star is tough, you know, because North Star effectively means that’s the one direction you should be going towards, and I think if you try to use one or the other alone to do that, I think it would be really hard. And one of the reasons, maybe start with incrementality when we talk about the causal studies and experiments that you run, they are really, really good, right? In that they tell you if I had the ad and I didn’t have the ad, how effective was that ad in getting me extra incremental conversions that I might not have gotten?
That’s great. That’s what you want, but the problem is you can only run them for a certain amount of time, it takes a lot of energy and time to set up that experiment. Make sure you have no contamination. Everything’s set up correctly, and then run analysis on it to make sure you understand. Okay, which slice was the most incremental, and then it’s only for that certain period of time, so it’s only reflective of that time period. So, for example, if you run it during spring, how relevant would it be for Black Friday? That’s a little bit of a thing that you have to kind of figure out.
In addition, it costs you a lot, meaning that every time you hold back the ad, you lose some of the sales that you might have gotten at the time. So, ideally, you would run the incrementality studies all the time, but then you’re having that holdout that never actually converts into sales. That is expensive. That affects your bottom line as a business, and so most customers are not able to run it all the time.
And then if you look on the flip side, you look at attribution. Well, attribution is all the time. You get all that data. It’s streaming in. It’s great for real-time analysis and understanding where the business is going at that moment. But then the problem is you don’t know which attribution or which conversions were incremental and not. What if you didn’t have the ad? Would you still have gotten that conversion? That is where the attribution model is not able to tell you, so that’s why when you look at it, I think if you try to follow either one as a North Star, it will lead you in very different directions.
I think the best practice is probably to utilize both and then really use kind of the judgment, the experiences, the different various tools that might be out there to help you effectively try to triangulate what the right North Star is.
Ginny: So helpful, and also puts the marketer right in the center of the decision making. Exactly based on the data that’s coming in.
How do we accurately measure channels that don’t have a click? YouTube top of funnel video view campaigns, for example, podcast ads. How do we measure that within this framework without over crediting the last click channels that happen to be at the end of the journey?
John: Yeah, I think that’s one of the biggest challenges that the industry is going through right now, right? Last click is just such an easy thing to do, right? Okay, you see the last click, you give it the credit, that’s great, but what that really misses out on is all of the upper funnel media that plays such a huge role in getting you to that last click, getting that customer primed and ready to actually convert. You have to find a way to appropriately assign that credit, and I think that’s where attribution falls a little bit short.
So you want to be able to try to move away from first of all the click. So ideally, moving towards more like data-driven (DDA), especially one that incorporates views, because that’s one of the biggest weaknesses of the attribution model. So media like YouTube, TikTok, CTV, all of those miss out on a last-click attribution model. So you really want to try to move away from that. You want to try to build in more incrementality. You want to utilize MMMs as another triangulation source. There’s a number of things you can do to kind of get off of the last click attribution model.
Ginny: Okay, and if you’re still on last click, what do you recommend? Is it just, move over to data-driven attribution?
John:
Okay, we’re ready to move off. Yep. What’s the plan of attack there? Okay, if you’re ready to move off, then I would say there’s probably three things to follow, right? The first one is I would move the attribution model off of last click.
So what that means is that you want to move to data-driven attribution. So you can build the full path of all the different events that happened, and not just consider clicks. Add the views in there as well. So there’s a little bit of Data Strength that you need to make sure that you can pull in views, especially if you’re looking at cross publishers, right? If you want to get views from Facebook, TikTok, YouTube, you’re going to have to source those from different places and put it all in one place so that you can do this new attribution model.
The second thing I would do is I would run incrementality studies so that you can calibrate the attribution model. So, for example, you want to run the study so it can test to say which media was more effective than others, which format was more effective, creatives, keywords. All of those are really good items to test in a causal test.
The last thing that I would do is kind of create this triangulation area that you can do, where you can get an MMM to also run similar analysis a little bit more frequently than maybe even quarterly, which then allows you to say, okay, my MMM, which tends to value upper funnel media a little bit better, and then my attribution, which will probably value the lower funnel ones a little bit better. The two combined will probably give you the full picture of the entire funnel.
Ginny: And thinking about bringing your cross-channel data, you can bring cost data from your campaigns into Google Analytics. Is that playing a role in attribution, or is that really more about helping for understanding where you should be allocating your budget.
John: Yeah, I think it does a little bit of both. So in Google Analytics what you’re able to do is you kind of have two different types of data connectors. The first one I would say is more at the event level. So you say, okay, I’m really focused on the attribution model. So I need all the clicks, all the views from all of my different publishers. So this would include Snapchat, Facebook, Pinterest. So let me get all that event level data in.
There’s another type of data import that you can also do, which is more at the aggregate level. So what that means is, hey, I just want to see the last two years of all of my Facebook spend, Facebook impressions, as well as my Pinterest, my CTV, and I want to bring all of that in. That will help you with the MMM model.
And then maybe a third one is cost, right? So you need the media cost for all of this, and that one can also be an aggregate. When you have those three things combined, it will give you really strong attribution in analytics. It will give you really strong MMM results in analytics, and then with the cost data, it will then allow you to kind of calculate the ROAS as well as start doing planning and optimization to say, give me the best ROAS with this spent.
Ginny: Okay, so bringing that data in will give you a better, bigger picture and inform your planning.
John: Yeah.
Ginny: All right. Most practitioners are held to a specific ROAS or CPA target, and how do they explain to their clients or their stakeholders who are used to seeing that one-to-one attribution why a holistic model that might show lower immediate returns is actually more accurate?
John: That’s always a little counterintuitive, right? But I think one of the practitioners I talked to in the past have said you need to make sure you plant seeds. If you’re always harvesting, then at some point you’re going to run out of things to harvest, and so it’s really important you plant that seed for the next cycle, right?
So therefore, like even if you’re a commerce player that’s really focused on Black Friday, it doesn’t mean you spend all of your spend on Black Friday. I think everyone knows that, but at the same time, that ROAS on Black Friday is probably going to look a lot better than your June ad spend, so advertising you kind of need to be able to make adjustments throughout the year to do that. And one way you can do that is so, for example, in your attribution model, if you do it correctly and you give enough credit to the upper funnel, then that ROAS is fine.
Other ways that people have done it, is they look at that ROAS and say, well, maybe I should look at incremental ROAS. So if I run a causal study, can I figure that out? That will also help you with upper funnel.
And then the last one is they’ll say, well, make kind of modifications to the ROAS. So for example, like even if the ROAS, let’s say in June is 3x larger than it normally should be, like in Black Friday. Well, I’m gonna drop it down by a third because I know that advertising spend will help me in November.
Ginny: I’m so glad you brought up the Black Friday. We are going to be hurtling towards holiday season, and I know a lot of companies are already dialing up their spend to essentially start feeding that funnel, and rather than turning on the pumps on Black Friday, so any kind of holiday tips of advice for making sure that merchants are ready for the season and they’ve got their measurement in place and their data plan in place.
John: Ah, it’s a good one. I mean, when I think about Black Friday, I definitely think it’s like a unique phenomenon, right? There’s so much happening at that moment. So what you want to make sure, I think, is you want to first of all, you want you want to have your data strength there, right?
Because if you don’t have the data strength, kind of especially with more real time than not, then it’s going to be really hard to pick up on the signals, the differentiations that are happening during that time. So now I’m just thinking about like that last three to seven days right before Black Friday. So having that real time data will help the kind of optimization tools kind of really do their thing the most accurate way.
The second one is I really believe in this planting of the seed. So you want to think you want to look at your full year or maybe the nine months before you leading up to Black Friday and thinking about all the different media that you know plays a role - through because of incrementality tests, through MMMs - that say this is going, this investment will pay off in six to nine months.
And then having that full picture to say this is how I’m going to do a steady pulse of advertising versus just the last minute stuff. A lot of the last minute stuff sometimes a lot of them will be very incremental, but some of them you might have gone just because Black Friday is so powerful. So really, trying to figure out which media you can do to kind of plant that seed, super important.
Ginny: And I think also I’m sure helps those conversations when you’re asking for more media budget from the CFO or the board or your clients. So all of it kind of fits together. So what I hear you saying is start now as soon as you can and have a long term investment plan for having an always on data strength and measurement plan.
John: That’s right.
Ginny: Well, thank you so much, John. This has been great, and really appreciate your time. Absolutely happy to be here. Thank you. All right.
**
Ginny: All right, navigating media measurement today might feel complex, but as John shared, having a multi pronged framework is going to give you a fuller view, more robust insights into your media effectiveness.
Here’s what I took away from our chat.
Number one, triangulate your data instead of chasing a North Star. No single measurement tool is going to tell you the whole story. Real time attribution, periodic incrementality holdouts, and MMM or media mixed modeling all serve distinct purposes, acting like different gauges on an airplane’s dashboard, to John’s analogy.
Number two. Move beyond last click and incorporate view and mid-funnel signals if you’re not doing that already. Relying strictly on last click attribution can undervalue your upper funnel channels like YouTube, CTV, social video, which prime customers to convert. Switch to data-driven attribution model that is going to incorporate that view data across publishers.
Number three, plant seeds early rather than just harvest. chasing short-term ROAS can lead to underinvesting in those upper funnel growth channels, especially before peak sales. relevant as Black Friday will be here before we know it.
And if possible, import your event level and aggregate campaign data across channels into Google Analytics and use that for budgeting and planning purposes.
If running an MMM like Meridian sounds daunting, be sure to check out our special companion episode. I chat with Adventure Media’s Patrick Gilbert and Nachama Tegan about how they made Meridian part of their measurement stack and did it with little technical know-how before going into it. So that’s gonna be a great conversation. Be sure to check that out.
And if you’ve enjoyed this episode, subscribe, leave a review.check the show notes for helpful links. Thanks so much for listening. Until next time.
