Rephrasing the question of "What is incrementality?"

Emma: Hello and welcome back to the Commerce Collective podcast brought to you by Flywheel. I am your host Emma Irwin, and today we are talking about incrementality - again! We're going to cover what we learned in the last year and where we think it's leading us. Without further ado, let's re-meet our guest for today's episode.

Gabe: Gabe Fishbein, VP of Product at Flywheel.

Emma: Let’s start here with you giving me your definition of incrementality.

Gabe: I tell a joke that is - if you get three people in a room and have them define incrementality, you'll get four definitions. And so I like to keep it simple, which is a sale that would not have happened without seeing an ad, or it could have been a customer that purchased or wouldn't have purchased without seeing an ad. And so when you get past philosophy and into measurement, you start to see all the different ways that people define it. But overall, it's “How effective is a specific ad unit or series of ad units that drive a user journey into ending in a purchase culminating in a purchase?”

Incrementality has been the buzzword across different folks in the industry over the past year. And it's something that we still really, truly believe in as a concept. It just sort of requires this shift in what you pay attention to.

Historically, everyone obsessed over ACOS and ROAS, but now we’re seeing the customer as this much more valuable person, that it's not just a purchase but a person behind that purchase. The picture expands to be much bigger than just ROAS or ACOS.

Wind the clock back 10 or 12 years and there were a bunch of brands that were building out these DTC businesses on the thesis that if you have customer data and you can build a relationship with the customer, they're going to be worth a lot more.

I think a great example is Dollar Shave Club, which raised a whole bunch of money and netted a big old sale to a large CPG. And what did they end up with? Distribution at Target.

All of these businesses were built on the assumption that if you know your customer and you can email your customer and build a relationship with your customer and get a subscription with your customer, then they're going to be a lot more valuable to you.

The challenge is that nobody wants to have 50 subscriptions and get their soap from one place and their pillows from another. And so we're seeing this happen in tandem with a lot of the clean room tech that's starting to come out which is then giving brands the ability to begin to own a little bit more of the relationship with the customer in a privacy-safe way so that you can begin to say, “Hey, who are the kind of customers that I want to show ads to and where are they in their journey?

Then it’s how do I measure grabbing a new customer or getting a customer to switch channels into one that I can measure well? 

Emma: The last time that incrementality was discussed in depth on the podcast, we went into the best approach being to look at consumer behavior across categories, essentially creating these queries with some assumptions and searching for whether or not consumers are behaving in this “clicky or scrolly” manner. And therefore that helps us understand incrementality at this very high level. But with more data and clean rooms, can you tell me about how we're now looking towards the evolution of incrementality?

Gabe: One of the big challenges with the “clicky and scrolly” sort of customer behavior is that it’s just changing. It isn’t irrelevant, but retailers are continuing to increase the level of personalization in search results. So it's not just where you are, it's not just what device you're using, but it's what are your past purchases and what you have a higher propensity to buy.

So the historical way of measuring incrementality was - let's take all of these averages across all of these different weird data sources. Let's make a bunch of estimates and then try to back into what somebody would or wouldn't have done and subtract it from what they ended up doing.

But, you can't look at the people, you just have to look at these broad-based averages that exist within your campaign metrics or whatever. So it leads to this incongruence between the thing that you want to measure and the thing you're measuring. The thing that you want to measure is customer behavior, but the thing that you're measuring is just a bunch of averages and percentages that try and tell you what portion of your sales would or wouldn't have happened. It just doesn't have the nuance necessary.

Emma: Ok, how do we measure that customer behavior so that it's beyond just an estimate?

Gabe: Enter the clean rooms. I think anybody who's spent any time in any sphere of ecommerce over the last three years is familiar with these clean rooms that exist. What the clean rooms do is give you the ability to see customer behavior at an aggregated, privacy-safe level. Instead of asking what would or wouldn't the customer have done, you can now actually look at the data and say, “Ok, what hadn't they done before - that I can say deterministically they have not done - and then they exhibited a certain behavior.”

So that's where you can then say, “Somebody is now new to my brand, and new to the channel.” We know that there are oftentimes consumers who aren’t purchasing laundry detergent for the first time, but maybe it’s the first time they’re purchasing it via ecommerce, which is channel-shifting behavior. There's this moment of truth where someone is now searching for laundry detergent no longer in the aisle of the brick-and-mortar store, but now via ecommerce.

The subscription behavior and the repeat purchase behavior are so strong there that it's really important to capture that customer - not just figuring all the touch points leading up that give them a higher likelihood of buying - but also measuring the downstream impact after that first purchase. Did they buy that same SKU again? Did they buy another SKU within that brand? Did they buy another SKU in a different category within your portfolio?

This then allows you to remove all of that funny estimation and say, “Who is new? How much did I pay to acquire them? How much am I paying to keep them? And how much are they purchasing?”

Emma: So we're talking about net new-to-channel customers and measuring their value based on the purchasing behavior with the brand over time. This is taking it a step back but I am curious, can a customer or can a sale still be considered incremental if someone has purchased something from the brand before? So they're not net new, but is it possible for incrementality to live on and span past that first new-to-brand purchase?

Gabe: I think the question that you seem to be getting at is if somebody were purchasing the brand of laundry detergent in the store, could you or how would you measure that? Especially if they might also be purchasing it online, they might not be incremental.

Unfortunately, there's not a great way to do it. The next obvious question everybody's asking is, hey, how do you measure across these things?

The whole point of the walled garden is data. But you can’t get the data in or out in a way that you can map across. Did somebody browse in-store and then purchase via e-commerce from a different retailer? You can’t measure that, but you can use high-level estimates.

There's always going to be this area that you're not going to see, which is why I talked about not just new to brand and new to channel, but the shift of customers from a place where you were having to measure them in aggregate via the orders that you're sending to a specific brick and mortar retailer, to the ability to aggregate these customers in a privacy-safe way and understand at a high level what their behavior is in a very measurable channel with a lot of signals that you can then take further action on.

That's the difference and that's the net marginal additive value that even if they're not incremental to your top-line report to Wall Street revenue, they're still incremental to your business and you can drive additional value via further activity against those customers.

Emma: Would you say that advertisers are ready to be looking at incrementality in this way?

Gabe:  Our goal as an agency is to always push the thinking forward. And so are they ready? It varies, brand to brand and person to person as well. I  see our job as measurers and executors, but also as educators. Everybody wants the same thing, which is sales. Generally speaking, I think you could look at it as, “What is the value addition to a top-line number?”

However you calculate that - we're getting better and better and closer and closer as we sort of try and tie the two together.

And so are they ready?

I hope so, but if they're not, I hope we can continue to bring people up to speed and continue to have them sort of pressure test our thinking and push back where they disagree or don't understand and that we focus on really good principles and we get smarter together.

Emma: As you were talking, I was thinking about clean rooms in general. What about measuring incrementality for retailers that don't have a clean room readily available for use?

Gabe: You get back to some of that funny estimation math. The principles generally still apply across retailers, which is where on average, if something is ranked low, it probably is more incremental and you can do this subtraction with differences in an SP and all the math.

But you see many of the non-Amazon retailers subtly making moves to either build this kind of tech or build out some of these metrics, these new to-brand or new to the channel or new to-category metrics that are oftentimes still a good way to sense out if the customers are new, which is ultimately the thing that you want to drive.

Emma: Do you have a feeling that maybe incrementality won't be one of the metrics that we push as something that matters not too far down the line?

Gabe: I don’t think incrementality is a metric. We haven't touched on this yet, but incrementality is a code name for a different thing that you're trying to figure out. Where I'd like to push the thinking is toward more specific and measurable outcomes.

So new-to-channel customers, new-to-brand customers, new subscriptions, these are the much more measurable things. And then you get out of the need to try and agree or disagree on a definition of incrementality. It's just getting much more granular and prescriptive about what you want to measure. And then, how do you optimize against that metric? Because the industry will not ever have a standard definition of incrementality, it's almost sort of not worth chasing as a singular metric.

Emma: It's kind of like, why did we expect people to be able to agree on the definition of a made-up word to begin with?

Gabe. Exactly. 2020 is when we launched incrementality stuff and when you typed it into Google Docs, it was underlined in red because it was not a real word. I think we want to continue just to be more specific and more academic when it comes to something as important as measuring impact on your overall business.

Emma: Tell me a bit more about what you’re thinking about when it comes to the future of incrementality and metrics or concepts that matter.

Gabe: That's the really fun, experimental stuff that we're doing around some of the broader-based impacts to sales across a bunch of different dimensions. Redefining things as new to the brand is the first little scratch on it, but overall just continuing to advance the ball, not just on the measurement piece, but also on the action ability piece.

So all of this is great, but can you affect any of these metrics?

That's where this clean room tech starts to become a bit more useful. You get into things like bottled audiences and other ways that you can influence some of these metrics in a way to grow your top line.

And so this is the fun experimental stuff. I like to say that we went from sort of CPC and ROAS to CAC and LTV, and so it's just going to continue to evolve, it's going to continue to be measured in different ways. It’s figuring out how you manage against your measurement to drive the outcome that ultimately the client wants. It's working very closely with clients, hearing what they care about and then just obsessing over chasing that.