Do Web 3 quests work?
Measuring the native Web 3 ad unit
This first stage of the mythological journey — which we have designated the ‘call to adventure’ — signifies that destiny has summoned the hero and transferred his spiritual center of gravity from within the pale of his society to a zone unknown.
Joseph Campbell, The Hero with a Thousand Faces
A subject of continual amusement in Web 3 is how much of it is both incredibly novel, and a complete rerun of similar mechanics in Web 2.
Take questing, for example, or ‘offer walls’ as they’d be known to anyone who works in Web 2 gaming: Publishers like Layer 3 or Rabbithole or Galxe offer a destination for current or potential Web 3 users to collect points and rewards for engagement with Web 3 protocols. These quests (‘offers’ are slightly more descriptive if less heroic) are virtual goods giveaways: sometimes tradable stuff like tokens, often just questing ‘points’ within the platform. The quest’s challenges—they’re usually a sequence of steps—are often something simple like connecting to Twitter (thus revealing the user’s Twitter handle, much like retail loyalty programs ask for a phone number). Sometimes it’s more complex and meaningful like swapping or bridging funds into a certain token or protocol, a real first-step to using the end product.
By incentivizing that initial taste of a dApp, game, or protocol, the questing platforms (and their clients) hope to draw long-term users into the experience. The questing platforms charge for the campaign much like Facebook or Google charge for ads, though the business models are somewhat different. Most ads campaigns these days are CPA (cost per action), while quests charge either on a per-campaign or per-completed-quest basis. Whatever the model, if there’s a customer acquisition cost (CAC), then there’s a return on advertising spend (ROAS) that one can use to judge the questing campaign.
To date, nobody has been able to really measure questing ROAS because Web 3 has been devoid of real cross-Web 2/Web3 measurement. Spindl fixes this, providing an easy way for advertisers to quantify metrics like monetization and retention for any inbound channel: Twitter, Telegram, Discord, quests, whatever.
The Case Study
In this case study, Lyra, a decentralized options-trading protocol, already uses Spindl to measure their user retention and monetization from all channels. Though embryonic, Web 3 marketing is already ‘omni-channel’ in that marketers use multiple strategies operating in tandem and measured side-by-side. A channel that doesn’t show up in the attribution dashboard essentially doesn’t exist from the marketer’s POV. Here, we can see Lyra’s inbound channels, measured by users who generate more than $50 options premium (Lyra’s unique active-user definition).
Spindl worked with Layer 3 directly, who provided us the wallet IDs of the users that participated in their quest. One aspect of quests are unique to Web 3 and potentially very powerful: Because users sign in with wallets on the questing site (both to collect rewards and as an anti-Sybil measure), quests are some of the biggest ‘wallet aware’ publishers around. That’s both more respectful of Web 3 privacy—people are cool with on-chain data being public—and much easier from the attribution perspective. We don’t have to muck around with legacy Web 2 events like clicks and pageviews or transient and invasive forms of identity like cookies. (Yes, we’re turning this into its own product; more soon.)
Even with the identity problem solved via wallets, this is somewhat trickier than it might sound, as a priori we have no idea what incremental usage the Layer 3 quest actually drove. Was it really the questing platform, or did that user actually first come in via Twitter or Telegram?
Note: The plot above isn’t simply the aggregated stats from a simplistic Google Analytics-like approach that looks at link URL parameters, nor is it simply running a Dune query with those wallet IDs (and assuming all engagement came from the quest). No, it’s the output of a consistent attribution model that evaluates all the events leading up to the user monetization event, and judging which one drove the incremental value. As with quests themselves, this is a reboot of the attribution function in the Web 2 space, but fully adapted to the Web 3 context.
Show me the numbers
So did the numbers work out?
In short, yes, the quest campaign was ROAS positive, to the tune of 1.88x or so. In human terms, that means for every dollar you put into the quest, you got $1.88 back…a success.
As a comp, this is about what you’d get for a healthy Facebook campaign nowadays. Given the absolute bloodbath that Web 3 marketing tools like token drops typically are, this is a big success. Coupled to accurate measurement, designers of quests (typically the platforms themselves, sometimes working with the advertiser) will have much more data with which to optimize their campaigns.
Some more numbers from the Lyra/Layer 3 campaign:
Layer 3 generated around $450,000 in trading premium for Lyra, or about 8% of all premium generated by Lyra during that time. Much of that went to pay LPs, with the balance returned as fees (i.e. profit) to the protocol. Our ROAS number correctly accounts for all the LP and hedging costs of those trading premiums, and is based only on net profit to the protocol. User-wise, Layer 3 engaged a total of 2105 wallets, with just over half being new users who had never touched Lyra before.
While retention was below that of Lyra users in general, it’s typically true that paid media drives less monetizing and retaining users than organic media in general. You’d see the same in the split between Facebook Ads-acquired users and organic ones. It’s not surprising you’d see the same in Web 3, particularly with an ad format and targeting that are still in the very nascent stages. If we’re on day one of Web 3 marketing, and we’re hitting current FB-level ROAS, it’s a good sign for the ecosystem.
For now, Spindl will continue to work with Web 3 publishers like Layer 3 to measure their downstream value to advertisers like Lyra, and (very importantly) compare it accurately to all marketing channels. Web 3 marketing is basic now, but it won’t stay basic for long. Soon enough, nobody will be looking at makeshift Dune queries or single-channel analytics to judge the value of a growth campaign. It’s a startup cliché that you make what you measure. For too long Web 3 marketers haven’t measured much of anything. Now they can.
Sign up for more news from the Web 3 marketing trenches!