The long user march to the blockchain
How to get to a billion users on Web 3
This is the first in a series of case studies from Spindl, the attribution system for Web 3, on the gritty realities of Web 3 user growth, monetization, and retention. Subscribe to follow along.
In 1861, French civil engineer Charles Minard drew a flow chart representing Napoleon’s march to Moscow the winter of 1812, depicting with genius simplicity the brutal losses suffered by the French army.
In Napoleon’s doomed march to Moscow and back, we see how his army is rapidly decimated, first slowly via battle and attrition, and then quickly due to cold, illness and physical obstacles. Every Russian ambush or frozen river is represented by another sudden narrowing of the imperial march (note how the retreating line narrows on crossing the Berezina River).
The terminating hair-width line next to the departing swath illustrates the scale of the catastrophe. Minard represents no fewer than 5 different parameters—location, army size, march direction, geographic points of interest, temperature—in one easy-to-follow chart. Data visualization guru Edward Tufte, in his magnum opusThe Visual Display of Quantitative Information, declared: ‘It may well be the best statistical graphic ever drawn.’
It will seem a hyperbolic metaphor perhaps, but users marching across the creaky Web 3 user experience undergo a similarly fraught trip. A non-crypto degen from Web 2 who wants to check out what all the Web 3 fuss is about has about as much chance of getting and staying on-chain as a French infantryman did of reaching and holding Moscow in 1812.
Take the wallet sign-in process for many Web 3 projects: It’s so hideously hostile to normie users, I suspect if you’d shipped anything like that at Facebook, Zuck would have thrown you out a window. That’s of course changing with novel ‘Web 2.5’ hybrids like Reddit’s NFT avatar experiment, which represents a possible way forward to mass adoption. Estelle from Dubuque, who wants to play whatever social game went viral among her set (like Farmville was once popular among Cuban old ladies like my mother in Miami) isn’t going to custody a private key or even know what chain she’s on. She just wants to play cool games and have novel experiences unavailable on Web 2. To the extent Web 3 developers provide that experience—and you don’t get there without measuring metrics like user retention—then the blockchain may well serve as the foundation for a new Internet, but not otherwise.
A non-crypto degen from Web 2 who wants to check out what the Web 3 fuss is about has as much chance of getting and staying on-chain as a French infantryman did of reaching and holding Moscow in 1812.
Recently, Spindl worked with leading Ethereum scaling solution (and Spindl investor) Polygon to measure the success of a Polygon Football NFT mint (all finished, but you can see it here). The structure of the experience is fairly simple, thus serving as an ideal experiment for the Web 2 to 3 conversion funnel.
(As often happens when we configure a Spindl conversion event—effectively, a tiny subgraph that captures the critical value transfer—we have to do a bit of finagling given the differing ways developers architect user experiences into smart contracts. In the Polygon mint case, the user wallet didn’t actually mint the NFT, it simply signed a transaction, and a central Polygon wallet later transferred the NFT to the user (in order to skip the funnel-killing step of gas fees in MATIC). Polygon smoothed their onboarding, and Spindl mapped the low-level detail to the higher-level funnel concept that really matters. We do that a lot.)
So here’s what the user funnel looks like:
At the top of the funnel, we have the usual Web 2 suspects of Twitter, Telegram, etc. (though with widely varying degrees of engagement, as we’ll see). The Polygon marketing team used Spindl leaks in their posts, which was the ideal user path as we got the full funnel drop-off from click to chain.
Given the viral nature of NFT mints, direct links to the Polygon mint site were shared separately on services like Twitter and Telegram. Even though that traffic didn’t come via Spindl links, we picked up those clicks via page-view events that Polygon fired from their front-end. Whether via Spindl links or not, Spindl eventually catches all the page-views and on-chain conversion events, which is the source of all the below data.
Starting with the direct data, we see that the Twitter funnel for Polygon performed exceptionally well, driving an overall Tweet click to mint conversion rate of about 25%, which is very good. As a comp, clickthrough rates on media like Facebook ads are in the mid single-digits, and the conversion rate on your typical landing page is <10%. The Polygon tweet did very well indeed. Shares on LinkedIn and Instagram got relatively little engagement.
Here’s a screenshot from the Spindl dashboard (contact us for a demo) with full funnel numbers. Perhaps most noteworthy is the number of mints per unique wallet is something like 14:1, which is crazy high (more on that in a bit).
Then there’s the ‘organic’ traffic, which is traffic that goes directly to the Spindl-instrumented Polygon site without going through a Spindl link. Once again, Twitter performed extraordinarily well, with click-to-chain conversion rate of almost 40% (!). Telegram also did well, and Facebook and YouTube did adequately (though at lower volumes). Google performed miserably, which is an interesting inversion. On Web 2, Twitter traffic usually performs terribly while very directed Google traffic does well; Web 3 is apparently the reverse.
Unfortunately, Discord doesn’t pass its referrer URL in the HTTP link, making their traffic difficult to separate. You’d have to use direct Spindl links from Discord to track that traffic, and the mint ended so quickly the Polygon marketing team never even bothered to post separately. The ‘null’ traffic was partly Discord, and (very likely) bot traffic going to the Polygon site(s) and programmatically harvesting the mint.
How do we know? For starters, the average mint/unique wallet number of 14-ish was way too high: nobody loves football that much. Also, a useful post hoc analysis is taking all wallets in the mint or drop, and looking at the first transfer transaction after the NFT collection. Plot that as you would a network graph, and what you see is what looks like bacteria colonies on a Petri dish:
This is a common pattern in airdrops that we’ve seen before: ‘sybil’ attackers creating wallets to collect the drop or mint, and then transferring it to one wallet for an immediate dump. This isn’t that bad (we’ve seen worse).
The thickness of the line corresponds to the number of transfers, so some attacks were a dozen or fewer wallets minting everything and then passing to the dump wallet, while some where more subtle and trafficked the NFTs as singleton mints. This is what fraud in Web 3 looks like. You see it again and again in any drop situation, and represents nothing but pure loss to the developer or advertiser.
You can see the fraud also in the histogram of NFTs minted, with a large fraction of wallets minting all 32 NFTs. Soccer AKA football is popular, but it’s not that popular. This was clearly programmatic drop-farming. The NFTs were soon listed on OpenSea, where they trade for low floors…yet not so low to avoid the interest of the fraudsters.
On the Web 2 side, the attribution system is often drafted into helping with what’s still a nagging problem: filtering any fake-seeming data from the attribution machine. In Web 3, the challenge is more on the targeting side, since drops constitute the only form of targetable paid media that developers can use (dare I say ‘ads’?).
Could there be some way to flag these wallets before transferring the NFT? Well, after the fact, yes, as fraudulent wallets have one post-mint transaction: transferring the NFT to the sales wallet and making off with their bag. So blacklisting is easy.
Before the drop, you do see that older wallets tended to have more post-mint engagement, with most wallets engaging with Polygon further (82% vs. 14%). A simple classifier with a wallet transaction history as feature inputs would likely smoosh most of the current fraud (until the sibyls adapt, and so on the ML arms race).
So did the mint work?
320,000 mints happened in just under 48 hours, so that much was a success. Did it increase engagement with Polygon? It did in the weeks after, as we see both WAUs and weekly transaction counts spiked, as they often do for already engaged wallets after a drop or mint (they tend to work less well in new-user acquisition). Of course, success depends on the success metric—’you make what you measure’—and in this case we chose on-chain transactions (which is what L2’s tend to care about). A game or Defi protocol would have have very different goals, and very different Spindl conversion events being fired. Marketing ‘lift’ is something we’re still trying to standardize among our various integration partners.
As far as we know, this is one of the first real Web 2 to Web 3 conversion funnels measured and shared publicly. Much of the traffic performed very well indeed: Napoleon’s troops got to Moscow! The bigger question is whether they stay here (and we’ll be posting about retention and cohort analyses in future case studies).
With ever more engaging consumer apps, there’s no reason why Web 3 shouldn’t be able to drive mass adoption via both new and old marketing channels, if it can manage to measure what’s going on. Thus far, Web 3 has been flying blind, but no longer.
If the "organic" traffic is some unknown % of traffic originating from discord, and the rest likely bots hitting the Polygon site directly, why not just disable organic traffic entirely? By limiting traffic to originating sites such as Twitter, you are getting the benefit of whatever bot/spam protection they hav e in place - although I guess it wouldn't be that hard for a bot to grab the Spindl tracking URL from a Twitter post. Still, its one more gate to go through and considering that it looks like bots were responsible for ~6X more unique minters than the next best performing channel (Twitter), it might make sense just to drive all traffic through known trusted channels. Yes, I know this defeats the purpose of a "trustless" network such as web 3.