Web3 Won’t Save Us: How product analytics contaminates decentralized movements
We’ve all heard that web3 is here to save us.
That web3 will rescue the internet from the walled gardens, attention hoarding, and algorithm-powered addictions that web2 platforms have established over the past decade.
Evangelists in the web3 community tell us that decentralized ownership, collective decision-making, and composable design will return control of the internet back to everyday people like you and I.
But web3 won’t save us — in fact, right now, web3 can’t save us.
In the next 4 minutes, I’ll explain why...
Why Web3 Is In Danger
We’re starting to see the move from cryptocurrencies, exchanges, and blockchain infrastructure layers into the application layer of web3 — known as ‘dapps’ (short for ‘decentralized apps’). This growing surface area of web3 platforms is driving a tsunami surge in venture capital investment.
Investment into web3 startups grew from $2.1B in 2020 to over $30B in 2021. All this venture capital funding will be used to hire designers, developers and PMs from web2 startups that are eager to build the next wave of products for the web3 world.
As those employees migrate over to web3, they bring with them the processes and ways-of-working from their previous employers. Chief among these is the use of web analytics and ‘north star metrics’ to inform data-based decision-making. Looking back a decade from now, this will be seen as the source of web3’s future problems...
According to Abbie Kouzmanoff, Group Product Manager at Amplitude, there are three different “games” companies can play when using behavioral data from web analytics to make important decisions: engagement, tasks completed, or transactions. In plain English, this means that the products we all use are optimized by specialized teams to extract increasingly more time or money from users (ie. you and I).
In a 2010 interview, Sean Ellis described north star metrics as “a single metric, that if improved, makes everything better, easier, etc.” He said that “with a North Star metric, everyone wins - the company and the customer.” But over the 12 years since, the concept has diverged from its original meaning. North star metrics today are generally completely unrelated to delivering more value for the customer or end user.
North Star Metrics Won’t Build A Better Web
Audius, a decentralized digital streaming platform, is the largest consumer dapp in the web3 world with over 6 million monthly active users. Audius is most often compared to Spotify, Soundcloud and Apple Music as a potential future competitor.
Spotify’s north star metric is “time spent listening”. If Audius wants to create a better streaming platform than Spotify, should they really focus on maximizing the amount of time that listeners spend engaged on their platform? Or should they instead focus on understanding the biggest pain points within the music industry and the biggest frustrations fans have with existing streaming platforms? I would bet my ETH on the latter.
Same goes for Medium versus Mirror.xyz — its decentralized alternative. Medium’s north star metric is “total time reading”. If Mirror.xyz focuses on beating Medium on this data point, they only need to fill their user experience with sidebar distractions and rabbit hole recommendations to keep users forever engaged. But is that how they should go about delivering a better publishing platform for individuals? At the end of the day, that is the promise of web3.
How Did We Get Here?
David Perell’s essay The Microwave Economy opens with this gut-puncher line:
"We’ve overwhelmingly used our wealth to make the world cheaper instead of more beautiful, more functional instead of more meaningful.”
The evolution of tech follows an analogous path. We’ve overwhelmingly used our advancements in technology to make the internet more engaging instead of more impactful, more addictive instead of more valuable.
The cornerstone of our internet addiction is social media. The companies that invented infinite scroll and algorithmic content feeds have influenced the shape of the web2 internet in ways that aren’t always appreciated. For example, the founders of three of the largest web analytics tools today all worked at a major social media company immediately prior to founding their respective startups: Amplitude’s CTO worked on Google+, Mixpanel’s CEO interned at Slide.com (which built social widgets for Facebook), and Heap’s CEO was a PM at Facebook.
It made sense for social media companies to optimize towards engagement — with ad-based revenue models, they knew that the longer user’s eyeballs were on their products, the more dollar bills they earned. By building analytics tools to track engagement, they had a directly influenceable metric that would grow revenue.
But the analytics tools they built are no longer confined to social media. Amplitude, Mixpanel and Heap have become the decision-making engines for every single startup in the world. Even the major startup accelerators have dedicated modules on implementing analytics tools and choosing which “game” you’ll play for your north star metric.
You wouldn’t be considered faint of heart if you thought that the proliferation of behavioral data as the central currency of decision-making in web2 was beyond repair — I’m tempted to agree. But that’s exactly why we need to work now to ensure the same thing doesn’t happen in web3.
So, How Do We Fix This?
Quick recap: the behavioral data that’s used by web analytics tools makes companies optimize what users are doing, so they end up trying to maximize the time and money users spend with them. On the opposite side of the coin, we can see that the promise of web3, according to the Ethereum Foundation, is to “put power in the hands of individuals rather than corporations.”
If we are to transition the central figure of the web from gatekeeper companies to the empowered and collaborative individual, we’ll need to find a way to guide the teams building decentralized apps towards optimizing for value and impact rather than time and money spent.
Don’t get me wrong, I don’t think web3 teams should abandon data-based decision-making. Rather, I think we need to build a new robust, scalable, quantifiable source of data based on people’s priorities. What are customers’ biggest problems? What pain points are impacting them most right now? What goals and values are most important to them today? To build a people-centric internet, our data must shift from analyzing behavior to analyzing priorities.
The 100 Year Old Solution
Thankfully, we already have methods for quantifying people’s priorities — in fact, we’ve had them for quite some time. And it’s pretty simple too :)
‘Pairwise Comparison’ is a voting method which takes two options and asks people to pick which one they feel strongest about. These pair votes create a web of comparisons which can be used to calculate a relative importance score for every option in the set.
L. L. Thurstone, pioneer of the modern IQ test, first introduced a scientific approach to using pairwise comparisons all the way back in 1927. His approach went on to inspire the ELO Rating System used in Chess and the Glicko Rating System that powers player leaderboards in many of the best known digital games in existence like Dota, Go, Chess.com, Dota, and Pokémon.
At my startup OpinionX, we’ve built a platform for running pairwise comparison surveys. OpinionX is used by over 1,500 companies including Unbounce, Instacart and web3 startup Gnosis to stack rank user priorities so that they can better inform key product decisions.
While stack ranking on OpinionX today looks more like a survey tool comparable to SurveyMonkey or Google Forms, we’re working on integrations with Segment and Stripe that leverage product and revenue data to predict and segment the priorities of your entire userbase based on only a tiny subset of participants.
Web3 carries the genuine promise of building the internet for the individual user. But unless we rethink the causes that led web2 astray, we’re destined to repeat its mistakes. I believe that giving companies better data — data that tells them what people care most about rather than just what they’re clicking on — will be the key to building a healthier, more impactful and more valuable internet for us all.
If you enjoyed this essay, subscribe to our newsletter for more essays on actionable user research: