Video: The Only Product Metric That Matters
Are People Using Your Product?
Sounds like a simple question, but the answer can be really complex — like how scientists still debate what, exactly, keeps bicycles upright.
Fortunately, Josh Elman, a current Partner at Greylock Ventures and formerly of LinkedIn, Twitter and Facebook, gives us a framework to understand the question — the product question, not the bike question — so we can answer logically in a way that pertains to our individual products.
Watch the video (duration: 25 minutes, 7 seconds):
Key Message
“If you really try to benchmark against everybody else, you are not understanding what’s meaningful about your product.” – Josh Elman
How Can You Tell If People Are Using Your Product?
Consider the following as they relate specifically to your product:
- Purpose: Why do they use your product?
- Core Action: What is the key behavior?
- Cycle: Do they use your product at the frequency you expect?
The temptation is to compare your product against an established product. But when you look at the Core Actions and Cycles, the comparison may be ridiculous.
For example:
| Product | Purpose | Core Action & Cycle |
| Stay connected with friends | Check in with friends and see what they’re up to. Sharing 2-3x/day. | |
| Airbnb | Book a place to stay | Search and book trip options. 3-5x/year. |
| Find and be found | Find: Search for other members 2-3x week. Be found: Respond to inquiries when you receive them 1-2x per year. | |
| Yelp | Find local businesses/services, and review them | Searchers: Search for businesses 2-3x week. Reviewers: Write a new review 1x/month. |
I know how tempting it can be to look at your daily user number and be disheartened because it’s not like Facebook’s daily user number.
But if you’re not making a new social network, that daily user number may not be your key metric. Look at Yelp, for example. Is looking for a plumber going to be a daily habit? Probably not.
Instead of trying to optimize for a daily user metric, a product like Yelp would focus on a weekly user number to see if users are coming back within the right time frame to use Yelp for its intended purpose.
Your Core Users Are Your Best Users
Next, Josh provides insights into determining your core users (e.g., your best users).
| Concept | Definition | Track |
| Top of Mind | Do they come directly, not because of a reminder, notification, or special deal? | % users with direct traffic / engagement |
| Recurring | Do they come back again and again? | % users who return weekly/monthly |
| Referring | Do they share your product with others? | k-factor and % users who are referrers |
Unexpected Gold
To close out this talk, Josh looks back at his time at Twitter when they were struggling with getting people to move from casual users to core users.
They discovered a cohort of folks who initially abandoned the product but came back. These “bounce back” users provided a treasure trove of information, helping Twitter better understand different activation buckets and onboarding.
Questions they asked the bounce backs included:
- What prompted you to sign up and try our product in the first place?
- What did not meet your expectations / what was hard to figure out?
- Why did you come back to give the product another try?
- What worked the second time that got you using it more frequently?
If you’re fortunate enough to have users give you a second chance, be sure to respect and take advantage of the insights they can provide you.
Funnel Issues
If not enough people are showing up, Josh recommends incentivizing or engineering Word of Mouth, showing off what you can do with the product, or leveraging a network effect.
If people sign up but don’t stick, consider onboarding not as a first-time user activity but as a way to get new users to become core users.
Specific onboarding activities include:
- Learn Flow: Teach your users your product as they sign up. Don’t rush it.
- Welcome Week: If users don’t do it the first day, nudge them to return and keep going. Use email to educate new users on the product.
- Social Pressure: If friends are on the system, use them to welcome, teach, and connect with the new folks.
How About You?
Bicycle dynamics isn’t exactly an in-demand realm of scientific inquiry so we may never know for sure how bikes stay upright.
But every product manager needs to understand how to develop profitable, sticky products that users love.
Have you tried any of Josh’s recommendations, or done something differently? What did you find out?
