Why I Don't Trust Vibes-Only Startups: Metrics I Actually Care About

Outline

Hook: “Our users love it!” Cool. How many came back this week? “We’re getting great feedback!” Great. How many paid? “The vibe is immaculate!” The vibe doesn’t pay your bills. Show me retention, usage frequency, and willingness to pay.

Core Argument: “Vibes” and anecdotes feel good but mask reality. The metrics that matter aren’t downloads or signups—they’re retention, engaged usage, and paying customers. If you can’t measure it, you can’t improve it. And if you’re avoiding metrics, you probably know something’s wrong.

Key Sections:

The Vibes-Only Trap
- What it looks like: “Everyone we talk to loves it!”
- Red flags: Cherry-picked testimonials, vague success stories
- Why it’s dangerous: Feels like validation, but isn’t
- The reality: 10 enthusiastic users don’t mean 1000 will care
- Personal experience: Almost got fooled by early positive vibes
Vanity Metrics vs. Real Metrics
- Vanity: Downloads, signups, page views, social followers
- Real: DAU/MAU, retention curves, revenue, NPS from paying users
- Why vanity feels good: Big numbers, ego boost
- Why real matters: Predicts long-term viability
- Example: 10,000 signups, 100 active users → Problem
The Metrics I Actually Track
- Retention cohorts: Week 1, Week 4, Week 12 retention
- Activation rate: % of signups who complete core action
- Engaged usage: Not just login, but using core features
- Time to value: How fast do new users get their first win?
- Willingness to pay: Free trial → paid conversion rate
- NPS from paying customers (free users don’t count)
- Support ticket rate: Higher = something’s broken
- Feature usage distribution: Which features actually get used?
For 99 Minds: The Numbers That Matter
- Ideas captured per week (engagement)
- Return rate: % who capture 2nd, 3rd, 10th idea (retention)
- Search usage: Are people finding past ideas? (value realization)
- Free-to-paid conversion (validation)
- Churn rate and reasons (what’s breaking)
- Not tracking: Total ideas stored, app opens (vanity)
Red Flags I Watch For
- Startup talks about “momentum” but not retention
- Celebrates big launch numbers, silent about week 2
- “We’re pre-revenue” after 2 years
- Quotes from non-paying users
- Focuses on features shipped, not features used
- Dashboard full of green arrows but no revenue
How to Set Up Real Metrics (Tech Stack)
- Event tracking: PostHog, Mixpanel, or Amplitude
- What to track: Core actions, not every click
- Retention cohorts: Weekly views, grouped by signup date
- Revenue tracking: Stripe + custom dashboard
- User feedback: In-app NPS, support ticket analysis
- Cost per user: Essential for sustainability math
When Vibes Actually Do Matter
- Pre-launch: Qualitative feedback guides design
- V1: Intense user interviews > metrics
- Edge cases: Users tell you what metrics miss
- Culture: Happy team ships better products
- Balance: Metrics show what, vibes show why
- The synthesis: Data-informed, not data-driven
The Metrics Hierarchy
- Tier 1: Revenue, paying users (survival)
- Tier 2: Retention, engagement (product-market fit)
- Tier 3: Growth rate, CAC, LTV (scalability)
- Tier 4: Everything else (interesting but not critical)
- Focus: Nail Tier 1-2 before obsessing over Tier 3-4

Examples/Stories:

99 Minds: Early vibes were great, but retention showed UX issues
Client project: Celebrated 1000 signups, 50 stayed → pivot needed
Competitor: Huge launch, no retention, dead in 6 months
Success: Boring metrics (70% retention) → sustainable business
Personal: Stopped tracking vanity metrics, focus improved

Takeaways:

Vibes feel good, metrics tell truth
Track: Retention, engagement, revenue
Ignore: Signups, downloads, page views
Red flag: Celebrating growth without retention
The goal: Data shows what’s working, vibes show why

Cross-Links:

← “Building Tools for a Law Firm” (Series 2-15)
→ “Idea Cemeteries” (Series 2-17)
→ “From One-Off Project to Sellable SaaS” (Series 2-18)
← “Your MVP Is Trash” (Series 2-11)