💡 Idea validation framework

+ 4 ways to validate your next startup idea

Hey hey 👋,

Welcome to the 184 founders and investors who have joined the Consumer Startups family since the last post! If you haven’t subscribed, join thousands of other consumer founders and investors by subscribing here:

---

For today’s post, I am bringing back one of the most popular posts in the last 2 years - the idea validation framework.

TLDR:

There are four main ways to validate a startup idea:

1) Signal Aggregation - Conduct experiments (e.g. Landing page, Ads CTR, etc) to validate initial hypotheses without launching a product.

2) Strong Beta - Get strong traction with a Beta product. Metrics will differ based on the vertical (e.g. 40% retention for social vs $10K monthly net revenue for marketplace).

3) True Fans - Find early users that will be very disappointed if the product goes away. The number of these fans does not matter as much.

4) Visionary - Have a clear vision about the product you want to build and how you will build it.

---

💡 How to validate your next startup idea

Some background info…

In the year or so since the pandemic sent us all into our homes, I have interviewed over 100 founders (mostly venture-backed) in the early-stage consumer space for this newsletter, Consumer Startups. It’s been such a humbling experience to learn from such amazing people solving important problems.

One question I’ve asked in nearly every interview is, “how did you validate your idea?”.  As a young entrepreneur, I used to think that all founders followed the lean methodology popularized by Eric Ries in his book, The Lean Startup. In the book, Ries proposes methods that help startups manage the uncertainty that characterizes startups’ business prospects. He advises that early-stage companies focus on establishing the minimal viable product and the Build-Test-Learn loop as a way to find problem/solution and product/market fit for a product idea.

The framework itself is helpful and offers potential experiments for idea validation. However, the more I spoke with successful founders, the more obvious the gaps in the framework became to me. I simplified all of them into two main reasons:

1. The framework doesn’t provide founders with enough examples about how other successful startups have validated their ideas, nor does it personalize to B2C founders.

2. It might not be the best approach for all startups.  Every startup is born into unique circumstances ranging from differences in industries to leadership goals. There are a ton of ways to validate a startup idea and achieve massive success.

My goal: Create a better idea validation framework for B2C Founders.

The data I have gathered through 100+ interviews is generalizable because I’ve targeted founders working across different sectors, ranging from online marketplaces to direct-to-consumer products. 90% of the founders I have interviewed have successfully raised venture capital funding AND reached a level of traction that’s validated their initial product-market fit.

To find patterns, I began by first summarizing each founder’s core insights on their own idea validation journey (check out the initial insights list here).  Then I grouped relevant quotes by theme on a Miro board.

(Initial thematic coding)

*Note: The term idea validation can be vague. For this study, an idea is successfully validated if there is significant early traction for the company after the product launch. There is no one representative metric due to significant variance across industries. Common metrics are user and revenue growth, and it is different from the product-market fit idea proposed by the lean framework.

Four approaches to idea validation

Four main approaches emerged from my interviews with B2C founders. All four approaches have panned out to be fruitful for the founders involved, and there is no clear ranking among them. 

1. The first approach is Signal Aggregation. This approach does not require shipping a Beta product, but it does involve using market research and running experiments, such as leveraging a landing page and social media ads to validate both the viability and demand for the product.

2. The second approach is Strong Beta. This approach requires the successful launch of a Beta product, which is usually buggy and not elegant in terms of the user interface.

3. The third approach is True Fans. Similar to Strong Beta, the True Fans approach also involves launching an early product; however, the goal is to identify a cohort of enthusiastic users rather than achieving strong traction in terms of the number of users.

4. The fourth approach is Visionary. This approach features the least amount of research and experiments. It’s built on a clear vision from a founder who has a deep, personal connection to the problem. In these cases, the entire product is often built with minimal hypothesis testing. 

1) Signal Aggregation

The Signal Aggregation approach is a great way to quickly validate an idea without building an MVP. A signal is defined as evidence that proves the potential viability and profitability of an idea. Founders conduct small experiments to collect those signals. Here is a list of examples of this approach that I have seen work for founders: 

1. Launch a landing page and assess waitlist sign-ups

2. Launch on Product Hunt to gauge interest from tech-savvy early-adopters

3. Conduct research to see if the idea has been attempted in the past or in another country

4. Design and presale the product without the actual product to see how many people click on ‘buy’

5. Run social media ads and assess click-through-rate

6. Build out a social media account (e.g. Instagram, Slack, Facebook, Discord, etc) to gauge engagement from target users

7. Make a TikTok and assess engagement to see if the idea resonates with people 

8. Talk to people about the idea and ask if they would be willing to pay for the product during customer interviews

Case Study - Yotta Savings

Company Overview: 

Yotta Savings is an app that allows users to win lottery tickets by depositing money in a savings account. It graduated from Y-Combinator and raised a $13.2M Series A in January of 2021, just a few months after raising a seed round. Yotta has acquired over 130k users that have deposited over $200M in total [as of January 2021]. 

What no one knows though, is that before founding Yotta, the app’s founders experimented without 4-5 ideas that did not work out, but they focused on the idea behind Yotta after experimenting through Signal Aggregation

Signal Aggregation:

The team used three different methods to gather signals for this idea.

1) They got positive feedback from people who worked on Premium Bonds in the UK - the same model has worked in the UK for decades. 

2) They found that Michigan Credit Unions had piloted a similar prize-linked savings account product called Save to Win, which acquired over 10k members and received more than $8.5M in the first year of the pilot program.

3) They ran FB ads with a landing page and built up a waitlist of 2k+ people.

Who it is best for - This method is the fastest way to execute without building a functional prototype. It is also helpful for industries that require large upfront capital or present structural barriers to entry, like governmental regulations in the FinTech space.  

On the flip side, this method might not give the most direct insight into the needs of the user since click-throughs on ads, or waitlist sign-ups do not indicate how users will use the product.

2) Strong Beta

Getting strong traction from an early (Beta product) is another popular way for a founder to validate their idea, but success metrics differ across industries. For example, tens of thousands of users and a 40% one-month retention rate are considered strong for a consumer social product, while $10K - $50K net monthly revenue is favorable for a B2C marketplace. 

It usually starts with an initial hypothesis for a potential solution to a big problem.  The founder builds the simplest version necessary to validate their hypothesis. Beta products are often difficult to use and do not have the same aesthetic appeal as other products in the marketplace due to the speed at which they are made. This approach is most similar to the lean approach, which focuses on launching a product and iterating quickly. 

Case Study - Run the World

Company Overview:

Run the World is a startup that makes it easier for anyone to create and monetize a virtual event. It was founded in 2019 and has since raised over $14M from GGV, Founders Fund, a16z, and more. Between March 2020 and the beginning of 2021, there were more than 15K events hosted on Run the World in100+ countries.

Beta:

Founder Xiaoyin and her team built the Beta version of the virtual event platform in 14 days. It was a low-code solution that combined existing products such as Wix, Slack, Mailchimp, Eventbrite, and several other tools. Xiaoyin hosted the first event herself using the Beta product and was able to make over $3,000 dollars from it. They helped run 4 other virtual events and made over $70K in ticket sales. They used the momentum from this strong Beta performance to raise a $4M seed from a16z, Pear VC, and GSR Ventures. 

Who it is best for - Early traction from a Beta product is the strongest indication of whether or not the product is solving a strong user pain point. It helps the team to better understand the user behaviors driving the use of the product. It is also a helpful method for novel ideas, such as consumer social apps, since consumers won’t know whether they like the product until they have tried it. 

On the flip side, this method can be more time-consuming than Signal Aggregation since it takes time to ship a functional product.

3) True Fans

This approach is similar to Strong Beta in that it also requires the launch of a Beta product. However, the main differentiation is that instead of looking for significant traction in terms of revenue growth and user acquisition, the founder is looking for fanatic users that love the product despite its limited features. Fanatic users are defined as anyone who would be disappointed if the product went away. Many of the founders I spoke with mentioned that the number of fans does not matter beyond 20-50 users that absolutely love to use the product. 

Case Study - Boomy

Company Overview:

Boomy leverages machine learning and AI to empower users to create music instantly and earn royalties from Spotify and other streaming services. It was founded by a music industry veteran who has a deep relationship with this mission of democratizing music creation. Since the MVP launch in 2019, the platform has attracted 100s of thousands of users from around the world that have collectively created over 2.2M songs, amounting to  ~2.5% of the world’s recorded music.

True fans and MVP:

The founder and his team started Boomy as a part of Boost VC, which is an accelerator program in the Draper Venture Network. They launched an MVP product within three weeks, and unfortunately, the MVP didn’t produce great music. In fact, it took six minutes for the system to make a song, and 98% of the time, the song was rejected by users, meaning users thought the music created was bad. Sometimes, the system would even break and create a blank song. However, that remaining 2% was really good and got saved. Users that saw the 2% of good music stuck around and got excited about the songs they’d created. Those early users gave the team enough conviction that the idea was worth pursuing.

Who it is best for - This approach is helpful for products that do not easily lend themselves to virality, such as a productivity tool. This method sheds light on the user persona most likely to become an early adopter of the product. However, similar to the Strong Beta approach, it can be more time-consuming than Signal Aggregation or Visionary due to the required engineering time.

4) Visionary

This is the least common approach from the founders I have interviewed since it requires a clear vision about the product you want to build, and how you will build it. In most cases, the founder has a close personal connection to the problem they are solving and knows what needs to be done to solve it. 

Case Study - Geneva

Company overview:

Geneva offers a community group chat product to empower community leaders to build private communities that foster meaningful connections. It uses the similar internet relay chat (IRC) framework that Discord and Slack are using to build a chatting platform for private communities. The founder has a clear vision about the UI/UX of the product and believes that channel-based communication is more organized and efficient than having everything flow in a single thread, like iMessenger or WhatsApp. They were founded in 2019 and have raised over $22M to date. They are used by many groups, ranging from frats and sororities to various affinity groups (e.g. menstrual activism groups).

Building with a clear vision:

They are not building traditional consumer tech. Geneva’s founder believes that they are building a utility app. They are not using the traditional iterative lean startup method since they have a clear idea about what they want to build in the next five years. 

In our conversation, Geneva’s founder likened building Geneva to building a house. In the early days, they aimed to focus on the foundation, which meant designing the backend to scale out of the gate and building the front end to give their communities the foundational communication layers they would need to facilitate genuine conversations. They started with the synchronous chat feature, then built asynchronous chat, and finally expanded to other mediums like audio and video.

Who it is best for - This approach is commonly applied by founders that have a clear vision of what they are trying to build, and have a strong personal connection to the problem space. It is a risky approach since it can be costly in terms of time and capital if the vision is wrong, but it can be fruitful if the founder is experienced and has the resources and network to build a good product.  

Some other closing thoughts…

These four approaches are patterns I have seen from talking to other consumer founders and they might not generalize well to B2B or deep tech. Also, they are not mutually exclusive. For example, it is common for a founder to use Signal Aggregation before launching any product to gain a quick understanding of the market demand and then use the True Fans approach to assess whether or not there is a cohort of users that love the early product.

It is also important to note that idea validation is different from finding the product-market fit for this study. Several of the founders I interviewed mentioned that they were yet to achieve product-market fit despite reaching thousands of users. Idea validation empowers the founder to commit to the startup full-time, but the product-market fit is a measure that works beyond the viability of a product idea.