Startup Idea Validation – How to Deal with Bad Data

Have you ever had a friend that always over exaggerates and makes fake promises? You know the one we’re talking about here; they tell you “I will definitely reach out for lunch when I’m in town” or “I would never be caught dead in that restaurant,” and then you see them posting pictures on their social media. Lo and behold, they’re out to eat in your city with another friend at that very place they said they would never go! So frustrating! You know the statements they give you are usually bad data, you cannot rely on it being true all of the time. You need to have the tools to recognize and avoid taking the bad data into consideration while validating your business idea as well.

According to Rob Fitzpatrick’s The Mom Test, there are three types of bad data- compliments, ideas, and fluff. Let’s take a few moments to identify what these are and how to get to the good data underneath!

Fluff

Now I know you’re probably picturing your favorite soft, fuzzy blanket or a big cloud, but that’s not the kind of fluff we’re here to advise you to avoid. Fluff is the bad data we observe coming from our flaky friend in the first paragraph. These are the general statements like “I usually,” “I always,” or “I never”. Rob also considers fluff to be future-tense promises like “I will” or “I would” and hypothetical maybes such as “I might” or “I could”. If you’re gauging interest from potential customers these are easy to take as facts when they’re being thrown around. It may even be an ego booster to take their “I would totally purchase” as fact, when in reality they are providing you with false hope. 

One thing you should note is that avoiding asking fluff-inducing questions can help you dodge getting so many soft and fuzzy answers like above. “Do you ever…”, “Would you ever…”, and “What do you usually…” are just a few examples Rob lists for us. Dancing around fluff-inducers doesn’t have to be a hard rule though, they can be a good way to start the conversation, allowing you to crack into the good answers you’re after with follow-up questions.

So, how do you battle the fluff if it comes up? Ask them about past experiences and how they handled it. Ask “What did you do the last time you had troubles with xyz”? Listen closely to not only the problem they had but more importantly, the way they handled it. This will give you more insight to take back to your idea. Questioning them to get deeper answers will help you gather more meaningful data and throw out any fluff that is, well, extra fluffy and not valuable or concrete. 

Compliments

Everybody loves compliments, right? We’re just as guilty as you of eating up all that attention when it’s given. While they might make you feel good in the moment, they are not always entirely truthful and can set unrealistic expectations. Most people in your circles will tell you that you have a neat idea, but do you know why? A good practice is to learn if they would use your product or service in their life – do they run into the issue you’re trying to solve often or at all? They may love your idea but not actually encounter the problem you are attempting to solve, therefore having no need for your product or service. They will most likely not become a paying user or customer, so you must continue your validation elsewhere with other people to make sure you’re hitting the mark. If they do encounter said issue, get them to tell you more! What solutions do they currently use or have used in the past? Use their previous methodologies as a gauge for the effectiveness and even demand for your product or service. 

Rob shares that a great way to avoid getting a collection of compliments is to not mention your idea outright in your conversation. Keep your startup idea tucked away in your back pocket while you’re feeling the water and seeking validation. “You don’t need to end up with what you wanted to hear in order to have a good conversation. You just need to get to the truth” Rob notes. If those compliments do slip into the conversation, let it slip one ear and out the other. Focus on asking more questions to get the feedback you’re seeking!

Ideas

Let’s talk about the final type of bad data, ideas. This can be observed when you’re sharing your idea and your coworker chimes in “Oh wow, that’s so cool. What if it did ____ too?” with a brilliant addition. It’s easy to take every suggestion and jot it down; it doesn’t sound like it would be too hard to include right? After taking suggestion upon suggestion you will soon find that your product has so many extra features, some of which your target audience won’t even touch after you bent over backwards to get them included. We’re not trying to say you should avoid listening to feature suggestions, but be a little wary. You don’t want to waste valuable time and money that could be spent on other improvements your users actually need. 

We need to take a minute and find out if these “great ideas” are valuable to spend time on. Question the person suggesting add-ons a little deeper. Why would they value the add-on? Would it solve an issue they have often? Do they know other people with this problem? Be sure to check their motivations and take time to evaluate with other potential customers. It might be just the thing your product was missing (or a great addition for later down the road)!

Validation can be a long process, but getting to know the needs and wants of your audience is so crucial to success. Just beneath the bad data, the compliments, ideas, and fluff, is the honest, valuable data underneath. Now get out there and get your idea validated!