This is the fourth post in a series of posts answering questions we’ve received as part of our Hands-On With Lean workshops.
You can find the first three posts here:
1) How Do I Turn Ideas Into a Viable Product?
In this post, we’ll talk about how to determine whether to pursue or drop your product idea.
Question: “What do I measure or track to determine if I should pursue or drop my product idea?”
The specific metric/ set of metrics you should track when determining if an idea is worth pursuing may be dependent on your specific product, industry, business model, and/or company phase.
Generally speaking, however, you should be tracking a combination of qualitative and quantitative research and metrics.
Remember, the points I’m making below address the question of metrics when trying to determine whether to continue pursuing an idea.
Once you’ve made that determination, you will have a whole new set of metrics you should be focused on and tracking.
Metrics to Track When Determining Whether to Pursue a Product Idea
When you are starting with just a problem assumption and an idea, a great place for qualitative data is user interviews.
Identify who you think your ideal user or customer is, then go out and interview them.
BUT: What do I ask when I interview potential users and customers?
A couple of good resources on conducting user interviews are this blog post on Ash Maurya’s website titled “How To Interview Your Users And Get Useful Feedback”, and this recorded lecture “How to Run a User Interview” from Stanford’s recent online class titled “How To Start a Startup”, which was given by Sam Altman, President of Y Combinator.
Some interviewing guidelines:
- Try to get your interviewees to describe the problem you’re assuming they are having without “leading the witness”.
- Get your interviewees to describe how they are currently fulfilling this need you assume they have? I.e.: What are their current alternatives?
- Understand what your interviewees’ current workflow is when they are implementing a solution to the problem you are attempting to solve for them. Is that workflow a source of pain for them? Are there specific parts of the process that are pain points?
Although qualitative metrics are important to track for the lifetime of your product or company, there is a point when your focus will move from qualitative to quantitative metrics.
When you’re trying to determine whether to pursue an idea, you should identify metrics that measure how effectively you are:
1) Driving traffic to your site, or how many people you are able to make aware of your product/ service (acquisition), then
2) How effectively you can convert that traffic from interested observers to invested stakeholders (activation)
Depending on your business model, “converting” someone from an observer (visitor to your site, landing page, blog, etc.) may take the form of getting them to sign up to your email list, commit a certain amount of money in the form of a pre-order of your product, sign up to learn more, or create an account.
Tip: See if you can combine acquisition with activation (the % of visitors to your site who end up signing up, subscribing, etc.).
Can you ask users to sign up to learn more, or get more information?
Can you offer them a special discount if they sign up upon their first visit to your site?
This can potentially be a strong indicator shows that you are solving enough of a pain that they’re willing to sign up right then and there.
BUT: How do I set goals or target values for my quantitative metrics?
There are a number of ways you could potentially set your targets, but more importantly, set them.
Here’s why you should never track a metric that you consider critical to growing your business without setting a target first:
If you don’t have a target for each of your metrics, you will never be able to determine whether 1 million page views on your site is good, average, or bad news.
Another example: Without targets, you won’t be able to decide whether 10% conversion rate means you are killing it or getting killed.
How to set targets:
1) Use historical performance: Know how this metric has performed in the past and use that past performance to set your current target.
2) Industry standards: There are vendors that provide competitive intelligence reports that can provide benchmarks for a certain industry.
(Search for “competitive intelligence” to find a list of tools and resources.)
Note that competitive intelligence reports are usually only meaningful to you if you have significant traffic, and may not be meaningful when you’re first starting out.
3) Measure your current performance, and set your target to be current + 10%: This is probably my favorite, because really, once you experiment with trying to increase performance of a metric by 10%, you’ll be able to tell whether that is a feasible goal or not, do some analysis on why, and more intelligently decide on what your next target should be.
In summary, to determine whether or not to pursue a product idea, start with metrics.
Learn what you can through qualitative research first.
If you are getting strong signals that there is a real problem to be solved, and that your proposed solution would be compelling to potential users, move on to qualitative research.
Identify key metrics that will tell you whether you’re able to convert people from interested observers to invested stakeholders.
Set goals, see if you can meet those goals, and keep learning at every step along the way.
If you’re meeting your goals and targets, and are able to get potential users to have some skin in the game (Giving you their email address? Setting aside time on their calendar to talk to you? Giving you $$ in exchange for the promise of value?), then you’re on the right track.
Ultimately, the decision to pursue your product idea comes down to a lot more than the above.
This includes (but is not limited to):
- Whether pursuing your idea is the right decision for your current life circumstances
- Whether you have the ability to support yourself and/or your team financially for long enough to get to the next phase in your product or company’s lifecycle
- Whether you love this idea enough that you are willing to wake up every day and work really hard on making it happen.
Only you know the answers to these questions, so after you’ve gone through all the metrics and analysis, make sure to check your gut. It’s probably smarter than you think.
If this post was in any way helpful, please share it with others!
Also, make sure to sign up (if you haven't already) for announcements, the 27Sprints newsletter (which includes exclusive content, insight, and the occasional insider joke or bad pun for subscribers), and updates on upcoming events, discount codes, etc.
Thanks for reading!