Hypothesis-Driven Product Development

Posted: May 15, 2021 in Data, Guides, Product Management
Tags: , , , , , , ,

The most common Agile framework is Scrum, which typically involves a product manager managing a backlog of user stories (outputs) using the user story framework:


As a… <persona>

I want… <intent>

So that… <benefit>


As a player

I want to be able to easily access new games

So that I can have fun playing the latest games that I haven’t played before

Since user stories are output-focused rather than outcome-focused, it becomes easy to fall into the build trap of delivering output after output with no understanding of whether it delivered any value to the customer or business. One of the reasons is that unless tracking is part of the DoD, to track the value would often require additional tracking user stories in the product backlog which are easily ignored when in a project led environment or when there is pressure to get after delivering a new unrelated user story (output).

Now, In the 1920s, Ronald Fisher developed the theory behind the p value and Jerzy Neyman and Egon Pearson developed the theory of hypothesis testing, which as a result formed an Agile/Lean technique called Hypothesis-Driven Product Development which is outcome-focused, delivers a measurable conclusion and enables continued learning. A hypothesis framework consists of:


We believe… <capability>

Will result in… <outcome>

As measured by… <KPI>

Example 1

We believe that by providing players with an easy way to access new games

Will result in an increase in game plays

As measured by a higher number of game plays per player and new game engagement

Example 2

We believe that by offering new players a 5 day achievement-based promotion

Will result in new players retaining longer

As measured by a decrease in churn rate by 5%

The immediate benefit of using a hypothesis-driven framework especially for uncertain product iterations is that the product team are forced to ensure that the outcome is measurable before delivering the output/feature. Since it will be measurable, it will be possible to learn and validate the hypothesis aka validated learning.

The ideal scenario would be to run multiple experiments concurrently to reach the same outcome so that you can learn quicker (rapid experimentation). A test failing means progress, that you’ve learned what doesn’t work, so you can progress in a positive way to experiment on a different idea to solve the problem.

It’s easy to migrate from the Scrum to Kanban framework or vice versa, but migrating to the hypothesis-driven framework is significantly more challenging as it involves a culture of empowerment and learning, with trust and patience being critical elements to start with whilst the team gets used to the new framework, data structure and the validation capabilities, which is needed before the product team can conduct rapid experiments effectively.

If you are entering a mature market and you are more certain that the solutions will solve the problem, a standard user story is more appropriate, but the most efficient way of delivering outcomes where you are uncertain that the solution will solve the problem is hypothesis-driven product development, rather than spending months guessing with user stories without any learning.

With tools available to easily conduct remote customer interviews (UserZoom, Lookback.io), A/B testing (Firebase, Maxymiser) and prototyping (Sketch, Figma), it makes it easier more than ever for empowered product teams to efficiently conduct experiments to validate that the solution will solve the problem.

Good luck in your experimentation journey!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s