Posts Tagged ‘Validated Learning’

The most common Agile framework is Scrum, which typically involves a product manager managing a backlog of user stories (outputs) using the user story framework:

Template

As a… <persona>

I want… <intent>

So that… <benefit>

Example

As a player

I want to be able to easily access new games

So that I can have fun playing the latest games that I haven’t played before


Since user stories are output-focused rather than outcome-focused, it becomes easy to fall into the build trap of delivering output after output with no understanding of whether it delivered any value to the customer or business. One of the reasons is that unless tracking is part of the DoD, to track the value would often require additional tracking user stories in the product backlog which are easily ignored when in a project led environment or when there is pressure to get after delivering a new unrelated user story (output).

Now, In the 1920s, Ronald Fisher developed the theory behind the p value and Jerzy Neyman and Egon Pearson developed the theory of hypothesis testing, which as a result formed an Agile/Lean technique called Hypothesis-Driven Product Development which is outcome-focused, delivers a measurable conclusion and enables continued learning. A hypothesis framework consists of:

Template

We believe… <capability>

Will result in… <outcome>

As measured by… <KPI>

Example 1

We believe that by providing players with an easy way to access new games

Will result in an increase in game plays

As measured by a higher number of game plays per player and new game engagement

Example 2

We believe that by offering new players a 5 day achievement-based promotion

Will result in new players retaining longer

As measured by a decrease in churn rate by 5%


The immediate benefit of using a hypothesis-driven framework especially for uncertain product iterations is that the product team are forced to ensure that the outcome is measurable before delivering the output/feature. Since it will be measurable, it will be possible to learn and validate the hypothesis aka validated learning.

The ideal scenario would be to run multiple experiments concurrently to reach the same outcome so that you can learn quicker (rapid experimentation). A test failing means progress, that you’ve learned what doesn’t work, so you can progress in a positive way to experiment on a different idea to solve the problem.

It’s easy to migrate from the Scrum to Kanban framework or vice versa, but migrating to the hypothesis-driven framework is significantly more challenging as it involves a culture of empowerment and learning, with trust and patience being critical elements to start with whilst the team gets used to the new framework, data structure and the validation capabilities, which is needed before the product team can conduct rapid experiments effectively.

If you are entering a mature market and you are more certain that the solutions will solve the problem, a standard user story is more appropriate, but the most efficient way of delivering outcomes where you are uncertain that the solution will solve the problem is hypothesis-driven product development, rather than spending months guessing with user stories without any learning.

With tools available to easily conduct remote customer interviews (UserZoom, Lookback.io), A/B testing (Firebase, Maxymiser) and prototyping (Sketch, Figma), it makes it easier more than ever for empowered product teams to efficiently conduct experiments to validate that the solution will solve the problem.

Good luck in your experimentation journey!

Mmp

MVP (Minimum Viable Product) – the minimum amount of features needed to validate the business hypothesis with target customers.

MMP (Minimum Marketable Product) – the minimum amount of additional features on top of MVP which will allow marketing to start growing the product.


Validated learning is one of the five principles of the Lean Startup method and the MVP technique aims to test the business hypothesis. MVP was introduced in 2001 by Frank Robinson but popularised by Eric Ries through his book The Lean Startup.

Since startups tend to work under conditions of extreme uncertainty with limited resources, validating the hypothesis with target customers early in an efficient way using a prototype, wireframes, surveys or marketing material becomes even more important if it is to avoid the scenario of spending months or years building a product which customers don’t need or want (does not have product/market fit).

Achieving product/market fit would involve multiple iterations on the MVP based on target customer feedback, but once the product/market fit is validated it’s time to build the product for real and head towards an MMP by adding features to enable marketing to start growing and scaling the product, with the first set of additional features usually focusing on MarTech and improving the core customer experience.

Now, with an Agile mindset of iterating frequently based on value, it makes the MVP technique similar to Agile product development – building a product that customers need, want and loves by frequently talking to customers/target customers and we only need to look at three of the twelve Agile principles (also introduced in 2001) to see this:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Simplicity–the art of maximizing the amount of work not done–is essential.

Welcome changing requirements. Agile processes harness change for the customer’s competitive advantage.

Because of this, I prefer to use a broader definition of MVP: the minimum amount of effort needed to learn. This is because you can apply the MVP technique (Agile mindset) to a multitude of scenarios outside of launching a new product at a startup where you get the same benefits of reducing wasted money/effort/time by learning sooner rather than later, whether it’s a:

  • New Product – validate before building the whole product
  • New Feature – experiment rapidly before building and rolling out the full feature
  • New Process – being inclusive/gaining feedback before a full roll-out
  • Retrospective – ensure teams are empowered to make changes before having retros
  • Complex Solutions – start discussions early with assumptions before waiting for concrete answers
  • New House – view before purchasing
  • New Job – research/interview before accepting a job
  • New relationship – dating before marriage
  • New Car – test drive before purchase

Start small and fail fast!

Adding a new feature to an existing product is the most common scenario where you can use an MVP approach, but also where it’s most common for businesses to spend months building a new feature that turns out to be low value to customers. Similar to a new product, it’s important to validate new features where the projected value is uncertain by building a lightweight prototype/wireframes to validate with target customers when conducting interviews.

Saying this, if you have qualitative/quantitive data which gives high confidence that solving the problem will be valuable and time to market is important, then it’s equally effective to just develop and go live with the basics you need for the new feature to function at the right quality, then iterate in an Agile way.

When there is uncertainty, break it down, start small, test and learn quickly and it’s surprising how much easier and efficient the problem is to solve.

With tools to easily conduct remote customer interviews (UserZoom, Lookback.io), A/B testing (Firebase, Maxymiser) and prototyping (Sketch, Figma), it makes it easier more than ever for empowered product teams to efficiently conduct experiments to validate that the solution will solve the problem.