TDi – Test Driven Ideas

Test Driven Ideas - Lean Startup


In recent times I have been heavily involved with product teams at the idea phase. An idea phase is not just a new product, but a new feature, service or desired outcome aligned against a presumed customer need. Usually and particularly within enterprise environments, the idea phase has been surrounded with teams ready to build something and these teams get frustrated when they feel thinking or planning slows them down. I get this, having been a developer for many years, it sometimes feels frustrating when you’re not building something. However repeating a well overused, but still one of my favourite quotes :

“Remember, if we’re building something that nobody wants, it doesn’t matter if we’re doing it on time and on budget.” – Eric Ries

Recognising waste in the system, I want to introduce a core behaviour to ideas that’s been in software implementation for many years. TEST FIRST! Applying this loosely to a system thinkings level, the Deming PDSA (Plan-Do-Study-Act) approach can be really effective to eliminate waste and reduce costs when applied to ideas. This fits with Deming and many others who emphasise  putting quality checks at the front of the system. Ideas should have quality checks before any substantial implementation is underway.

So how does this help me?

Well Lean Startup has coined the phrase  BUILD-MEASURE-LEARN as an approach to testing or validating your idea. In my opinion this is a loose derivative from ideas such as PDSA which came long before it and I’m sure can be traced back for a very long time in our history. A key point I would like to raise is that the term BUILD-MEASURE-LEARN is contradictionary to the explanation and should not be taken literally. The way to approach this should be :

  1. Learn – Define what you need to learn before building anything
  2. Build – Build just enough in order to learn
  3. Measure – Measure the outcome against your success criteria

Although Lean Startup clearly states identify your hypotheses and success criteria before building anything to test your riskiest assumptions, I’ve seen many teams skip this bit and jump into solution thinking, before fully understanding what and why they are building something. Sometimes when challenged, I’m greeted with common responses, “but we know what we are doing”, “We know what our customers need”, “Well we have to build something to learn what needs are”, “I’m copying competitors who obviously have customers so there is no risk” and so on. None of these statements follow a test first approach which ironically is often implemented using test first engineering behaviours such as TDD (Test Driven Development) and BDD (Behaviour Driven Development).

TDi in Practice

So Test Driven Ideas (TDi) is just a simple reminder to re-enforce the PDSA or BUILD-MEASURE-LEARN process in practice for ideas at a (system) level. It repositions the loop to decide what to learn first, then create tests around that before any implementation takes place. You shouldn’t build anything unless you, define the test first. To define the test you need to understand what you need to learn and what success looks like for that test to pass.

An idea can exist a long time before a user story in a product backlog and the team should be free to work at any part of the system to provide value, including testing ideas. Running minimal viable experiments to test ideas provide value by testing assumptions.

To put this in practice (Build-Measure-Learn and Plan-Do-Study-Act) :

  1. Decide what you need to learn (Outcomes, Needs, Problems,Risky Assumptions, Set Hypotheses).
  2. Decide how to measure what you need to learn and define what success looks like to confirm or invalidate your idea or belief.
  3. Then build the absolute minimum which will enable you to learn and test your idea (Test).
  4. Analyse your results and learn.
  5. Act (Stop, Pivot, Continue)

This supports PDSA as point 2 is the Plan, point 3 is Do, the outcome of point 4 is Study and point 5 is Act. There are very effective ways to construct tests or experiments which are popular amongst the lean startup community such as producing an Experiment Map.

So TDi is  re-enforcing the behaviour of BUILD-MEASURE-LEARN from lean startup and PDSA at an idea level to maximise learning and de-risk ideas before significant implementation. Implementation of course can be the test as stated in point 3.

For those from a software background, you’ll understand that the test first practice allows you to know when you have done just enough work or effort to pass the test and when the test has past you have met the executed outcomes. Many ideas die because they are not implemented with a clear definition of what success looks like against the customer need. Many  ideas are implemented without a clear understanding of how it should be implemented to have the desired affect. Setting the criteria on how to test an idea, for the idea to be viable to explore further can really help reduce waste by assumption and waste through the lack of the ability to focus minimum efforts against what needs to be learnt.

Please feel free to leave feedback.


Leave a Reply

Your email address will not be published. Required fields are marked *