craigstrong

Beta Release

A Beta phase is an early attempt to test that you’re solving the basic customer needs you have identified, your solution works as intended and with this you can scale.

When you first decide to release a Beta program, it’s important to recognise you still have a lot of learning to do. A Beta release is not an excuse to get away with a sub-standard products or a shortcut on quality. This is your opportunity to learn and maximise value for the customer and your business whilst de-risking the potential effects of discovering damaging issues after launch. Beta programs can de-risks the chance of failure, effort, brand damage or worst case product retraction.

Running a Beta program usually involves a cross-functional, cross-departmental project where you will identify and coordinate the necessary activities, dependencies and learntogether as you capture feedback.

What kind of things will you be expected to do in Beta ?

  1. Releasing regular updates and version changes of your product using feedback and learnings  obtained from the customers using your Beta product.
  2. Establishing close relationships, to support the customer being able to use the product and to maximise learning on the products effectiveness.
  3. Measuring and testing your hypotheses against customer cohorts using experiments.
  4. Setting up specialised release programs and marketing campaigns for early adopters to participate. Some examples :
  5. Setting up clear and transparent data policies and feedback channels with the customers consent. (Attention to local data protection laws required)
  6. Generating some basic on boarding and support materials.
  7. Regularly and sharing learnings with your teams.

The Beta Learning Process

You might know a great deal, but your on a journey to learn as much as possible as fast as possible and discover the unknowns. Remember it’s impossible to be aware of the Unknown,Unknowns you are yet to encounter until you have encountered them. As these emerge, you will need to understand and solve them. Exploring the many possibilities to solve a problem is a beneficial way to maximise growth, customer value and release potential in your business model whilst also learning how to change the solution as needed to be more effective.

Hopefully with the early Beta launch of your product, you are taking small, rapid learning steps with an observational state, supported by controlled time based experiments. A tried an tested practice to support learning is the Lean based Plan-Do-Check-Act  (PDCA) introduce by William Edwards Deming, long before the now popularised Lean Startup Build-Measure-Learn methodology. Maybe somewhat pedantic, I tend to favour defining the PDCA process as it explicitly calls out the ‘Plan’ phase first. Although  Build-Measure-Learn advocates this through Hypotheses, it helps remove an assumption to just jump in and learn afterwards; a behaviour observed far too often. To maximise learning in Beta, the scientific method should always start with a structured hypotheses first with clear success criteria.

Make sure you explicitly set the criteria before you run the experiment, and make a record of it. It’s too easy to fudge the numbers later and rob yourself of any valuable insight. –

https://www.teaguehopkins.com/2014/03/set-minimum-success-criteria/

5 Things to look out for when launching a Beta program :

  1. You need to grow and learn as fast as possible, but at a pace that your business operations and culture can sustain. Consider soft launching and managing customer cohorts to maximise learning as you apply changes. Ideally you would structure a specific Beta program such as those demonstrated in the example above.
  2. Early adopter expectations of quality in terms of your product can be managed better if Beta is stated on your product. Generally this sends a signal to expect some minor bugs and issues, which can result in a little more tolerance from customers. Customers being explicitly aware of this are likely to provide bit of slack as you explore the unknown goes a long way and typically will be more engaged with providing feedback.
  3. The effort to maintain and transfer knowledge across your organisation will increase as your team sizes grow and new skills and resources will emerge. Make all information as visible and transparent as possible. Be mindful to observe the impact on services as you engage customers. This will inform your business model.
  4. It takes time to diagnose issues and learn. I’d highly recommend you have good enough diagnosis tools in place. Don’t over do it in the early days though as it will increase cost, time and effort to manage tools.  There are plenty of great tools out there to kick off such as Kibana or Crashanalytics.
  5. Consider keeping a low key initially and avoid the big bang marketing campaigns. Introduce this when your confidence grows.

If you don’t have the might of Apple and Google behind you to build your own Beta platform, there are a number of great tools available that can help you structured and manage a Beta program. These tools can also be useful for giving you more structure as well should you be new to the Beta process. Here are a few examples :

5 Beta Test Platforms & Tools

  1. ErliBird
  2. Centercode
  3. Testflight (Mobile)
  4. BetaList (Startup Feedback Community)
  5. BetaFamily

Please use the comments below if you would like to add useful links, experiences or insights to share with readers.


Recommended Reading – Beta Releases :

  1. The 5 Types of Beta Testing Programs and Why 4 of Them Suck
  2. How To Conduct A Closed Beta Testing with Zero Budget
  3. Gov.uk Service Manual – How the beta phase works
  4. Which Beta users platform would you recommend ?
  5. Tips for a Successful Alpha Release:Focus on the Entire solution not just the software

Leave a Reply

Your email address will not be published. Required fields are marked *