Search the PMO Conference Library

Watch back all our PMO Conference sessions

PMO Library | Free Articles | Inside PMO | PMO Book Shelf

PMO Conference 2022 \\ Rapid PMO Experiments to Turbo-Charge Delivery – John McIntyre

Business demands always outstrip supply. How do you decide which projects and features to focus on?
Your product managers may talk about building small ‘MVPs’ but what does that really mean?
In this interactive session, John McIntyre from HotPMO introduces the concept of Rapid Delivery Experiments which will change the way your portfolio approves projects, whilst massively reducing uncertainty and risk.

Session Recording


Session Presentation

>>> Presentation Notes

Session Notes:

It doesn’t matter if the results don’t align with preconceived expectations – finding out is the whole purpose

John’s subtitle for his talk was “reducing energy to get stuff done”.  The focus was on methods to determine what is the most resource-efficient activity for developing your PMO performance.

The primary idea is to rigorously identify and test any assumptions (and the hypotheses behind them), that might be the basis of a proposal or plan of action.  Moreover, to start by working on the assumptions that potentially hold the greatest risks.  You will need to create an assumptions log with some level of risk assigned to each assumption, and then define how each can be validated (or not).

Two key abbreviations emerged from this presentation – RAT (Riskiest Assumption Test) and MVP (Minimum Viable Testing).  The latter is the key to getting rapid results and sound decisions.  MVT essentially represents the minimum amount of effort needed to learn about the feasibility of an idea. The process is intended to be “quick and dirty” as opposed to detailed and formal.  However, it is important that the experiment:

  • is aligned to the business plan or strategy
  • is low cost
  • provides reasonably strong evidence (for or against)
  • is quick and easy to set up
  • is relatively short in duration
  • results in a decision

It doesn’t matter if the results don’t align with preconceived expectations – finding out is the whole purpose.  If the result is inconclusive or too vague, then modify the experiment to generate more, or more refined, information.

There is no standard structure for these experiments – can be as simple as a discussion between the affected parties, or can involve surveys or the generation of sample products to get feedback from potential users. However, the experiment’s design should take into consideration the relevant variables or metrics associated with the hypotheses, and the population you are targeting.

John provided a range of hints and tips for utilising this technique, which many will find really valuable in the volatile circumstances of today.

Many thanks to PMO reporter Catherine Askew for this article.

A good place to begin your own experimentation is to start with your current project prioritisation process

A thought-provoking session from John presenting an alternative, speedier approach to determining which projects to focus on when faced with competing business demands.  A quick guide to how to devise an experiment, with tips on what to look for in a worthwhile experiment and a note of the things that can potentially go wrong.

Moving away from the MVP (Minimum Viable Product) and embracing RATs (Riskiest Assumption Tests), by identifying your Riskiest Assumption and Testing it through experimentation.

John’s description of how business experiments provide the means to reduce risk quickly, by removing uncertainty, determining if your general direction is the right one, using data to test if your idea will work and enabling you to avoid wasting time and energy on ideas that won’t work.

By experimenting assumptions can be tested in an Agile, cost effective and evidence-based manner.

In designing the experiment, you need to identify your assumptions, write the hypothesis, select the experiment, define the variables and metrics of your hypothesis and define the population that you are targeting.  After running the experiment, you need to analyse your results, drawing conclusions to make data driven decisions to determine your next steps.

A business hypothesis is an assumption which your future operating model, or project relies on, and defines what you need to learn about to understand if your vision might succeed.  A good hypothesis should be:

  • Testable – can you validate it as being true or false?
  • Precise – have you clearly defined what success looks like?
  • Discrete – does it have one thing you want to test?

John’s recommendations for what makes a good experiment are to be clear on what you are testing, keep the costs low, be clear when you need your results so ‘go cheap and fast’.  Design experiments that produce the strongest evidence, given your constraints and reduce uncertainty as much as you can before you build anything (Discovery then Validation).
What goes wrong?

  1. Not committing enough time
  2. Gold plating (look for cheap and dirty)
  3. Overthinking
  4. Incomparable or weak data
  5. Not making decisions (this exercise is not about learning but about decisions).

A good place to begin your own experimentation is to start with your current project prioritisation process and try it against completed projects from previous years.  Would your hypothesis be proven using the data?

John finished the session by challenging his audience to go back to their desks make that ‘first note’ and begin experimenting.

Many thanks to PMO Reporter Graham Gunn for this article.

Was this article helpful?
0 out of 5 stars
5 Stars 0%
4 Stars 0%
3 Stars 0%
2 Stars 0%
1 Stars 0%
Please Share Your Feedback
How Can We Improve This Article?
Table of Contents