Watch back all our PMO Conference sessions
Risk Assessment – Beyond the Heat Map
This session opened PMOLearn! in Edinburgh in November 2023 – the subtitle was Quantitative Methods to Inform Proposals and Delivery and was taught by John Greenwood, a leading expert in risk management.
Risk assessment has continued to rely on heat maps and subjective matrices, despite several books and papers published in the last 20 years that demonstrate their shortcomings.
With the availability of cheap, high-performance computers, and intuitive modelling software, risk analysts and managers have the tools at their disposal to break away from 3-, 4-, or 5- point scales for probability and impact, and provide evidence-based assessments of the effect of risks on objectives.
This session provides an overview of assessing and modelling schedule and cost impacts of event risks and estimating uncertainties; and will show how this is used to inform decisions and commitments. It will also discuss how risk severity is dependent on the objectives being considered.
Takeaways – What You’ll Learn
- An appreciation of quantitative analysis – that it has aspects that are objective, but also has some subjective judgment.
- An approach to risk-based costing and scheduling and a look at some of the tools available.
- Ideas for using the outputs to inform business decisions.
Recorded Lesson
Lessons Deck
Download the Lesson DeckLesson Overview
John introduces the importance of challenging single-value estimates in project management. They delve into the inherent uncertainties in project planning, forecasting, and the likelihood of projects deviating from defined end dates and costs.
This session is all about using risk management and quantitative analysis to assess confidence levels in project estimates. John touches upon the three main objectives of risk management: changing the future, forecasting, and knowledge capture.
For the PMO, there is a detailed breakdown of risk management tasks, including proactively resolving uncertainty, stabilizing assumptions, and addressing ambiguities. The PMO should recognise the importance of additional work and focused efforts during project delivery to manage risks effectively.
Planning and Estimates
John addresses the importance of accurate project planning, particularly in terms of schedules, costs, and risk management. The focus is on improving estimates and reducing variances during project delivery. He emphasises the interconnection of project management elements such as schedules, cost models, and risk registers.
In terms of the quality of schedules, John referenced standards like the GA (Government Accountability Office) standard and the DCMA (Defense Contract Management Agency) standard.
A simple example involving the construction of foundations highlights the impact of unforeseen risks, such as unstable ground, on project costs and timelines. The importance of early risk assessment and mitigation strategies is stressed.
The First Exercise
John introduces the first exercise related to a project involving buying and installing equipment, emphasising the identification of risks, assumptions, and uncertainties in the project plan. The audience is encouraged to collaborate and generate a list of potential risks for discussion.
The discussion revolves around the various factors and uncertainties that can impact a project, particularly focusing on a project involving site preparation and equipment installation. The following key points were discussed:
- Specialist Resources and Equipment Types: The need for specialist resources and considerations related to equipment types and site characteristics were highlighted.
- Staffing and Leadership: The importance of staffing, leadership, and factors like scope creep and external influences that may derail the project were discussed.
- Budget Considerations: Various budget-related concerns, including site preparation, installation, and potential budget overruns, were addressed.
- Risk Management and Testing/Acceptance: John delved into risk management aspects, emphasising the importance of thorough testing and acceptance processes to avoid project ambiguities and potential rework.
- Ambiguities and Changes: The discussion covered areas where ambiguities can arise, leading to possible project delivery issues. Topics included requirement changes, design changes, and unforeseen alterations in site preparation.
- Estimating and Uncertainty: John explored the concept of estimating uncertainty, emphasizing that project planning involves numerous guesses and variables. The discussion included factors affecting the duration and fixed costs, such as labour, materials, and equipment.
- Pandemics and External Factors: External factors like pandemics, weather conditions, staff availability, and equipment delivery delays were discussed as potential sources of uncertainty impacting project timelines and costs.
- Impact of Late Equipment Delivery: The conversation touched upon how the timing and awareness of late equipment deliveries can significantly affect project costs and timelines, depending on whether the installation work is postponed or continued with standby teams.
John also touched on issues related to equipment procurement, internal authorisation, delivery lead times, quality inspection, and the importance of lessons learned in project management. He also gave a brief overview of Monte Carlo techniques for risk analysis in project scheduling.
Forecasting the Future
In project management, statistical analysis, specifically an S-curve, can be used to understand the uncertainty and risks associated with project completion dates. John explains that the S-curve is generated by running simulations of project completion dates based on historical data and uncertainties.
The analysis involves creating a histogram and an S-curve, where the histogram represents the frequency of project completion in different time periods, and the S-curve represents the cumulative distribution of project completion dates. John thinks these tools are particularly useful when entering into contracts with customers, as they provide insights into the likelihood of meeting deadlines.
Additionally, John introduces the concepts of P20, P50, and P80, representing the 20%, 50%, and 80% probability of completing the project by a certain date. These probabilities help in understanding the level of certainty associated with the project completion dates.
John also touches on the importance of sensitivity analysis, which involves assessing the impact of individual risks on the overall project completion date. This analysis helps in identifying critical risks that need proactive management.
Q&A
John discusses the importance of informed decision-making in project management and emphasises the need for effective risk management. He addressed the cost and benefits of risk management, highlighting the fine line between preventing and rectifying issues. Finding a balance and determining the “sweet spot” to avoid excessive costs while ensuring effective risk mitigation.
The Q&A session covers topics such as stakeholder management, quantifying risks, and the prerequisites for quantitative risk analysis.
In response to a question about managing stakeholder expectations, John suggests starting with simple scenarios, like estimating commute times, to illustrate the concepts of uncertainty and risk. He uses this example to explain making informed decisions based on risk analysis. He also mentions the importance of engaging stakeholders and warming them up to accept and understand risk-related information.
Another question is about the cost of quantitative risk analysis compared to traditional methods like the five-by-five matrix. John recommends a book by Doug Hubbard, “The Failure of Risk Management,” which argues against the limitations of the five-by-five matrix. He discussed the prerequisites for quantitative risk analysis, emphasising the importance of numbers over opinions. Assuming good project management practices are in place, the additional overhead for quantitative analysis mainly involves assessing the likelihood of risks and estimating costs for fallback plans. He also mentioned the cost of software but highlighted that it’s a one-time expense.