You can find all our articles for Free Members
The Subjectivity Of Risk (And What To Do About It)
This article has been contributed by Judi Heykoop, member at the House of PMO. Find out more about Judi at the end of the article.
Imagine you’re not at your desk but instead standing in the dawn light in Yosemite National Park, looking up at vertical rock. You’re at the base of ‘El Capitan’; a granite monolith with a precipice rising 1,100m above floor of Yosemite Valley. There are rock climbers up there but they appear as tiny dots moving along El Capitan’s seemingly sheer surface. This thing makes the giant sequoia trees that Yosemite is famous for look like twigs.
Staring up at El Capitan you ponder the risk of ascending it. It looks incredibly challenging. Now, how about doing so with no climbing gear and no safety rope – just your bare hands (what rock climbers refer to as ‘free soloing’)? You feel nauseous. One wrong move and you’re finished.
Sure as risk equals probability times impact, this feels like a like a red. But is it?
Let’s try and document this:
Risk: ‘There is a risk that if I attempt to free solo El Capitan I may fall up to 1,000 metres onto a glacial granite valley floor’.
Probability – I’m going with ‘almost certain’.
Impact – ‘Not pretty’. (Severe, unsurvivable).
Mitigating Actions: Just don’t! 1) Complete 10 years of rock climbing training, 2) Wait for optimal weather conditions, 3) Achieve minimal body fat, 4) Get some grippy climbing shoes and chalk, 5) Study this climbing route until you see it in your sleep, 6) Learn to like heights?
Is this a correct assessment? That would depend upon who you ask. Alex Honnold, the American rock climber who famously became the first person to ‘free solo’ El Capitan might rate this as Green. We could maybe agree that the impact remains severe, but we would likely encounter a range of views regarding probability. As someone who’s only ever climbed an easy route on an indoor climbing wall am I even qualified to try and size this risk?
Risk is a future chance of loss given exposure to a hazard – in this case a literal fall from a rock face. Risk estimates, or qualitative ratings of risk, are necessarily projections of future consequences. By definition, the true probability of the risk event and its consequences cannot be known in advance. We therefore rely on subjective judgments to fill-in information about an uncertain future.
The Potential Impacts of Risk Subjectivity
As PMOs it can be an ongoing challenge to try and get risks to be assessed rationally by the Project or Programme team. The perceived chance of something negative happening is all too often reported based on a person’s opinion, emotions, gut feeling or intuition. Realistically, it is impossible to remove subjectivity from the risk assessment process entirely. Recognising this and minimising it as far as possible is key.
The ramifications of erroneous risk assessments can be big. Some risks might be overestimated, leading to excessive precautions or missed opportunities, while others might be underestimated, leading to unexpected negative consequences. I might actually give free soloing El Capitan a go if the team are saying it is low-risk. Rock climbing analogy aside, this has financial implications; it can lead to misallocation of resources, excessive spending on unnecessary precautions, or insufficient investment in managing actual risks.
Subjective perceptions can also introduce biases in decision-making, leading to decisions that might not be aligned with the actual level of risk. In practice, this means poor choices or missed opportunities. When certain risks are downplayed due to subjective perceptions, it can leave organisations vulnerable to unforeseen events or threats. This can ultimately damage reputation if risks materialise, leading to a loss of trust and credibility.
Finally, overly risk-averse perspectives can lead to missed opportunities for growth or to wider delay.
Acknowledging and Addressing Subjectivity
We need to recognise where and when subjectivity enters and might create problems in risk assessment and risk-based decision-making. Assessors typically rely on their domain expertise, historical data, and past experiences to estimate likelihood. This introduces inherent subjectivity, as we know individuals perceive risks differently.
A seasoned rock climber may rate the probability of an incident occurring as ‘possible’ (3 on a scale of 5). In contrast, a less experienced climber may see the same scenario as ‘almost certain’ (5 on the scale). Cognitive biases such as availability bias (relying on recent experiences) and optimism bias (underestimating risk) further impact these evaluations. Put simply, wherever there is a reliance on judgment, subjectivity can emerge due to differences in perceptions, values, and experiences among stakeholders. This can be compounded by groupthink during risk assessment activities.
There will always be a reliance on judgement in the risk assessment process and there is no guarantee that it will be made by reasonable approximation. At times, it may, in fact, be a judgement made by an inexperienced novice (like me attempting to assess rock climbing risks). The need for judgement introduces subjectivity and bias and, logically, therefore, uncertainty and the likelihood of inaccuracy. So what can we do about it?
Give Me Data / Show Me The Science
As we know, qualitative risk analysis involves identifying threats, how likely they are to happen, and the potential impacts if they do. The results are typically shown using a Probability/Impact ranking matrix. This type of analysis will also categorise risks, either by source or effect.
Risk Of Not Surviving El Capitan Free Solo – Qualitative Analysis
Category: Resource Risk
Unlike quantitative risk analysis, which uses verifiable data to analyse the risk, qualitative risk analysis operates in a more generalised, “big-picture” space.
During a typical project, qualitative risk analysis will happen first. From there, risk managers can draw on data to address specific risks in more detail. So, while they do have two distinctions, they don’t compete for supremacy; they’re both parts of the larger risk management process. We have an initial (qualitative) assessment of free-soloing El Capitan. Now imagine a rock climbing forum had been collecting data on the number of successful free solo attempts of El Capitan over a period of time. That’s information I’d want!
We could probably all agree that a greater reliance upon data-driven assessments sounds sensible – follow the science! This is not an AI plug, but there’s huge potential for data analytics platforms like Power BI and Power Apps to transform risk assessment in many sectors. While accuracy is a concern, I put the El Capitan question to Chat GPT (an AI app). Interestingly while it could not confirm a precise survival rate of a free solo climb of El Capitan, it noted that the survival rate could be considered extremely low compared to roped climbing and informed me that several climbers have died attempting free solo climbs of El Capitan. I stand by my qualitative assessment!
Where it is an option, using historical data and analytics to support decision-making can help mitigate the impact of subjective opinions. Often, though, when we’re assessing delivery risks in practice we do not have available datasets that can be drawn upon to help measure any given risk that’s identified. Or perhaps the data exists within your organisation but the process of sourcing and verifying it would be disproportionate in terms of effort.
Reducing Subjectivity In Qualitative Risk Analysis
What, on a practical level can we do to reduce subjectivity in qualitative risk analysis?
- Seek Advice – For starters, the process should ideally involve several (preferably experienced) people. To state the obvious, the accuracy and detail of the analysis depends upon previous team experience. Where there is little relevant experience of the risk type in question, calling upon a subject matter expert is a good move. To help risk assess the probability of meeting your maker when free-soloing El Capitan, speaking to an experienced free-soloist sounds sensible. Speaking to multiple experienced free-soloists sounds even better. Or as a more relatable example, perhaps, when assessing a vendor delivery risk, speaking to areas of the business that have previously dealt with that vendor and can comment upon reliability over an extended period would help improve risk rating accuracy. Maybe they don’t have easily accessible KPIs on previous performance of the vendor, but details of previous delays and challenges can still be very useful information to have.
- Encourage Challenge – Within teams which are built up by members who have expertise in different areas, departments or functions, the risk assessment is focused on a broad range of knowledge and experience. Interestingly, bringing together both more experienced and less experienced individuals can also decrease subjectivity; more experienced members will bring in their knowledge but also biases, whereas less experienced members might be helpful in mitigating the latter by questioning the status quo. Then there’s the risk management culture. If you foster an environment where constructive debate and discussion about risks are encouraged, this can highlight diverse perspectives and potentially identify blind spots. You may not be able to influence this at an organisation level, but you can certainly encourage it within a project/programme.
- Speak the Same Risk Language – What else can you do to reduce subjectivity day-to-day? Making sure we’re all speaking the same language to begin with is pretty key. To stick with the extreme example of a rock climber, what does minor injury versus moderate injury actually mean when we agree upon an impact rating? What I consider minor you may consider moderate. Further defining the categories you’re asking the risk manager to use, preferably using some clear examples, can help produce consistent risk assessments. The same applies to assigning clear parameters to probability. Enter the standardised risk management framework! As an extension of this, clear risk tolerance guidelines help in setting a common understanding of what is an acceptable level of risk within the organisation.
- The A Word – Finally, regular review and audit of risk assessment processes ensures that the approach remains updated, relevant, and effective in addressing evolving risk scenarios.
By employing these strategies, PMOs can help minimise the impact of subjective risk perceptions, leading to more objective and comprehensive risk management practices.
Summary
Risk subjectivity encapsulates how people perceive and evaluate risks due to personal experiences, cultural influences, cognitive biases, and emotional factors. This subjective nature often leads to varied interpretations and assessments of risks. It reduces the accuracy of risk assessments, creates biases in decision-making and increases vulnerability. Recognising and addressing these subjective biases is crucial in developing comprehensive risk management strategies that balance different perspectives, helping to make more informed and objective decisions.
Alex Honnold has declined to comment on my risk assessment. I still think it’s a red.
Judi Heykoop is a PMO professional with over 15+ years of experience delivering large-scale regulatory change within financial services. With a consultancy background, she has developed programme management strategies and tailored governance frameworks for the delivery of £multi-million programmes. Most recently Judi has set up and run the PMO for the Bank of England’s ISO 20022 Programme; a complex five year programme delivering critical changes within payment operations.
Judi has a masters in corporate and commercial law from the London School of Economics and holds MSP, Agile and SCRUM Master certifications.
