The Trap of Risk Assessment Tools

Humans understand and respond to narrative innately. Impactful events are explained as stories, successful calls to action are delivered as a descriptions of a desired or undesired future, and people make decisions every day through explanations.

We do this because narratives are a natural form for communicating information and insight. Narratives are therefor also a natural form for risk communication. In regards to risk, narratives help listeners visualize and develop internalized models of risk, which in turn represent the truth in a way that numbers alone won't (and can't) for most individuals and most situations.

Much of the current cybersecurity and enterprise risk management world is premised on the idea (or ideal) that risks can be meaningfully summarized in ordinal form, such as “very high likelihood”, “moderate impact.” These english ordinals are then often converted to numbers. Sometimes math is performed, and a magic risk number is derived. Some models even use dollars or dollar ranges in their outputs. However, focusing on getting to the “right number” - and reporting that is focused on numbers - denies the reader the benefit of context. It makes it hard for decision makers to object to the assumptions embodied in the inputs and the calculations. Numbers or dollars alone convey objectiveness and authority, even while that is not necessarily the case. The work of "getting to the numbers" behind closed doors only serves to exacerbate this issue. In the best case systematizes the subtle biases of the assessors, and in the worst case distorts the model to fit preconceived notions about organizational priorities, or conform to personal risk worldviews. Caution should be advised for those using numbers alone to represent risk.

In contrast, risk assessment communication that focuses on narrative and which embraces dialog allows for discoveries and insights by the decision makers, provides an opportunity to question assumptions, and enables sharing and alignment of perspectives. Risk decisions, everyday and tough risk decisions alike, are best borne of discussion, and will be for the foreseeable future.

If you are responsible for risk assessment at your organization, don’t fall pray to a tool’s promise, or the unquestioned illusion of objectivity and certainty that comes from numbers. If you are performing risk assessment where the focus is populating fields in a spreadsheet or application, no matter how advanced it is, you run the risk that the one thing that everyone needs to consider is getting lost through reductionism. Do the numbers if required, but wrap it in an informative discussion, and design and facilitate the discussion to convey risks in business terms that supports business decision making.