Smarkets Blog

Measuring risk is inherent in every decision we make. Usually we don’t even notice that we’re making a risk-assessment when, say, stepping into a lift or crossing the road, but we do so on a second-to-second basis all the time. In spite of the near unconscious level at which we process these decisions, the fact remains that for each of them our brains have made a complex analysis of the different variables of which we’re aware within a given situation and decided on a particular course. We’re astoundingly good at this, given we need to do it on a minute-by-minute basis every day of our lives, keeping track of huge numbers of minor details without really thinking. However, we are by our nature fallible and do often get things wrong.

Given how important the accurate estimation of risk is to both the financial or betting industries on both an institutional and individual level, it’s important to understand how this is accomplished. It’s also useful to consider the potential pitfalls of inaccurate assessment, as well as how such errors can be avoided.

How Do We Measure Risk?

As previously mentioned, most of us make nearly unconscious risk-assessments all the time in our heads. This usually works well for everyday situations, but we all have our own biases, fallacies, and irrational ways of reacting to events to which we can fall victim in decision-making. So when it comes to important decisions such as in what way to trade millions of dollars of shares or which medical treatment should get government funding, we need help maintaining objectivity and rationality. Using statistical models to analyse which is the best outcome as well as how likely negative outcomes are allows us to disregard our own biases and assess decisions objectively, thereby allowing us to make better decisions. Essentially, these models are prosthetics which augment our ability to make the right call in situations too complex for our normal faculties to handle effectively.

We construct these models by gathering data, whether through scientific enquiry or passive methods such as records of stocks traded, which are publicly accessible. When we have enough data, we can analyse it and draw conclusions to apply as needed. There are a huge number of statistical tools that allow us to do this, and while I’m no statistician, at their base level many are easy to understand. Take the Law of Large Numbers, for instance, which states that a large enough sampling of data from the same set will tend to converge around the “expected value” (the weighted average of all possible values for the variable being measured). In essence, it means that the more times we repeat the same test, the closer we come to knowing the truth. For instance, the expected value for rolling a standard 6-sided die is 3.5; the more times we roll the die, the more closely the average of the results found will converge on that number.

There are numerous other powerful tools we can bring to bear on problems of assessing risk, such as Bayesian Probability, and they are all used in the assembly of models used for just that purpose. These models, in turn, allow the creation of complex algorithms and computer programs used to trade on financial markets worldwide. In fact, the vast majority of trading on US stock exchanges is done by programs known as “algo-bots.” This is both hugely positive and introduces enormous potential flaws into the system upon which the entire global economy is now based, as without any human oversight, programs can cause huge problems when they encounter situations they’re unable to handle.

I’ve written about the problems associated with exclusively using theoretical models to make decisions before, but a good example of the problems which occur when things go wrong is the “Flash Crash” of 2010. On the 6th of May that year, at 2:32pm, a mutual fund’s algo-bot attempted to trade $4.1bn worth of a particular type of futures commodity, a huge sale by any standard, which combined with increased volatility in trading that day to trigger a selling spree by other automated programs in the same market. As selling worsened over the course of the next 10 minutes, the fever spread to the stock markets and caused the Dow Jones Industrial Average to tumble by 600 points in a matter of minutes. The fall was only stopped when an automated monitoring program kicked in and stopped trading for five seconds, alleviating the pressure of so many programs selling and allowing the markets to start recovering (which itself took only minutes). Five seconds may not seem like much time to you or I, but in the time-frame of High Frequency Trading and algo-bots it’s more than enough time for such a precipitous drop to begin to reverse itself. By 3:00pm most of those 600 points lost had been regained, marking the largest one-day swing in stock market history.

This incident, short and sharp, which caused the largest one-day swing in the Dow Jones’ history, was entirely automated and perfectly illustrates the problem with inaccurately modeling risk. After Black Monday in 1987, where a very similar problem occurred with automated trading programs, the risk of using the primitive trading programs of the time (which accounted for only 10% of trading back then) was considered too great without human oversight. Unfortunately, our current system is built around such algorithmically-derived programs, and so one of the few things we can do is build safeguards like the system which stopped trading during the Crash. It also highlights the issue of inadequate modeling in the basic design of the algorithms governing the bots’ behaviour. It isn’t possible to anticipate every single situation using a model derived from past data, but it’s definitely possible to include situations such as the Flash Crash -which lie at the far end of the normal distribution of probabilities- in that model.

Modeling risk when betting on live events is arguably less easy than doing so with pure data placed at something of a remove from the real world, such as the financial markets. That being said, the potential losses are more limited, you are for instance unable to lose $2.1bn betting on horses (though I’m sure there are those who have tried). If the financial markets represent the most complex man-made system ever made, then consider how complex it is to assess risk statistically on betting exchanges fundamentally reliant on real world situations with a near-infinite number of variables.

The 2012 Grand National is a prime example of such unaccounted-for risk. The favourite horse, Synchronised, which had won the Cheltenham Gold Cup only weeks before and looked on track to win, fell at a notorious jump and had to be put down. At a stroke, this unanticipated event dashed the hopes and stakes of those who’d joined the consensus on its ability to win, not taking into account the dangerousness of the course or the likelihood of any one horse falling in spreading their bets.

While the incident has served to further fuel an already fierce debate over the safety of National Hunt-style racing in Britain, it also reinforces the point that even with the best modeling, it’s impossible to anticipate the chance of a particular horse falling and dying from past performance (as it obviously only happens once). The best that can be done is to include the possibility of any horse falling in a model that informs your spread on an exchange or between different bookmakers. It provides evidence for the argument that it’s fundamentally more difficult to account for risks when modeling exchanges based more firmly on the real than, say, the commodities futures markets.

The failure of a statistical modeling of specific risks in instances like the one I describe above allows us to see the limits of such an approach. While it’s always good to know the overall likelihood of an event occurring and be able to change your approach to suit that probability, the fact remains that knowing about risks and being able to truly account for them are two different things. It’s always tempting to believe we can fully control events if we understand them, but despite our best efforts that remains an impossibility. The most we can achieve is to measure and account for as much risk as we can.