Every day, while folks are stuck at home, politicians, public health officials, and slick talking heads point to charts showing the latest statistics on the coronavirus pandemic as they attempt to predict what might happen next in your neck of the woods. Underlying these graphics are various forecasting models, which you should approach with a healthy dose of skepticism.
It is tempting to view the models as oracles that will help predict how the disease will spread, tell you what to do and when to do it. But these models are simplified versions of realty. Reality is reality. Models should be read with the greatest care. They are not a substitute for controlled scientific experiments that generate relevant data.
Models certainly provide information that can create a framework for understanding a situation. But models, including those used to predict COVID-19′s trajectory, aren’t crystal balls. A model is simply a tool. It consists of raw data, along with assumptions based on our best guesses at the time, that together shape an overall forecast.
A model is only as good as its underlying data, which is in short supply. For example, there is still plenty of uncertainty about how many COVID-19 deaths may occur over the next six months under various social distancing and mask wearing scenarios. Also, a model’s accuracy is constrained by uncertainty about how many people are or have been infected.
Assumptions aren’t facts. Put another way, models are constrained by what is known and what is assumed. Understanding these underlying assumptions helps explain why some forecasts have a sunny disposition, while others can’t be pessimistic enough.
There are also economic models. Financial mavens develop them to take stock of how the pandemic has impacted the economy and where they see it and markets heading. With so many countries experiencing sharp declines in gross domestic product, there is a lot of forecasting about what shape the recovery will take. Will there be a quick V-shaped recovery or will it be U-shaped? Or maybe a little bit of both?
These models also have their limitations. Recall how Long-Term Capital Management, an industry-leading hedge fund run by a renowned team of mathematical experts that included two Nobel Prize winners, developed complex quantitative models to analyze markets and placed huge bets on the assumption, among others, that Russia would never default on its bonds. They did a lousy job of stress testing their assumptions and they bet wrong. In September 1998, the firm had to be bailed out by a consortium of Wall Street banks to prevent the bottom dropping out of the financial system.
This episode was a coming attraction for the harrowing financial crisis a decade later in September 2008, which was perhaps the biggest event of the 21st century until COVID-19. Prior to the 2008 crisis, a key assumption in many models was that housing prices would always go up. Indeed, one cause of the meltdown was the quant movement: the proliferation of quantitative models for designing and analyzing financial products as well as for risk management. Many finance professionals mistakenly believed that quantitative tools had allowed them to conquer risk. Products such as derivatives, subprime mortgage-backed securities and activities that relied heavily on quantitative models were at the heart of how financial firms expanded their activities to take more and greater risks.
And of course, with the presidential election just months away, Americans still remember how 2016 election models forecast Hilary Clinton waltzing into the White House. Between now and Nov. 3, many people will take election forecasts with an extra grain of salt or three.
Given the events of the last several months, people should keep a simple fact in mind: Models should not be asked to carry any more than they can bear. So when you hear about models put on your hmmm face.
