10 OCTOBER 2018
by Raul Elizalde
This article also appeared on Forbes.com.
Watching how paint dries is not most people’s idea of fun. Nor is watching a sandpile grow, grain by grain. Or is it?
Scientists are not like most people. Some became quite intrigued by what happens to a sandpile as it grows. Their exploration led to break-through insights about stock market risk. The bottom line: As a sandpile grows, all sort of sand “avalanches” take place, but it is impossible to predict how big or how often they occur.
Sometimes a few grains roll down the slope, while occasionally a large avalanche carves a big section of the sandpile. The size and frequency of those avalanches, mathematically speaking, bear a notable resemblance to the size and frequency of earthquakes, solar flares, river floods, forest fires, and stock market returns. Intriguingly, all of them have defied attempts at prediction. The question is why.
Are market crashes inevitable?
One disconcerting aspect is that large avalanches, epic earthquakes or giant forest fires do not seem to be very special: They appear to be just less frequent, scaled-up versions of small ones. If this is true, then a stock market crash may not be special at all, but merely a larger-than-usual down day, and just as unpredictable. This would present a big challenge to traditional investment methods.
The sandpile study was introduced in a 1987 paper by Per Bak, Chao Tang and Kurt Wiesenfeld, three scientists working at the Physics Department at the Brookhaven National Laboratory. Ironically, the paper was presented to Physical Review Letters a few months before the stock market crash of October 1987, still today the largest ever one-day drop. The title was "Self-Organized Criticality" and falls within a branch of mathematics known as Complexity Theory, which studies how systems can organize themselves into unexpected behaviors arising from the interaction of its smallest and seemingly independent components.
The crucial point of their paper was that sandpile avalanches could not be predicted, and not because of randomness (there was no random component in their model) or because the authors could not figure out how to come up with equations to describe it. Rather, they found it impossible in a fundamental sense to set up equations that would describe the sandpile model analytically, so there was no way to predict what the sandpile would do. The only way to observe its behavior was to set up the model in a computer and let it run.
Likewise, stock prices have defeated all forecasting efforts, and may well belong to the same set of basic unpredictability. While occasionally somebody may seem to be on the right side of an investment ahead of a big move, this is a far cry from actually forecasting such move with any kind of precision in terms of timing and size. For each “hunch” that is successful, a myriad others fail. Despite anecdotes, there seems to be no clear evidence that investors who get a big move “right” are anything but lucky.
Can we tell if a bubble is about to burst?
Other scientists disagree with this notion, and note that market crashes are indeed “special.” Professor Didier Sornette, for example, a physicist at the Swiss Federal Institute of Technology, argued that a market crash is not simply a scaled-up version of a normal down day but a true outlier to market behavior. In fact, he claims that ahead of critical points the market starts giving off some clues. His work focuses on interpreting these clues and identify when a bubble may be forming and, crucially, when it ends.
He made the interesting observation that bubbles do not necessarily form in steady, long bull markets. For a bubble to form, price gains have to accelerate at a “super-exponential” rate.
It is well documented that prices tend to go up faster before a crash. This may seem counter-intuitive, but it makes sense in terms of “rational expectations.” For investors to remain invested in a market that is becoming more risky, prices have to rise faster in order to compensate for the growing probability of a crash. Otherwise, people would exit the market earlier and a bubble would never form.
This also means that it is a mistake to think of investors as a bunch of clueless, greed-driven lemmings falling off a cliff during a market crash. For example, during the real estate boom of the mid-2000s people kept buying homes despite an abundance of media articles pointing out that the property market was swept in a mania. There was no question, even then, that the market was overheated. So why did people continue to buy homes?
In Professor Sornette’s model, a bubble is a market heading to a critical point. But a crash is not the only possible post-crisis outcome: Prices can also stop rising and reach a higher plateau. It is precisely because of the small but real probability that a bubble will not crash but simply stop growing that it is rational for some investors to stay in the market, even when if they think that it has gone too far, too fast.
The critical point where bubbles end happens as investors begin to think that the rally is over. It is when this opinion travels deep into the system and becomes generalized that the system ends up in a crash. The paradox here is that a crash is often (and mistakenly) characterized as “market chaos.” In fact, it is the opposite: a crash reflects a highly ordered market, when everyone does the same thing (i.e. sell). A truly “chaotic” market is one where everyone is doing something different, interactions offset each other and price volatility remains low.
Real-life and computer simulations
A truly stunning result of these investigations is that the real-life frequency and size of market returns bear a notable resemblance to what is obtained by running very simple computer models. This also goes for earthquakes, solar flares, forest fires, and river floods: most of the simulations yield similar results to real life where events are frequent but small, but occasionally some gigantic one appears from nowhere.
Whether Professor Sornette is right or not that a critical point can be anticipated, the entire concept of market self-organization deals a blow to the “fundamental” approach to investing in equity markets – the idea that opinion-based research can lead to investment success when it seems quite apparent that outcomes cannot be predicted even when initial conditions are known.
While this is a radical statement, some investors seem to agree – they have been abandoning active mutual funds for years, while embracing index-tracking ETFs. This is rational because if markets are truly unpredictable, then it is a waste to pay any manager for their forecasts when they are unlikely to beat an index other than by luck.Hindsight is not foresightOne of the best illustrations of the trouble with “fundamental” analysis appears in the book Ubiquity by Mark Buchanan (another physicist interested in complexity theory). He imagines how an analyst of the sandpile world might write about an avalanche:
'"The trouble began a week ago in the West, where in the early evening a single grain of sand fell on a portion of our pile that was already very steep. This triggered a small avalanche, as a few grains toppled downhill toward the East. Unfortunately, the pile hasn’t been managed properly in the West, and these few grains entered into another region of the pile that was also already steep. Soon more grains toppled and throughout the night the avalanche grew in size by the next morning, it was well out of control. In retrospect, there is nothing surprising. One fateful grain falling a week ago led to a chain of events that swept catastrophe across the pile and into our own backyard here in the East. Had the Western authorities been more responsible, they could have removed some sand from the initial spot, and then none of this would have happened. It is a tragedy that we can only hope will never be repeated."
This is a remarkable passage because it resembles closely what one would read in an opinion-based analysis of a market event. The confusing illusion, of course, is that hindsight narratives of this kind could offer anything towards avoiding, let alone preventing, future disasters. In reality, no amount of knowledge of a sandpile system can possibly produce a usable forecast of the size and location of a major avalanche. It may be the same with a stock market crash.
It is not a big surprise, however, that many investors today remain interested in the forecasts of financial analysts regardless of their success. Humans in the past consulted oracles, crystal balls and tea leaves. It’s in our nature: As the proverb goes, “tell me a fact, and I'll learn tell me a truth, and I’ll believe but tell me a story and it will live in my heart forever.” We are attracted to story-telling, and when it comes to investing we seem to be searching for the most compelling narratives about the unknowable future, regardless of how accurate they turn out to be.
What investors can do
To be sure, we have made progress on many fronts. The weather, for example,
is a system where we can now produce short-term, usable forecasts. But other
systems have, at least so far, remained maddeningly unpredictable, such as
earthquakes, forest fires, and stock markets. Their complexity keeps them out
of our reach.
This does not mean that successful investing is impossible only that the more we learn about market behavior, the more it seems that trying to deal with uncertainty is more important than pretending that we can have any certainty.
More precisely, managing risk seems to be a better approach to investing than concocting forecasts on asset returns. This could mean, for example, finding ways of identifying when market participants start to align on one side of a trade by measuring correlations, or measuring returns to flash a warning when they start growing at “super-exponential” rates.
These are difficult concepts, and it is doubtful that investors will embrace them anytime soon. But it seems safe to assume that traditional investment approaches will continue to do little to protect investors from catastrophes that keep showing up from time to time.