Why Market Outcomes Are Hard to Predict

It is a recurring conundrum as to why the expected outcomes from a business or public program do not always turn out as predicted. Sometimes the outcome is better than expected, but mostly they fall short of what is planned for. This leads to business closures and program failures. There are three major and interrelated reasons for this lack of predictability. The first is that there is no systematic, rigorous way to determine outcomes as for many other areas such as mathematics or physics. The second is that there are many levels of decision-making involved in determining an outcome and those making predictions do not always look at the correct level of decision-making which will actually govern the outcome of the market. The third is that decisions depend on a cascading series of decisions through layers of a market which must all come to pass as predicted. In the end, what these three issues means is that market theory has no predictive power. Predictions are not made by any process that allows for someone else to replicate them. Also, if outcomes are not as predicted they are treated as the anomaly and not blamed on the predictive process. There is no iterative learning process as a result. Predictions continue to be made in the same manner with the same lack of predictability and the same variable outcomes. The decision process is not faulted or even reviewed. The lack of a systematic system to evaluate outcomes means that not all possibilities are included. The decision-making becomes ad hoc, dependent on gut feelings, and things we “know” play an outsized role in the process. The final decisions represent more a shot in the dark, a hope that we got it right than well-informed guesses. Most outcomes are just assumed as flowing in the manner in which we envision they will play out. This is just a form of Magical Thinking. We believe X will result if we do Y and are surprised when W happens. Cash for Clunkers or gun buybacks easily represent this problem. Cash for Clunkers had the desired effect of motivating people to trade in their older vehicles for cash to get them off the roads, but it also motivated them to trade in vehicles that were no longer on the roads and did not represent an improvement in air quality since they were not working. Gun buybacks get guns off the street, but also out of the closets, garages, and elsewhere, many of which are inoperable. In the end, neither were fully successful. Even if a more rigorous process is followed and a more well-thought out outcome arrived at, the actual outcome may still prove different than expected. The problem is that in a market there are layers of decision-making that affect the outcome. You can evaluate a market on one level and arrive at an appropriate outcome only to find that the actual locus of the decision occurs on a different level. Usually the level evaluated is higher than the one where the critical decisions are made. The predicted result depends on the outcomes flowing down through the multiple layers of decision-making, it does not take into account the upward influence of decisions made at the lower levels that may have more impact. It is not always obvious that the wrong level has been chosen until the market is allowed to operate and we don’t get the expected outcome. Charter schools and vouchers are a good representation of this problem. The idea behind them is that parents will choose better schools for their children if given the opportunity which in turn will force all schools to improve. Education overall will improve and the educational system will become more efficient and productive. It turns out that is not how it plays out. Parents usually don’t have sufficient information to choose among the alternatives so that their choices do not drive improvements in education outcomes as expected, but mostly enrich the operators of the schools. The level of decision-making is too indirect to the desired outcome to have the expected result. The expected flow through of effects from this program do not occur because the expected governing level is at the top when it is actually much lower. It only appears that the level identified is a low level (parents), when in actuality it is much higher (schools). The third major difficulty is that some outcomes depend on a series of decisions that if not arrived at in the manner envisioned will lead to a different outcome than expected. It is expected that the effects will cascade through the layers and result in the predicted outcome. It is a complex, almost Rube Goldberg, way of reaching the final outcome rather than a rational process. It is more akin to a series of if/then statements that all must be true in order for the predicted outcome to prevail. Again, school vouchers are a good example of this since the success of the program depends on a series of decisions all moving in the predicted manner. Is there a way to systematize decision-making in the markets to improve outcomes? It is possible that a foolproof method of making correct decisions is not realistic, that we can come up with a way to make the right predictions each time. Markets may not lend themselves to such a system. There are just too many variables when it comes to human decision-making on a large scale. We can, however, improve the process by which we make decisions and thereby produce more predictable outcomes. First and foremost is to stop engaging in Magical Thinking, that whatever we develop in our heads is how it will play out. A more structured, thoughtful approach is needed and we need to be willing to review our decisions and outcomes and adjust future approaches to making decisions. Second, we need to try to identify where and how decisions are made and drill down as close as possible to that point in evaluating the probable outcomes. Third, we need to recognize how dependent our outcomes are on the series of decisions involved coming true and how we can influence that. No doubt this will require an extended iterative process to improve our decisions to where unexpected outcomes are the outliers and not the norm. Only with a more rigorous approach can we get better results.