Dr. Gerard Adams, a distinguished professor of economics, once began a 1980s class lecture describing the intricacies of assembling hundreds of data series and correlations that comprised the highly touted Wharton Econometric Forecasting Model. At the end of the modeling process, he would sit at a boardroom table with Nobel laureate Laurence Klein and the rest of their WEFA colleagues to review the results of the highly complex model. They would stare at the output for a few minutes, quietly place the report down on the table, and state in near unison…there is no way in hell that will happen.

Welcome to the world of forecasting models.

Forecasting capabilities have progressed significantly in the past thirty years, but Google any combination of derogatory words and the term “forecasting models” and you will find plenty of venom unleashed on the dark science of predctive modeling. The harshest comments tend to target weather and economic forecasting models.

Nassim Nicholas Taleb, who proudly counts himself among the current model detractors, recently tweeted “All models are wrong. Some are lethal.”

I will assign a 99.999% certainty to Taleb’s statement. While examples of spot-on model predictions exist, their degree of success falls within the range of blind squirrels finding nuts and broken clocks showing accurate results twice a day. No model stands the test of precision over time, but that does not mean we should abandon their use.

The Value of Predictive Models

Venturing into the danger zone of interpreting Taleb, his comments on models are less a criticism of algorithmic accuracy than a rebuke of how we use and rely on models. Models do not offer answers, but they can help guide business decision making if used in the right context. Taleb’s best known criticism targets the turkeys who act on modeled likely outcomes while ignoring the possibilities of tail risks, especially those carrying profound consequences if they occur.

If all models are wrong, and they should not eliminate consideration of any potential outcomes, why use them? Static models are fine for academic and theoretical purposes to help discover how systems function and to assess the influence of an isolated variable when holding all others constant. But as this blog space emphasizes, the use of ceteris paribus analysis is futile for business decision making. Dynamic business environments require dynamic models.

Models earn their poor reputation when used as a reference only at the beginning of a planning process. All models, from economic and financial investment models to weather and cyberattack models, represent dynamic systems with constantly changing variables. Any snapshot of a future prediction becomes increasingly irrelevant and error-prone as the system variables change over time, especially in systems with highly volatile variables.

The value of models is their ability to not only support business analysis and plan development, but also guide corrective actions throughout the execution of a plan.

In a business setting, organizational objectives set desired destinations. In the planning stage, models can help

1. determine the resources, timing, and road map needed to reach the destination,
2. identify the potential risks associated with the business plan, and
3. develop contingencies using scenario analysis.

In the execution stage, dynamic modeling can provide the basis for ongoing monitoring and assessments of risk variables and trigger decisions to enact contingency plans.

Consider how far models have progressed for use in road trips. Back in the old days, we used a fold out map, weather conditions, and advice from friends to help plan the best route to a destination, how long it would take, and the gas needed.

Now, Google Maps, using a model based on historical traffic patterns, road conditions, and real-time Waze data, can provide highly accurate route options and time expectations. We can easily plan a primary route and contingency routes while adding additional considerations like fueling options and restaurants along the route. The model’s usefulness also does not end after the planning stage. Google Maps’ real-time updates can identify accidents and traffic patterns that will alter our original plan and offer alternative routes to our destination.

Similar to the models that power navigation systems, our business planning models should facilitate ongoing monitoring and dynamic assessments as we carry out business plans. 

The Future of Dynamic Modeling

The future value of models depends on their ability to help establish business plans, conduct scenario planning, and provide the basis for monitoring changes in decision variables and risk factors as plans are executed. The progress in modeling and monitoring factory manufacturing operations gives evidence for how models can greatly enhance proficiency in plan execution. This approach to system modeling is now extending to other industry sectors and business functions to support planning and business decision making.

The 2011 tsunami in Japan and floods in Thailand which exposed major supply chain weaknesses across the globe, particularly for computer and auto manufacturers. Supply chain modeling and monitoring platforms from companies like Resilinc now eliminate much of the risks of single sourcing, material premiums, and factory shutdowns.

In another example, IBM/The Weather Company and Monsanto/Climate Corp. have invested heavily in building models for agricultural production. Their model forecasts, variable monitoring, and decision guidance are now in use across the world.

Looking ahead, this melding of system modeling and real-time monitoring will extend to enterprise decision making. Consider today’s executive dashboards as a type of crude system model. Dashboards will typically include a set of key performance indicators (KPIs) and perhaps a set of key risk indicators (KRIs). Program developers and individual executives presumably choose to display specific KPIs and KRIs based on their important roles as variables and outcomes in the systems they manage.

The value of most dashboards can be further enhanced by clearly presenting a model of the business system with explicit probability and weights associated with each factor presented. Real-time monitoring would then better facilitate when to shift to contingency plans as specific variables approach critical thresholds.

Predictive modeling and business analytics have advanced tremendously since the days of the WEFA Econometric Forecasting Model. The complexity of current business decision making models ranges widely from those built on basic statistics or advanced statistics to deep learning neural networks.

Regardless of the source and development details, business decision makers should use all models with healthy skepticism. IBM, and others advancing the use of artificial intelligence, tend to avoid positioning new AI models as replacements for human decision-making processes and instead refer to technical solutions that augment decision making.

Taleb is correct to note static models are poor predictors of actual business outcomes. The value of business models is instead their ability to help explain systems and dynamically guide decision making until an objective is achieved.