1 December 2021 in Management Science
A four-year series of subjective probability forecasting tournaments sponsored by the U.S. intelligence community revealed a host of replicable drivers of predictive accuracy, including experimental interventions such as training in probabilistic reasoning, anti‐groupthink teaming, and tracking of talent. Drawing on these data, we propose a Bayesian BIN model (Bias, Information, Noise) for disentangling the underlying processes that enable forecasters and forecasting methods to improve—either by tamping down bias and noise in judgment or by ramping up the efficient extraction of valid information from the environment. The BIN model reveals that noise reduction plays a surprisingly consistent role across all three methods of enhancing performance. We see the BIN method as useful in focusing managerial interventions on what works when and why in a wide range of domains. An R-package called BINtools implements our method and is available on the first author’s personal website. This paper was accepted by Manel Baucells, decision analysis.
1 September 2023 in Management Science
A combination of point estimates from multiple judges often provides a more accurate aggregate estimate than a point estimate from a single judge, a phenomenon called “the wisdom of crowds.” However, if the judges use shared information when forming their estimates, the simple average will end up overemphasizing this common component at the expense of the judges’ private information. A decision maker could in theory obtain a more accurate estimate by appropriately combining all information behind the judges’ opinions. Although this information underlies the judges’ individual estimates, it is typically unobservable and thus cannot be directly aggregated by a decision maker. In this article, we propose a weighting of judges’ individual estimates that appropriately combines their collective information within a single estimation problem. Judges are asked to provide both a point estimate of the quantity of interest and a prediction of the average estimate that will be given by all other judges. Predictions of others are then used as part of a criterion to determine weights that are applied to each judge’s estimate to form an aggregate estimate. Our weighting procedure is robust to noise in the judges’ responses and can be expressed in closed form. We use both simulation and data from a collection of experimental studies to illustrate that the weighting procedure outperforms existing methods. An R package called metaggR implements our method and is available on the Comprehensive R Archive Network. This paper was accepted by Manel Baucells, behavioral economics and decision analysis. Funding: This work was supported by the Indiana University Kelley School of Business and INSEAD. Supplemental Material: The data files and e-companion are available at https://doi.org/10.1287/mnsc.2022.4648 .
1 September 2024 in Manufacturing & Service Operations Management
Problem definition: Loyalty programs have greatly expanded in scale and scope, and loyalty points issued by firms serve as a new form of currency alongside the traditional currency of money. In this paper, we study how consumers decide to pay with points or money for a purchase and how these decisions are affected by consumers’ points earning characteristics. Methodology/results: We develop a model of consumers’ payment choices and estimate it on proprietary loyalty program data from a major U.S. airline company using a hierarchical Bayesian framework. Our results demonstrate that mental accounting, the subjective perceived value of points, and the reference exchange rate play important roles in consumers’ payment choices. Moreover, the primary points earning source and the total earning level are jointly associated with consumers’ attitudes toward points and money: Consumers who earn many points and mostly with the focal firm tend to value points more than money, while those who earn few points or mostly through a cobranded credit card tend to value money more than points. To better understand heterogeneity in consumers’ attitudes toward points, we propose a probabilistic segmentation of consumers and identify four behavioral segments with distinctive characteristics. Through counterfactual analysis, we demonstrate how a firm can implement money and point pricing policies to optimally target and influence consumers’ payment choices. Managerial implications: It is important for firms to understand how consumers think about points and decide to pay with points or money as it affects firms’ cash flows, profitability, and consumer loyalty and engagement. We demonstrate that firms can design optimal price targeting policies by utilizing the significant level of heterogeneity in consumers’ attitudes toward points. Furthermore, our results indicate that firms that are expanding their partnership networks and offering more points earning sources should consider the impact that this would have on consumers’ mental accounting of money and points, and consequently their payment choices. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2022.0287 .
1 November 2022 in Operations Research
How much can rational people really disagree? If we can understand the limits of such disagreement, can we remove noise by labeling excess disagreement as irrational and then construct a group belief based on everyone's rational beliefs? Based on this idea, “Regularized Aggregation of One-Off Probability Predictions” by Satopää proposes a Bayesian aggregator that requires no user intervention and can be computed efficiently even for a large number of one-off probability predictions. To illustrate, the aggregator is evaluated on predictions collected during a four-year forecasting tournament sponsored by the U.S. intelligence community. The aggregator improves the squared error (a.k.a., the Brier score) of simple averaging by around 20% and other commonly used aggregators by 10%−25%. This advantage stems almost exclusively from improved calibration. An R package called braggR implements the method and is available on CRAN.
1 May 2023 in Management Science
Decision makers often ask experts to forecast a future state. Experts, however, can be biased. In the economics and psychology literature, one extensively studied behavioral bias is called herding. Under strong levels of herding, disclosure of public information may lower forecasting accuracy. This result, however, is derived only for point forecasts. In this paper, we consider experts’ probabilistic forecasts under herding, find a closed-form expression for the first two moments of a unique equilibrium forecast, and show that the experts report too similar locations and inflate the variance of their forecasts because of herding. Furthermore, we show that the negative externality of public information no longer holds. In addition to reacting to new information as expected, probabilistic forecasts contain more information about the experts’ full beliefs and interpersonal structure. This facilitates model estimation. To this end, we consider a one-shot setting with one forecast per expert and show that our model is identifiable up to an infinite number of solutions based on point forecasts but up to two solutions based on probabilistic forecasts. We then provide a Bayesian estimation procedure for these two solutions and apply it to economic forecasting data collected by the European Central Bank and the Federal Reserve Bank of Philadelphia. We find that, on average, the experts invest around 19% of their efforts into making similar forecasts. The level of herding shows an increasing trend from 1999 to 2007 but drops sharply during the financial crisis of 2007–2009 and then rises again until 2019.This paper was accepted by Yan Chen, behavioral economics and decision analysis.Supplemental Material: The electronic companion and data are available at https://doi.org/10.1287/mnsc.2022.4487 .
1 January 2024 in Decision