EVALUATING METHODS FOR MODELING AND AGGREGATING CONTINUOUS DISTRIBUTIONS OF FORECASTER BELIEF

Simple item page
dc.contributor.advisorWallsten, Thomasen_US
dc.contributor.advisorDougherty, Michaelen_US
dc.contributor.authorTidwell, Joeen_US
dc.contributor.departmentPsychologyen_US
dc.contributor.publisherDigital Repository at the University of Marylanden_US
dc.contributor.publisherUniversity of Maryland (College Park, Md.)en_US
dc.date.accessioned2017-09-14T05:32:11Z
dc.date.available2017-09-14T05:32:11Z
dc.date.issued2017en_US
dc.description.abstractThe ``Wisdom of the crowds'' is the concept that the average estimate of a group of judges is often more accurate than any single judge’s estimate. This dissertation explores a variety of elicitation, modeling, and aggregation methods for time-based forecasting questions at both the individual and consensus levels, and shows that accurate continuous forecast distributions can be modeled from relatively few judgments from individual forecasters. For individual forecasters, eliciting judgments with fixed versus random cut points, and modeling those judgments with least-squares methods led to the most accurate forecasts. While gamma distributions fit the empirical judgments more closely than exponential distributions, exponential fits yielded more accurate model forecasts, suggesting that the greater flexibility of the gamma distribution tended to over-fit the empirical forecasts. For consensus forecasts, random cut points across individual forecasters yielded more accurate forecasts than fixed cut points, suggesting that across a group of forecasters, random bins may help average over individual-level forecast errors introduced through partition dependence bias and an arbitrary set of fixed cut points. With respect to modeling methods, a mixture of forecaster distributions fit with a Bayesian Dirichlet-multinomial model performed best across a variety of metrics and yielded forecast accuracies on par with advanced discrete aggregation techniques. This model also provides a natural way to weight individual forecasters according to expertise and other factors. Differences in forecast accuracy between modeling methods varied greatly depending on when an event occurred relative to the range over which forecaster judgments were elicited, particularly when events occurred long after the last date for which forecasters provided judgments. In these cases, the modeled forecasts depend heavily on the assumptions of the model versus the elicited judgments, and forecasts should be cautiously interpreted as representing crowd belief. The results of this research shows that with a limited number of discrete elicited judgments, it is possible to obtain continuous aggregate models of forecaster belief that are as accurate as discrete forecast aggregation methods, but can also provide decision makers with forecasts for arbitrary partitions of the event space and can be easily integrated into a broad range of decision analyses.en_US
dc.identifierhttps://doi.org/10.13016/M2QN5ZC04
dc.identifier.urihttp://hdl.handle.net/1903/19856
dc.language.isoenen_US
dc.subject.pqcontrolledPsychologyen_US
dc.subject.pquncontrolledaggregationen_US
dc.subject.pquncontrolledforecastingen_US
dc.subject.pquncontrolledwisdom of the crowdsen_US
dc.titleEVALUATING METHODS FOR MODELING AND AGGREGATING CONTINUOUS DISTRIBUTIONS OF FORECASTER BELIEFen_US
dc.typeDissertationen_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Tidwell_umd_0117E_18117.pdf
Size:
1.11 MB
Format:
Adobe Portable Document Format
Download

Collections

UMD Theses and Dissertations
Psychology Theses and Dissertations