Crowdsourcing decision support: frugal human computation for efficient decision input acquisition

Abstract

When faced with data-intensive decision problems, individuals, businesses, and governmental decision-makers must balance trade-offs between optimality and the high cost of conducting a thorough decision process. The unprecedented availability of information online has created opportunities to make well-informed, near-optimal decisions more efficiently. A key challenge that remains is the difficulty of efficiently gathering the requisite details in a form suitable for making the decision.

Human computation and social media have opened new avenues for gathering relevant information or opinions in support of a decision-making process. It is now possible to coordinate paid web workers from online labor markets such as Amazon Mechanical Turk and others in a distributed search party for the needed information. However, the strategies that individuals employ when confronted with too much information--satisficing, information foraging, etc.--are more difficult to apply with a large, distributed group. Consequently, current distributed approaches are inherently wasteful of human time and effort.

This dissertation offers a method for coordinating workers to efficiently enter the inputs for spreadsheet decision models. As a basis for developing and understanding the idea, I developed AskSheet, a system that uses decision models represented as spreadsheets. The user provides a spreadsheet model of a decision, the formulas of which are analyzed to calculate the value of information for each of the decision inputs. With that, it is able to prioritize the inputs and make the decision input acquisition process more frugal. In doing so, it trades machine capacity for analyzing the model for a reduction in the cost and burden to the humans providing the needed information.