Clashes in the Infosphere, General Intelligence, and Metacognition: Final project report

Abstract

Humans confront the unexpected every day, deal with it, and often learn from it. AI agents, on the other hand, are typically brittle—they tend to break down as soon as something happens for which their creators did not explicitly anticipate. The central focus of our research project is this problem of brittleness which may also be the single most important problem facing AI research. Our approach to brittleness is to model a common method that humans use to deal with the unexpected, namely to note occurrences of the unexpected (i.e., anomalies), to assess any problem signaled by the anomaly, and then to guide a response or solution that resolves it. The result is the Note-Assess-Guide procedure of what we call the Metacognitive Loop or MCL. To do this, we have implemented MCL-based systems that enable agents to help themselves; they must establish expectations and monitor them, note failed expectations, assess their causes, and then choose appropriate responses. Activities for this project have developed and refined a human-dialog agent and a robot navigation system to test the generality of this approach.