ENRICHING COMMUNICATION BETWEEN HUMANS AND AI AGENTS

Abstract

Equipping AI agents with effective, human-compatible communication capabilities is pivotal to enabling them to effectively serve and aid humans. On one hand, agents should understand humans, being able to infer intentions and extract knowledge from language utterances. On the other hand, they should also help humans understand them, conveying (un)certainties and proactively consulting humans when facing difficult situations.

This dissertation presents new training and evaluation frameworks that enrich communication between humans and AI agents. These frameworks improve two capabilities of an agent: (1) the ability to learn through natural communication with humans and (2) the ability to request and interpret information from humans during task execution. Regarding the first capability, I study the possibility and challenges of training agents with noisy human ratings. Providing humans with more expressive tools for teaching agents, I propose a framework that employs descriptive language as the teaching medium. On the second capability, I introduce new benchmarks that evaluate an agent’s ability to exchange information with humans to successfully perform indoor navigation tasks. On these benchmarks, I build agents that are capable of requesting rich, contextually useful information and show that they significantly outperform those without such capability. I conclude the dissertation with discussions on how to develop more sophisticated communication capabilities for agents.