Facial and expression recognition for the blind using computer vision

Abstract

The majority of communication between humans is comprised of nonverbal cues. Visual cues, such as expressions and nodding, are not readily accessible to the blind. Our team has developed an assistive device based on computer vision, which relays facial recognition and expression information to a blind user and is designed to store images of people the user frequently interacts with and analyze faces for expressions. The control a user has over his/her surroundings while receiving real-time feedback contributes to a unique device. In order to design a device suited best to the blind user's needs, we engaged sighted and blind participants in surveys and interviews to understand their views and preferences regarding the availability of a computer vision system that, in real-time, can pro- vide information about identity and expressions of humans. This thesis discusses the development methodology, the selection of algorithms for recognizing faces and expressions and physical designs of the device, and the results on subject tests with blind participants to gauge the e ectiveness of our design.