Mutual Information-based RBM Neural Networks

Loading...
Thumbnail Image
Files
Publication or External Link
Date
2016
Authors
Peng, Kang-Hao
Advisor
Chellappa, Rama
Citation
Abstract
(Deep) neural networks are increasingly being used for various computer vision and pattern recognition tasks due to their strong ability to learn highly discriminative features. However, quantitative analysis of their classication ability and design philosophies are still nebulous. In this work, we use information theory to analyze the concatenated restricted Boltzmann machines (RBMs) and propose a mutual information-based RBM neural networks (MI-RBM). We develop a novel pretraining algorithm to maximize the mutual information between RBMs. Extensive experimental results on various classication tasks show the eectiveness of the proposed approach.
Notes
Rights