Mutual Information-based RBM Neural Networks
Mutual Information-based RBM Neural Networks
Loading...
Files
Publication or External Link
Date
2016
Authors
Peng, Kang-Hao
Advisor
Chellappa, Rama
Citation
DRUM DOI
Abstract
(Deep) neural networks are increasingly being used for various
computer vision and pattern recognition tasks due to their strong
ability to learn highly discriminative features. However, quantitative
analysis of their classication ability and design philosophies are still
nebulous. In this work, we use information theory to analyze the
concatenated restricted Boltzmann machines (RBMs) and propose a
mutual information-based RBM neural networks (MI-RBM). We
develop a novel pretraining algorithm to maximize the mutual
information between RBMs. Extensive experimental results on
various classication tasks show the eectiveness of the proposed
approach.