(Deep) neural networks are increasingly being used for various

computer vision and pattern recognition tasks due to their strong

ability to learn highly discriminative features. However, quantitative

analysis of their classication ability and design philosophies are still

nebulous. In this work, we use information theory to analyze the

concatenated restricted Boltzmann machines (RBMs) and propose a

mutual information-based RBM neural networks (MI-RBM). We

develop a novel pretraining algorithm to maximize the mutual

information between RBMs. Extensive experimental results on

various classication tasks show the eectiveness of the proposed

approach.