Hierarchical Neural Networks

Abstract

With the common three-layer neural network architectures, networks lack internal structure; as a consequence, it is very difficult to discern characteristics of the knowledge acquired by a network, in order to evaluate its reliability and applicability. An alternative neural-network architecture is presented, based on a hierarchical organization. Hierarchical networks consist of a number of loosely-coupled subnets, arranged in layers. Each subnet is intended to capture specific aspects of the input data. A subnet models a particular subset of the input variables, but the exact patterns and relationships among variables are determined by training the network as a whole. However, the hierarchy of subnets gives the network hints to look for patterns in the most promising directions. Their modular organization makes hierarchical neural networks easier to analyze, because one can focus on the analysis of one subnet at a time, rather than attempt to decipher the whole network at once.