Network Unfolding Algorithm and Universal Spatiotemporal Function Approximation

Abstract

It has previously been known that a feed-forward network with time-delay can be unfolded into a conventional feed-forward network with a time history as input. In this paper, We show explicitly how this unfolding operation can occur, with a newly defined Network Unfolding Algorithm (NUA) that involves creation of virtual units and moving all time delays to a preprocessing stage consisting of the time histories. The NUA provides a tool for analyzing the complexity of the ATNN. From this tool, we concluded that the ATNN reduces the cost of network complexity by at least a factor of O(n) compared to an unfolded Backpropagation net. We then applied the theorem of Funahashi, Hornik et al and Stone-Weierstrass to state the general function approximation ability of the ATNN. We furthermore show a lemma (Lemma 1) that the adaptation of time-delays is mathematically equivalent to the adjustment of interconnections on a unfolded feed-forward network provided there are a large enough number (h2nd) of hidden units. Since this number of hidden units is often impractically large, we can conclude that the TDNN and ATNN are thus more powerful than BP with a time history.