Neural Network Generation of Temporal Sequences from Single Static Vector Inputs using Varying Length Distal Target Sequences
Neural Network Generation of Temporal Sequences from Single Static Vector Inputs using Varying Length Distal Target Sequences
Files
Publication or External Link
Date
2007-02-01
Authors
Gittens, Shaun
Advisor
Citation
DRUM DOI
Abstract
Training an agent to operate in an environment whose
mappings are largely unknown is generally recognized to be exceptionally
difficult. Further, granting such a learning agent the ability to
produce an appropriate sequence of actions entirely from a single input
stimulus remains a key problem. Various reinforcement learning
techniques have been utilized to handle such learning tasks, but
convergence to optimal policies is not guaranteed for many of these
methods. Traditional supervised learning methods hold more assurances of
convergence, but these methods are not well suited for tasks where
desired actions in the output space of the learner, termed proximal
actions, are not available for training. Rather, target outputs from the
environment are distal from where the learning takes place. For example,
a child acquiring language who makes speech errors must learn to correct
them based on heard information that reaches his/her auditory cortex
which is distant from the motor cortical regions that control speech
output. While distal supervised learning techniques for neural networks
have been devised, it remains to be established how they can be trained
to produce sequences of proximal actions from only a single static
input. In this research, I develop an architecture which incorporates
recurrent multi-layered neural networks that possess some form of
history in the form of a context vector into the distal supervised
learning framework, enabling it to learn to generate correct proximal
sequences from single static input stimuli. This is in contrast to
existing distal learning methods designed for non-recurrent neural
network learners that utilize no concept of memory of their prior
behavior. Also, I adapt a technique in this research known as teacher
forcing for use in distal sequential learning settings which is shown to
result in more efficient usage of the recurrent neural network's context
layer. The effectiveness of my approach is demonstrated by applying it
to acquire varying length phoneme sequence generation behavior using
only previously heard and stored auditory phoneme sequences. The results
indicate that simple recurrent backpropagation networks can be
integrated with distal learning methods to create effective sequence
generators even when they do not constantly update current state
information.