Institute for Systems Research Technical Reports

Stochastic optimization is a direct application of the above root finding problem if the SA is driven by a gradient estimate of steady state performance. We study the convergence properties of an SA driven by agradient estimator which observes an increasing number of samples from the Markov chain at each step of the SA's recursion. To show almost sure convergence to the optimizer, a framework of verifiable conditions is introduced which builds on the general SA conditions proposed for the root finding problem.

We also consider a difficulty sometimes encountered in applicationswhen selecting the set used in the projection operator of the SA algorithm.Suppose there exists a well-behaved positive recurrent region of the state process parameter space where the convergence conditions are satisfied; this being the ideal set to project on. Unfortunately, the boundaries of this projection set are not known a priori when implementing the SA. Therefore, we consider the convergence properties when the projection set is chosen to include regions outside the well-behaved region. Specifically, we consider an SA applied to an M/M/1 which adjusts the service rate parameter when the projection set includes parameters that cause the queue to be transient.

Finally, we consider an alternative SA where the recursion is driven by a sample average of observations. We develop conditions implying convergence for this algorithm which are based on a uniform large deviation upper bound and we present specialized conditions implyingthis property for finite state Markov chains.

However, it has been a mystery where andhow such harmonic templates can come about. Here we present a biologicallyplausible model for how such templates can form in the early stages of theauditory system. The model demonstrates that {it any} broadband stimulussuch as noise or random click trains, suffices for generating thetemplates, and that there is no need for any delay-lines, oscillators, orother neural temporal structures.

The model consists of two key stages:cochlear filtering followed by coincidence detection. The cochlear stageprovides responses analogous to those seen on the auditory-nerve andcochlear nucleus. Specifically, it performs moderately sharp frequencyanalysis via a filter-bank with tonotopically ordered center frequencies(CFs); the rectified and phase-locked filter responses are further enhancedtemporally to resemble the synchronized responses of cells in the cochlearnucleus.

The second stage is a matrix of coincidence detectors thatcompute the average pair-wise instantaneous correlation (or product)between responses from all CFs across the channels. Model simulations showthat for any broadband stimulus, high coincidences occur between cochlearchannels that are exactly harmonic distances apart. Accumulatingcoincidences over time results in the formation of harmonic templates forall fundamental frequencies in the phase-locking frequency range.

Themodel explains the critical role played by three subtle but importantfactors in cochlear function: the nonlinear transformations following thefiltering stage; the rapid phase-shifts of the traveling wave near itsresonance; and the spectral resolution of the cochlear filters. Finally, wediscuss the physiological correlates and location of such a process and itsresulting templates.

We propose a new classification scheme, dubbed spectral classification, which uses the spectral characteristics of the image blocks to classify them into one of a finite number of classes. The spectral classifier is used in adaptive image coding based on the discrete wavelet transform and shown to outperform gain-based classifiers while requiring a lower computational complexity. The resulting image coding system provides one of the best available rate-distortion performances in the literature. Also, we introduce a family of multiresolution image coding systems with different constraints on the complexity. For the class of rate-scalable image coding systems, we address the problem of progressive transmission and propose a method for fast reconstruction of a subband-decomposed progressively transmitted image.

Another important problem studied in this dissertation is the transmission of images over noisy channels, especially for the wireless channels in which the characteristics of the channel is time-varying. We propose an adaptive rate allocation scheme to optimally choose the rates of the source coder and channel coder pair in a tandem source-channel coding framework. Also, we suggest two adaptive coding systems for quantization and transmission over a finite-state channel using a combined source and channel coding scheme. Finally, we develop simple table- lookup encoders to reduce the complexity of channel-optimized quantizers while providing a slightly inferior performance. We propose the use of lookup tables for transcoding in heterogeneous networks