Many video coding standards such as H.261, MPEG1, MPEG2, HDTV and H.263, are based on the hybrid motion-compensated DCT approach. The common implementations adopt the conventional DCT-based motion-compensated video coder structure in which every raw video bit must go before being encoded through the performance- critical feedback loop consisting of a DCT, an Inverse DCT (IDCT) and a spatial-domain motion estimation/compensation. This heavily loaded feedback loop not only increases the overall complexity of the coder but also limits the throughput and becomes the bottleneck of a real-time high-end digital video system. In this dissertation, we propose a fully DCT-based motion-compensated video coder structure which eliminates IDCT and moves DCT out of the loop, resulting in a simple feedback loop with only one major component: Transform-Domain Motion Estimation/Compensation. Furthermore, different components can be jointly optimized if they operate in the same transform domain. Based on pseudo phases and sinusoidal orthogonal principles, we develop DCT Pseudo Phase Techniques to estimate displacement directly from the DCT coefficients of shifted signals/images. We develop the DCT-based motion estimation (DXT-ME) algorithm with the computational complexity, O(N2), compared to O(N4) for the full search block matching approach (BKM-ME). Simulation shows that the DXT-ME algorithm performs as well as, BKM-ME and other fast search approaches in terms of mean-square-error per pel (MSE) and bits per sample (BPS). Furthermore it has inherently highly parallel operations in computing the pseudo phases, suitable for VLSI implementation.

We develop DCT-based subpixel motion estimation algorithms without the need of image interpolation as required by conventional methods, resulting in significant reduction in computations and data flow and flexible fully DCT-based coder design because the same hardware can support different levels of required accuracy. Simulation demonstrates comparable and even better performance of the DCT-based approach than BKM-ME in terms of MSE and BPS.

We devise the DCT-based motion compensation schemes via the bilinear and cubic interpolation without any increase in computations to achieve more accurate approximation and better visual quality for motion- compensated pictures. Less computations are required in DCT domain because of sparseness and block alignment of DCT blocks and use of fast DCT to reduce computations.