This paper provides a methodology to encode video objects in a scalable manner with regard to both content and quality. Content scalability and quality scalability have been identified as required features in order to support video coding across different environments. Following the object-based approach to coding video, we extend our previous work on motion-based segmentation by using a time recursive approach to segmenting image sequences and decomposing a video "shot" into its constituent objects. Our formulation of the segmentation problem enables us to design a codec in which the information (shape, texture and motion) pertaining to each video object is encoded independently of the other. The multiresolution wavelet decomposition used in encoding texture information is shown to be helpful in providing spatial scalability. Our codec design is also shown to be temporally scalable. This report was accepted for oral presentation at the IEEE International Symposium on Circuits & Systems, Monterey, Calif., May-June 1998.