Maximum Likelihood Estimation and Computation in a Random Effect Factor Model

Abstract

We first briefly review some multivariate statistical models such as Principal Component Analysis (PCA), Factor Analysis (FA), and Probabilistic PCA (PPCA). Alternatively, we approach PCA from the least-squares point of view. We introduce a Random Effect Factor Model I (REFM_1), which expresses the observed vectors up to random errors as a linear combination of a relatively small number of axis directions in a new coordinate system with random effect coefficients. Then, we characterize the maximum likelihood estimators (MLE) under REFM_1 by a profile likelihood method, that is, by maximizing the likelihood over mean and variance parameters first with the coordinate direction parameters component fixed, we have a restricted MLE in terms of the factor directions, and substituting the restricted estimates into the likelihood, finally maximizing the profile likelihood over the factor directions. We show that the maximizer of the profile likelihood function over the factor directions combined with the restricted MLE for other parameters when the factor directions are fixed is the joint MLE of the likelihood function. Some asymptotic properties of the MLE such as consistency and asymptotic normal distribution are established. In order to analyze the multivariate data from s groups (s > 1), we briefly review the Common Principal Components (CPC) model. Other Random Effect Factor Models are introduced. The model REFM_2 assumes all s groups have a common factor space but differing mean and variance parameters for factor loadings and error terms, and REFM_3 is a new model which has not only a common factor space but also an additional individual space belonging to each group only. We discuss the identifiability of parameters, and again use the profile likelihood method to find the MLE. We develop an EM algorithm to compute the MLE for REFM_1, and indicate extension s of the algorithm to REFM_2 and REFM_3. The performance of the algorithm on sim ulated data is described. Quasi-Newton methods are also used to calculate the ML E of the profile likelihood and they yield the same results as the EM algorithm . Finally, we apply the EM algorithm for REFM_1 estimation to a real data set on ultrasound cross-sectional images of the tongue during speech.