Maximum Likelihood Estimation and Computation in a Random Effect Factor Model
Maximum Likelihood Estimation and Computation in a Random Effect Factor Model
Files
Publication or External Link
Date
2004-08-04
Authors
Cheng, Yang
Advisor
Slud, Eric V
Citation
DRUM DOI
Abstract
We first briefly review some multivariate statistical models
such as Principal Component Analysis (PCA), Factor Analysis (FA),
and Probabilistic PCA (PPCA). Alternatively, we approach PCA from
the least-squares point of view. We introduce a Random Effect
Factor Model I (REFM_1), which expresses the observed vectors up
to random errors as a linear combination of a relatively small
number of axis directions in a new coordinate system with random
effect coefficients. Then, we characterize the maximum likelihood
estimators (MLE) under REFM_1 by a profile likelihood method,
that is, by maximizing the likelihood over mean and variance
parameters first with the coordinate direction parameters
component fixed, we have a restricted MLE in terms of the factor
directions, and substituting the restricted estimates into the likelihood,
finally maximizing the profile likelihood over the factor
directions. We show that the maximizer of the profile likelihood
function over the factor directions combined with the restricted MLE
for other parameters when the factor directions are fixed is the
joint MLE of the likelihood function. Some asymptotic properties
of the MLE such as consistency and asymptotic normal distribution
are established.
In order to analyze the multivariate data from s groups (s > 1),
we briefly review the Common Principal Components (CPC) model.
Other Random Effect Factor Models are introduced. The model
REFM_2 assumes all s groups have a common factor space but
differing mean and variance parameters for factor loadings and
error terms, and REFM_3 is a new model which has not only a
common factor space but also an additional individual space
belonging to each group only. We discuss the identifiability of
parameters, and again use the profile likelihood method to
find the MLE.
We develop an EM algorithm to compute the MLE for REFM_1, and indicate extension
s of the algorithm to REFM_2 and REFM_3. The performance of the algorithm on sim
ulated data is described. Quasi-Newton methods are also used to calculate the ML
E of the profile likelihood and they yield the same results as the EM algorithm
. Finally, we apply the EM algorithm for REFM_1 estimation to a real data set on
ultrasound cross-sectional images of the tongue during speech.