We consider a discrete-time linear system with correlated Gaussian plant and observation noises and non-Gaussian initial condition independent of the plant and observation noises. We firstly find a solution of the filtering problem; we find a representation for the conditional distribution of the state at time t given the observations up to time t - 1. This representation is in terms of a finite collection of easily- computable statistics. With this solution to the filtering problem, we then find representations for the MMSE and LLSE estimates of the state given the previous observations, and the mean-square error between the two. (Of course the MMSE estimate will in general be a nonlinear function of the observations, whereas the LLSE estimate is by definition linear and is given by the Kalman filtering equations.) We then consider the asymptotic behavior of the mean-square error between the MMSE and LLSE estimates as time tends to infinity. We find conditions on the system dynamics under which the effects of the initial condition die out; under these conditions the non-Gaussian nature of the initial condition becomes unimportant as t becomes large. The practical value of this result is clear - under these conditions, the LLSE estimate, which is usally less costly to generate than the MMSE estimate, is asymptotically as good as the MMSE estimate (i.e., asymptotically optimal) in the mean-square sense.