We obtain results similar to those for LQG problems on the control system structure for optimal linear quadratic regulator problems with Poisson noise disturbances. If the coefficient matrices of the system dynamics and the performance index are constant, the optimal control of the finite time problem converges to the time-invariant control of the infinite time problem quasi-uniformly, almost surely. Both the long term average cost criterion and the discounted cost criterion are investigated for infinite time problems.