The purpose of this paper is to develop a framework for designing controllers for finite state systems which are robust with respect to uncertainties. A deterministic model for uncertainties is introduced, leading to a dynamic game formulation of the robust control problem. This problem is solved using an appropriate information state. A risk-sensitive stochastic control problem is formulated and solved for Hidden Markov Models, corresponding to situations where the model for the uncertainties is stochastic. The two problems are related using small noise limits.