In this thesis two contributions are made to the area of mathematical finance. First, in order to explain the non-trivial skewness and kurtosis that is observed in the time series data of constant maturity swap (CMS) rates, we employ the pure jump Levy processes, i.e. in particular Variance Gamma process, to model the variation of unobservable economic factors. It is the first model to include Levy dynamics in the short rate modeling. Specifically, the Vasicek type of short rate framework is adopted, where the short rate is an affine combination of three mean-reverting state variables. Zero-coupon bonds and a few fixed income derivatives are developed under the model based on the transform method. It is expected that the Levy based short rate model would give more realistic explanations to the yield curve movements than Gaussian-based models. Second, the model parameters are estimated by the particle filter (PF) technique. The PF has not seen wide applications in the field of financial engineering, partly due to its stringent requirement on the computing capability. However, given cheap computing cost nowadays, the PF method is a flexible yet powerful tool in estimating state-space models with non-Gaussian dynamics, such as the Levy-based models. To customize the PF algorithm to our model, the continuous-time Levy short rate model is cast into the discrete format by first-order forward Euler approximation. The PF technique is used to retrieve values of the unobservable factors by sequentially using readily available market prices. The optimal set of model parameters are obtained by invoking the quasi-maximum likelihood estimation.