Nonlinear magnetization dynamic process in nano-scale magnetic systems is of great scientific interests for its application to magnetic recording technology and spintronic devices. In the dynamic process, thermal fluctuation effects are of critical importance since they are directly related to long term reliability of magnetic devices. Recently, a novel approach to modeling stochastic magnetization dynamics has been proposed[1,2,3]. In this approach, thermal bath effects are accounted for by introducing a jump-noise torque term into the precessional magnetization dynamics equation. In this dissertation, we develop a Monte Carlo type numerical technique for implementation of the approach. There are two central elements of our numerical technique: a midpoint'' finite-difference scheme for integration of deterministic precessions and a self-scattering'' scheme which results in time-homogenization of a jump-noise process. The numerical technique unconditionally preserves the micromagnetic constraint and appreciably simplifies the random component of Monte Carlo simulations. We perform and illustrate numerous Monte Carlo simulations in the dissertation using numerical examples. The Monte Carlo simulations are ideally suited for implementation on GPUs since they are intrinsically parallelizable in the sense that different realizations of stochastic magnetization dynamics can be computed concurrently. Therefore we develop a parallel algorithm and implement it using an Nvidia GPU card. A speed-up factor of more than 200 is achieved using this GPU implementation in comparison with the tranditional CPU single threaded implementation. Furthermore, we apply the jump-noise process driven magnetization dynamic equation to study random magnetization switching induced by thermal fluctuations. Numerical results demonstrate that the magnetization switching rate has a very different temperature dependence at relatively high and very low temperatures. The high temperature switching conforms to the Arrhenius law of thermal activation, whereas the low temperature switching has many features traditionally attributed to the phenomenon of macroscopic magnetization tunneling. The two temperature dependent regimes emerge directly from the properties of a jump-noise process while no quantum considerations are involved in our approach. Finally, we study the magnetization dynamics at elevated temperatures. We extend the jump-noise process driven magnetization dynamics approach and derive a generalization of the classical Landau-Lifshitz equation to describe magnetization dynamics around Curie temperature where the traditional micromagnetic constraint is not valid. The longitudinal and transverse damping terms in the generalized equation emerge directly from the mathematical structure of a jump-noise process which accounts for interactions with thermal bath.