Autonomous Drone Racing (ADR) is an annual competition, organized at the International Conference on Intelligent Robots and Systems (IROS), in which research groups all over the world participate to demonstrate the state-of-the-art technology in the autonomous aerial robotics field. This work describes the system development of the Autonomous Racing Drone System for the IROS ADR competition. A gate detection based, computationally light-weight visual-inertial localization (VIL) system is developed. We show that the proposed VIL system has a significantly lower memory usage than the state-of-the-art Monocular VIO systems which makes it suitable to run on resource constraint hardware. A non-linear model predictive control (NMPC) strategy is implemented for high-speed way-point navigation of the racing drone. We show that the NMPC strategy provides better trajectory tracking performance as compared with the traditional PD controller. The VIL system proposed in this work was utilized in the autonomous drone racing system which won the second-place in the IROS ADR 2019, Macau competition.