To increase traffic mobility and safety, several types of active traffic management (ATM) strategies, such as variable speed limit (VSL) and hard shoulder running (HSR), are implemented in many countries. While all kinds of ATM strategies show promise in releasing traffic congestion, many studies indicate that stand-alone strategies have very limited capability. This paper proposes an integrated VSL and HSR control strategy based on a reinforcement learning (RL) technique, Q-learning (QL). The proposed strategy bridges a direct connection between the traffic flow data and the ATM control strategies via intensive self-learning processes thus reduces the need for human knowledge. A typical congested interstate highway, I-270 in Maryland, U.S. is simulated using a dynamic traffic assignment (DTA) model to evaluate the proposed strategy. Simulation results indicated that the integrated strategy outperforms the stand-alone strategies and traditional feedback-based VSL strategy in mitigating congestions and reducing travel time on the freeway corridor.