Introduction Several studies suggest that physical activity may improve health-related quality of life. Other studies have shown that participation in physical activity differs among women of different racial/ethnic groups. This study aimed to determine whether the association between physical activity and health-related quality of life differs among women aged 40 to 64 years from different racial/ethnic groups. Methods We explored the association between physical activity level and health-related quality of life with descriptive statistics and multiple regression analyses adjusting for potential confounders among 11,887 women aged 40 to 64 years who identified themselves as Latinas, Asians, African Americans, or whites in the 2005 California Health Interview Survey. Results Although white women reported more regular physical activity than women of other racial/ethnic groups, Asian women reported fewer mentally and overall unhealthy days than women of other groups. Nonetheless, as physical activity increased, health-related quality of life improved only among white women (fewer physically unhealthy, mentally unhealthy, recent activity limitation, and overall unhealthy days) and among Latinas (fewer overall unhealthy days). Conclusion Future studies should try to confirm if and clarify why the association between physical activity level and health-related quality of life differs among these middle-aged women of different races/ethnicities. If confirmed, this association would imply that health care professionals and those who design public health interventions may need to vary their promotion methods and messages to encourage physical activity among women of different races/ethnicities.