Over the past century, multiple tests measuring musical ability have been developed, and research has been investigating individual differences in musical ability to answer questions about the components of musical ability and their dissociations in amusic patients, the innate vs. acquired nature of musical skill, and the potential transfer from musical training to other abilities. However, there has been little consensus on what exactly constitutes musical ability and how to best measure this construct. Previous research has used a variety of tasks assessing mainly perceptual skills (e.g., same/different judgments in sequentially presented melodies), and outcomes from these tasks range from single indices (e.g., pitch ability) to composite scores from multiple tasks (e.g., pitch, rhythm, loudness, timbre). The current study uses individual differences data from 15 representative musical ability tasks (including perception and production measures) to assess the unity and diversity of musical ability, and uses the resulting comprehensive latent measure of musical ability to evaluate previously theorized links between musical ability and individual differences in musical experience, working memory, intelligence, personality factors, and socio-economic status. Results from latent variable model comparisons suggest that musical ability is best represented by related but separate pitch, timing, perception, and production factors. Consistent with previous research, a latent measure of musical ability was positively related to musical training, working memory, and intelligence; in contrast, musical ability was not related to openness to experience or socio-economic status.