Cryptococcus neoformans (C. neoformans) is a pathogenic fungus with a global distribution. Humans become infected by inhaling the fungus from the environment, and the fungus initially colonizes the lungs. If the immune system fails to contain C. neoformans in the lungs, the fungus can disseminate to the blood and invade the central nervous system, resulting in fatal meningoencephalitis particularly in immunocompromised individuals including HIV/AIDS patients. Following brain invasion, C. neoformans will encounter host defenses involving resident as well as recruited immune cells in the brain. To overcome host defenses, C. neoformans possesses multiple virulence factors capable of modulating immune responses. The outcome of the interactions between the host and C. neoformans will determine the disease progression. In this review, we describe the current understanding of how C. neoformans migrates to the brain across the blood–brain barrier, and how the host immune system responds to the invading organism in the brain. We will also discuss the virulence factors that C. neoformans uses to modulate host immune responses.