Adaptive behavior requires that organisms choose wisely to gain rewards and avoid punishment. Reinforcement learning refers to the behavioral process of learning about the value of choices, based on previous choice outcomes. From an algorithmic point of view, rewards and punishments exist on opposite sides of a single value axis. However, simple distinctions between rewards and punishments and their theoretical expression on a single value axis hide considerable psychological complexities that underlie appetitive and aversive reinforcement learning. A broad set of neural circuits, including the amygdala and frontal-striatal systems, have been implicated in mediating learning from gains and losses. The ventral striatum (VS) and amygdala have been implicated in several aspects of this process. To examine the role of the VS and amygdala in learning from gains and losses, we compared the performance of macaque monkeys with VS lesions, with amygdala lesions, and un-operated controls on a series of reinforcement learning tasks. In these tasks monkeys gained or lost tokens, which were periodically cashed out for juice, as outcomes for choices. We found that monkeys with VS lesions had a deficit in learning to choose between cues that differed in reward magnitude. Monkeys with VS lesions performed as well as controls when choices involved a potential loss. In contrast, we found that monkeys with amygdala lesions performed as well as controls across all conditions. Further analysis revealed that the deficits we found in monkeys with VS lesions resulted from a reduction in motivation, rather than the monkeys’ inability to learn the stimulus-outcome contingency.