Cross-species transmission (CST) has led to many devastating epidemics, but is still a poorly understood phenomenon. HIV-1 and HIV-2 (human immunodeficiency virus 1 and 2), which have collectively caused over 35 million deaths, are the result of multiple CSTs from chimpanzees, gorillas, and sooty mangabeys. While the immediate history of HIV is known, there are over 45 lentiviruses that infect specific species of primates, and patterns of host switching are not well characterized. We thus took a phylogenetic approach to better understand the natural history of SIV recombination and CST. We modeled host species as a discrete character trait on the viral phylogeny and inferred historical host switches and the pairwise transmission rates between each pair of 24 primate hosts. We identify 14 novel, well-supported, ancient cross-species transmission events. We also find that lentiviral lineages vary widely in their ability to infect new host species: SIVcol (from colobus monkeys) is evolutionarily isolated, while SIVagms (from African green monkeys) frequently move between host subspecies. We also examine the origins of SIVcpz (the predecessor of HIV-1) in greater detail than previous studies, and find that there are still large portions of the genome with unknown origins. Observed patterns of CST are likely driven by a combination of ecological circumstance and innate immune factors.