The effects of so-called “psychedelic” or “hallucinogenic” substances are known for their strong conditionality on context. While the so-called culturalist approach to the study of hallucinations has won the favor of anthropologists, the vectors by which the features of visual and auditory imagery are structured by social context have been so far little explored. Using ethnographic data collected in a shamanic center of the Peruvian Amazon and an anthropological approach dialoguing with phenomenology and recent models of social cognition of Bayesian inspiration, I aim to shed light on the nature of these dynamics through an approach I call the “socialization of hallucinations.” Distinguishing two levels of socialization of hallucinations, I argue that cultural background and social interactions organize the relationship not only to the hallucinogenic experience, but also to its very phenomenological content. I account for the underpinnings of the socialization of hallucinations proposing such candidate factors as the education of attention, the categorization of perceptions, and the shaping of emotions and expectations. Considering psychedelic experiences in the light of their noetic properties and cognitive penetrability debates, I show that they are powerful vectors of cultural transmission. I question the ethical stakes of this claim, at a time when the use of psychedelics is becoming increasingly popular in the global North. I finally emphasize the importance of better understanding the extrapharmacological factors of the psychedelic experience and its subjective implications, and sketch out the basis for an interdisciplinary methodology in order to do so.