Ayahuasca is a plant concoction containing N,N-dimethyltryptamine (DMT) and certain β-carboline alkaloids from South America. Previous research in naturalistic settings has suggested that ingestion of ayahuasca can improve mental health and well-being; however, these studies were not placebo controlled and did not control for the possibility of expectation bias. This naturalistic observational study was designed to assess whether mental health changes were produced by ayahuasca or by set and setting. Assessments were made pre- and post-ayahuasca sessions in 30 experienced participants of ayahuasca retreats hosted in the Netherlands, Spain, and Germany. Participants consumed ayahuasca (N = 14) or placebo (N = 16). Analysis revealed a main effect of time on symptoms of depression, anxiety, and stress. Compared to baseline, symptoms reduced in both groups after the ceremony, independent of treatment. There was a main treatment × time interaction on implicit emotional empathy, indicating that ayahuasca increased emotional empathy to negative stimuli. The current findings suggest that improvements in mental health of participants of ayahuasca ceremonies can be driven by non-pharmacological factors that constitute a placebo response but also by pharmacological factors that are related to the use of ayahuasca. These findings stress the importance of placebo-controlled designs in psychedelic research and the need to further explore the contribution of non-pharmacological factors to the psychedelic experience.