Rank-heterogeneous effects of political messages: Evidence from randomized survey experiments testing 59 video treatments


Central to theories of political persuasion is treatment effect heterogeneity—the idea that people respond to political messages in different ways—so persuasion is easier when different messages are targeted to different audiences. The standard approach to testing for heterogeneity is to examine whether the effect of an individual message differs between subgroups of people (such as liberals versus conservatives). We describe the shortcomings of this approach, and propose an alternative: jointly examining many messages on the same political issue, and assessing whether the rank-order of their effects differs between subgroups (which we call “rank-heterogeneity”). Implementing this approach, we conduct two large-scale survey experiments spanning two policy issues, 59 message treatments, and over 40,000 American adults. Across experiments we find mixed evidence of rank-heterogeneity, suggesting that it depends upon the particular issue in question. However, in the case where we do observe strong evidence of rank-heterogeneity, its primary cause is consistent with the predictions of moral reframing theory, an influential account of heterogeneity in political persuasion. Alongside these implications for theory, our results have implications for political persuasion in practice.