Dialsmith

We recently got word of an intriguing use of our Perception Analyzer dials for political research being done by Dr. Charlton McIlwain, associate professor of media, culture and communication at New York University, and his colleague Dr. Stephen Maynord Caliendo, professor of political science at North Central College. The two have been studying the impact of race-based messaging in political ads for more than a decade and recently presented their latest findings, including the dial testing results, at the Midwest Political Science Association Conference.

We chatted with Dr. McIlwain to learn more about what they were looking for from dial testing and the anticipated, and not-so-anticipated, insights they gained from the results.

Q: What’s your focus at NYU and what type of research are you involved with?

Dr. McIlwain: I’m an associate professor of media, culture and communication; I’ve been at NYU for 14 years. Most of that time I’ve worked with Dr. Stephen Maynord Caliendo, professor of political science at North Central College on The Project on Race in Political Communication. We’ve worked mostly on research that tries to understand how political candidates construct race-based appeals and advertisements and what effect those appeals have or potentially have on voters. We’ve been looking at how candidates use race language in their messaging, what that looks like on the screen, and how people interpret those messages. We analyze how voters respond to it and how it affects their perception of the candidate and their likelihood to vote for that candidate.

Q: You added dial testing to your most recent study. Can you give us some background on the study and what you were hoping to find out?

Dr. McIlwain: We wanted to replicate a study we had done in the past. We set up an experiment where we wanted to expose people to political ads that had either no race-based message, one that had an implicit race-based message or one that had an explicit race-based message. The implicit condition only used racial imagery in the ad, and the explicit condition had both racial imagery and language.

For this study, we chose ads that featured a white candidate discussing the difference between himself and his opposing African American candidate. The ads were not created as attack ads, but the candidate did focus on pointing out that the other candidate was “different.” Our goal was to see if there was any impact on participant’s view of the candidate based on which ads they viewed.

Snapshot of dial testing results from Dr. McIlwain’s race-based messaging study. Image courtesy of Dr. Charlton McIlwain and The Project on Race in Political Communication.

We tracked 113 participants’ reactions with the Perception Analyzer dials. We looked to see if their ratings changed when the race-based content was introduced in the ad to see if there was a connection there. In the past we were only able to gauge at the end how participants felt about a candidate, but we wanted to see how the message, images and text during the ad impacted that end judgment while participants watched and rated it in the moment.

Q: What results did you get from the dial testing and how were those results used?

Dr. McIlwain: The dial results clearly showed that there are a lot of complicated and sophisticated things going on over the course of an ad that have an impact on where people end up at the end. We were able to see the movement (of the dial result lines) at precise moments— the clear static lines and then the stark point where the ratings start to fall in very close proximity to the moment when the race-based content was introduced.

Our primary finding was that people reacted most strongly at the moment where race-based content was introduced in both the implicit and explicit conditions. We also had a surprising finding: participants reacted more negatively to the implicit condition than the explicit condition.

Q: Are there follow up projects or new studies in the works that you plan on using the dials for?

Dr. McIlwain: We saw this study as a pilot study that showed us that there is reason to do multiple additional studies with the dials. We presented our findings in April at the Midwest Political Science Association Conference, and they were well received. People are excited about the dial testing as a new methodology to measure people’s responses to race-based appeals.

Typically the way things have happened with experiments like this [without dials] is that we show them different stimuli and ask them similar questions at the end of the ad; what we know very strongly from this type of research is that people will either lie, or they misrepresent, or they don’t know what they are really thinking or feeling. People try to express their reactions based on what they think is socially acceptable. The dials help us get those participant reactions in real-time—revealing more accurate reflections of what our participants are thinking and feeling at that time. 

Thanks to Dr. McIlwain for giving us an inside look into his fascinating research and his research methodology. You can find more information about The Project on Race in Political Communication by visiting their website.