Dialsmith

In case you haven’t heard, Qualtrics and Dialsmith announced a partnership that gives Qualtrics customers the ability to add Dialsmith’s online dial testing to their surveys. Qualtrics is offering the service through the Qualtrics Innovation Exchange Marketplace.

This is big for Qualtrics users who want to test and evaluate different types of media—from ads and animatics to sales pitches to full pilot episodes and everything in-between. To get a better idea of what this partnership means and the benefits it brings to Qualtrics users, we chatted with the two folks who made this all happen: Aaron Korach, Dialsmith manager of online products and services and Tucker Stockman, Qualtrics manager of global strategic partnerships and alliances.

First up, Tucker…

Give us a little background on the Qualtrics Innovation Exchange. What it’s purpose?

(Tucker) The Qualtrics Innovation Exchange was designed to give our clients access to expert content, professional services, and turnkey solutions. Everything you need to gather insights—all in one place.

Why the interest in adding Dialsmith as a partner?

(Tucker) We look for partners that complement our services and provide real value to the Qualtrics community. Dialsmith checks off those boxes, and brings a pedigree of delivering top shelf service that aligns well with the Qualtrics experience our customers are accustom to.

What’s the initial response you’re getting from Qualtrics customers and/or your sales team about adding Dialsmith to the IE?

(Tucker) I’ve already gotten quite a lot of excitement from our offices in the U.S. and EMEA regarding our partnership. In-the-moment dial testing was a natural fit with our clients based on the nature of the Qualtrics platform and we are excited to start supporting our mutual customers with this new integration.

And here’s round two with Aaron…

Why would someone consider adding dial testing to their Qualtrics survey?

(Aaron) Anyone looking to test and evaluate recorded media (advertisements, programming content, speeches, sales pitches, etc.) should strongly consider adding dial testing to their Qualtrics survey. Dialsmith dial testing adds a rich data layer to Qualtrics surveys with its unique ability to gather continuous, moment-to-moment feedback and display real-time second-by-second results. The results allow you to pinpoint “moments of truth” and zero in on which specific moments or segments are or aren’t hitting the mark.

What forms of media can you test and what media works best?

(Aaron) Dialsmith’s online dial testing supports the testing of any recorded video or audio from 30 seconds up to 60 minutes and everything in between. The types of media you can test include: advertisements, TV pilots, movie/TV trailers, radio programming (song evaluation and show content), speeches, debates, animatics, detail aids, and sales presentations and pitches. While media length can certainly vary, experience has taught us that less than 20 minutes is the ideal length. So when possible, we suggest cutting longer clips into multiple shorter ones to help keep respondents attentive and focused.

What factors should someone think about when considering adding dial testing to their survey?

(Aaron) As with all online research activities, it’s vital to set clear expectations for the respondents.  Detailed instructions along with clear expectation on estimated time to complete should be communicated at all stages of respondent interaction: from the invite to all in-survey instructions.

Ensure you have adequate time to test the survey setup and data structure.  Dialsmith and Qualtrics will result in multiple data tables that can be aggregated and combined, if the data structure (data map) is set up correctly.

Carefully consider the rating scale to be used to rate your media. Common scales are: “Positive to Negative,” “Like to Dislike,” and “Believable to Unbelievable.” Ensure that your scale focuses on your primary research objective.  If needed, you can split your sample to test the media using different scales. We typically recommend using a 0-100 numeric range scale.

When testing long-form content (anything longer than 5 minutes) consider breaking up the content into multiple clips. Although we test content up to 60 minutes (and in some cases even longer), as rule of thumb, the longer the content, the more challenging it is to keep respondents focused and engaged. As we are asking respondents to provide continuous feedback, keeping them engaged is vital to this type of online research. Many of our clients test full-length episodes or speeches but break them up into shorter clips with instructional text or follow up questions in-between.

If you’re testing radio or other audio-only content, you’ll want to consider the visual image you want to display as the audio plays. This can be a generic image (such as a radio or a speaker image) or can be used to remind respondents of the rating instructions.

How do results from online dial testing compare to those from in-person dial sessions?

(Aaron) In side-by-side testing, we’ve found that the overall sentiment is the same for online and in-person dial testing. In-person respondents tend to dial using a wider range whereas online respondents tend to move the slider a bit less to the extremes, but the overall sentiment and shape of the moment-to-moment lines are very consistent between online and in-person.

Thanks to Tucker and Aaron for their time. For more details on dial testing in Qualtrics surveys, head to the Qualtrics Innovation Exchange.