Dialsmith

This series of articles is part of a broader program, developed and sponsored by Dialsmith, centered on exposing the challenges around recall- and memory-bias in market research. You can continue to track news and updates from the program or participate in discussions by following on Twitter at @Dialsmith and #ExposingRecallMRx and by visiting the program web page at http://www.dialsmith.com/exposing-recall-mrx.

PART V | “MEMORABLE” HIGHLIGHTS FROM OUR IIeX PANEL

Authored by David Paull, Founder & CEO, Dialsmith

This is the fifth and final article in our series examining flawed recall and memory bias; how it impacts market research; and what we can do about it. In this final installment, I recap the live panel session I facilitated with our team of research experts at the recent IIeX North America conference. During the live session, the team dove deeper into some previously covered topics while also exploring controversial new ground, including how market researchers can potentially flip the script on memory manipulation by leveraging it as a marketing technique and the ethical consequences of doing so. We won’t cover the full session here but here are the highlights. The video of the full panel session can be viewed at the bottom of this post.

It’s one of our experts, Andrew Jeavons, who’s to blame for why we are here, given it was the article he authored from back in his Survey Analytics days that spurred the idea of putting this group together. That being the case, we kicked off the panel session by asking Andrew why this was such a “hot button” topic for him.

Jeavons responded, “My background is in psychology and after reading an article by Dr. Jonathan Lerner about memory work he had done with PTSD, it got me thinking that my own research was making a lot of assumptions about the way people’s memory works, which simply were not true…  Everybody knows we’re bad at remembering things—we’re bad at remembering names; we’re bad at remembering what happened in the past; we’re bad at remembering our emotions. And then it hit me that we’re basing our whole industry on some fairly shaky assumptions about human memory and how human memory works. And that was really the genesis of my article and evidently why I’ve landed up here on the stage today.”

I asked Elizabeth Merrick, our resident corporate researcher on the panel, for some real-world evidence of how customers’ inability to accurately recall experiences has impacted her research.    

Merrick responded, “Here’s one example. At a prior company, when sending out surveys, we were consistently seeing a 13 – 20 percent misattribution rate due in large part to recall problems. As result, you get this chaos in your data and have to wonder what you can trust. Because if I’m making future decisions based on the data that’s coming in to me—future decisions that could determine the allocation of millions of dollars in marketing budget—then I need to feel confident that this is not misdirected information. All data is not necessarily good data, and I’d be extremely remiss in my role, as someone who is helping build a business based on insights, to use information that I know how such a deep problem with it.”

I asked our academic researcher and memory manipulation expert, Dr. Elizabeth Loftus, if there were lessons learned from her decades of memory research that could be helpful to those of us in the market research community.

Loftus responded, “For many decades, I have been deliberately distorting people’s memories, making them believe (for example) that a certain car went thru a yield sign instead of a stop sign, or that the bad guy running from the scene of the crime had curly hair instead of straight hair. It’s surprisingly easy to expose people to leading questions, or to misinformation, or to erroneous versions by other witnesses, and find that this contaminates the memory of the individual you are studying. In my more recent research, we’ve shown just how far you can go with people. You can plant entirely false memories into the minds of people for things that didn’t happen and it effects peoples’ later thoughts, intentions, and behaviors. The takeaway here for those relying on memory is just how extremely malleable and susceptible it is to outside influences.”

Dr. Loftus’ response brought up an interesting and unexplored topic around the ethics of deliberately planting and/or manipulating memories and if it could or should be used as a marketing technique to impact consumer behavior. So, I asked Dr. Loftus to weigh in on the ethics issue.

Loftus responded, “Well, I’ve thought about the ethics of (intentionally manipulating memory) quite a bit and there’s no easy answer. I mean we’ve planted false memories that when someone was a kid, she got sick eating a particular food—like a hardboiled egg or strawberry ice cream—and the result was that she no longer wanted to eat those foods as much. We planted a warm, fuzzy childhood memory about a healthy food—in this case, asparagus—then our people wanted to eat more asparagus. And so, we can actually affect people’s nutrition and the types of foods they prefer eating—all through planted memories. So, you can see the potential positive implications of that. But should we deliberately plant memories in the minds of people so they can live healthier or happier lives or should we be banning the use of these techniques? I think that’s a personal decision.”

Merrick then added, “I think there’s a spectrum there: on one end, it’s scary to think about the manipulation that could happen. But on the other end, it just sounds a lot like advertising. You find this nugget of truth or this common feeling we all have, and leverage it. Going further down that path, I wonder if there are applications of this for post-purchase experiences. So, we see that something maybe went awry during your purchase or your journey with us as you were experiencing a particular brand. But what about the moments that did resonate or moments that were more important that could be brought up to the top? So, that, that’s what you remember instead. So, for example, when you remember a particular hotel or hotel brand, you think about the wonderful smells of the lobby and the friendly greeting you received at the reception desk and not the crinkled sheets. When we think about the entire journey with our customers, are there moments later on where we could use these ‘memory tactics’ to actually create stronger brands? So, rather than just trying to mitigate memory bias in our research, can we actually use it to our advantage to offset issues with our brands? Does this become part of what we are able to deliver to clients? It’s an interesting thought.”

Loftus again, “I noticed that you started to use the word “manipulation” but then backed off because there’s something that feels a little creepy about it. But who’s to say we should stop a parent from manipulating the memories of an overweight teenager or child in order to make them not like a fattening food or make them eat more of the healthy foods. My enemies (and yes, I have enemies in my line of work) will say, “There she goes again advocating for parents to lie to their children.” To which I would say, “Hello, Santa Claus?” Parents justify lying to their children, so what’s the lesser of the two evils? Having an obese child with health issues and a shortened lifespan or having a child experience a false memory? I know which side I’d come in on but maybe that’s because I already know how much fiction already exists in our memories.”

Next, I asked Andrew Jeavons to go first in offering up ways in which we can mitigate the impact of flawed recall and memory bias.

Jeavons responded, “It sounds very simple minded but it’s actually really important. You’ve got to think when you’re asking about semantic knowledge in a survey or if you’re asking about episodic memory and if you are asking about episodic memory, are you optimizing or making it easy for people to recall this information.”

Loftus followed up, “One thing that may seem kind of obvious is to at least get somebody’s recollection as soon as possible. Or if you could even get their thoughts and feelings earlier as they are happening that would be ideal. But getting a recollection as soon as possible before they have an opportunity to interact with other people I mean is one thing we’ve learned. Because if people are trying to recall things in a group, then the various responses are going to influence each other. I don’t care if this group is a focus group or group therapy or jury deliberation. The early responses are going to affect what people say later on. So, one thing we do in the research is try to get an early opinion before people have a chance to engage in activities that can contaminate their memory.”

Merrick added, “We’ve been talking about memory biases for our respondents, but we, as researchers, are also very prone to memory biases. I’m sure everybody in this room at some point has said, ‘Oh, well I remember this so let’s put this in.’ Especially when you’re working with qualitative data. There’ve been many times when I’ve worked with moderators and I say what’s your process? How are you going through this? Well, you know I’ve been really trying to take good notes along the way but a lot of it is stored in my mind…. And then I just do a data dump. This is problematic, and so there’s a huge opportunity in qual research to apply an impartial tool or impartial technique that can mitigate our biases too. If we really want our data to be good, we need to have better text analytics out there that are just constantly pulling in what people are saying and actually doing synthesis on that. The data is available. The processes are still getting better. But I’m not sure there is a cost affordable option right now for a lot of what I’m describing. It’s been cool to be on these calls and to hear about how close we’re getting. And I would imagine in the next few years, it’s going to be absolutely required that anytime you do something that is qualitative in nature that the analysis is not totally reliant on humans. I would encourage everybody in this room to not only think of the biases of your respondents but also think of your own.”

And a couple of questions from the audience…

You’ve been talking about memory in so far as what people know. But my challenge to you is… does it always matter? Because isn’t it in many cases equally, or more importantly, how they feel? And how someone feels may not be based on a specific thing that happened in the past but there can be a number of contributing reasons that contributes to a feeling?

Loftus responded, “Well, the thing about feelings, or emotions, is that they, like memories, are also malleable. Not only can we plant memory distortions or plant whole memories, but we can alter the feelings that people think they had about a product. In fact, some of our experiments are doing exactly that, right now. We’ve run tests where people have described feeling anxious to the level of a 2 on a 7-point scale, and later on, we’ve gone back to them and said that they told us they were a 4 on a 7-point scale. And the majority of people tested did not detect that we manipulated their feeling rating and consequently, accept it as their new feeling. So, I think you have a point that feelings are important, but they are malleable as well. 

In marketing/advertising research, there are a number of different techniques that suppliers use to overcome these (memory) issues but it feels like it’s all over the map. If we’re talking about a 13 – 20 % misattribution rate on a coupon (as in Merrick’s example) then how am I supposed to be able to validate results to the point where I can make key decisions about my ads?

Merrick responded, “Yes, I struggle with this too because there are not necessarily better alternatives out there. If you were looking at a very specific campaign, TV ads exclusively, for example, then there are some nice, effective technologies that you can apply. But in today’s reality, you have a media mix. So, you might have out of home, and print; you might have something that’s very geo-specific, and then you might have some national ads. And all of that becomes really complicated when you try to do attribution, and even more so, when you’re relying on recall. It makes me wonder if we have to pull back and consider changing the metrics that we really need to care about. Turn our attention from awareness and look more at the bottom of the funnel metrics, such as conversion, which is much less prone to recall bias.   

And that’s a wrap from our IIeX panel session but our program. You can track news and updates from our program, participate in discussions, and find links to resources on this topic by following us on Twitter or by visiting and bookmarking our program web page.

Here’s the video of the full panel session from IIeX.

REVIEW OF ALL THE ARTICLES IN THIS SERIES:

PART I | WHO ARE WE AND WHY ARE WE DOING THIS?

PART II | HERE’S THE PROBLEM WITH RECALL BASED RESEARCH

PART III | THE RIPPLE EFFECT

PART IV | CAN WE FIX THIS?

PART V | “MEMORABLE” HIGHLIGHTS FROM OUR IIeX PANEL