Design Meets Research

True story. At Sterling Brands in New York, we have a wonderful cleaning woman named Marta who comes to the office every night around seven o'clock to clean the place up. One evening not so long ago, we had a client review that ran rather late into the night. We were all gathered around the table in one of our conference rooms, where we had narrowed a wide range of package design options down to what we all considered the top three. We sat there with furrowed brows as we pondered the three finalists and attempted to make a democratic decision on the favorite. Suddenly a light bulb lit up over our client's head. With bright eyes and a sudden burst of enthusiasm, he jumped and said, “I know what to do! Let's get Marta in here and see what she thinks!”

Most designers have been in this type of situation—whether our client wants to get the opinion of a lovely cleaning woman, a dogwalker, a mother-in-law or an executive assistant. While creative directors might groan whenever this happens, what our client is really trying to do is assess some attitudinal distinction of the design via any consumer—whether that person is the logical target market of that particular product or not.

A consumer drawing from market research representing Target.

What our clients are seeking in today's incredibly competitive marketplace is some sense of safety—an insurance policy of sorts—a “gee, if Marta likes this design, it must be good” mentality to create a sense of confidence that the direction being taken with a new design is indeed a correct and meaningful choice.

But unfortunately, it isn't quite as easy as this. Back in the late 1800s, automobile maker Henry Ford remarked that if you'd questioned consumers about what they hoped for next in transportation, they would have said that what they wanted was a faster and a stronger horse. This was primarily due to the limits of our imagination as a population at the time. Culturally speaking, people are generally stymied by self-determined limits to create and invent. As a result, prior to the invention of the car, the mass pre-conceived boundaries of our collective imagination were constrained by notions of what seemed possible, tangible and even logical. The idea of an “automobile” was as foreign and mysterious a comprehension then as time travel might seem to be now.

Ford's commentary during the infancy of market research foreshadowed a sentiment about the discipline that is still very much an active one. Qualitative and quantitative market research often get a bad rap in the graphic design industry—and in the marketing world in general. Those that are vehemently against the practice argue that because consumers are generally uncomfortable with change, any type of research probing something truly innovative or revolutionary will likely scare people. Those that are skeptical will question the nature of behavioral dynamics involved in artificial group settings. Even those that are merely dubious will admit that research can stifle creativity.

A consumer drawing representing Starbucks.

Questioning consumers about their lives and choices first began back in 1790 with the first United States census, which was initiated primarily to determine voting demographics. It wasn't until 1920 when Procter & Gamble executives went door-to-door asking American housewives for their opinions on new products and packaging that the modern age of market research really began. But it was in 1930 that the discipline truly took shape when social scientists A.C. Nielsen and George Gallup started quantitative testing and surveying. In 1950 the first qualitative testing began by getting a very targeted audience of consumers with a similar set of needs or concerns to understand their motivation, purchase behavior and attitude development within a particular frame of reference. These pioneering market researchers used psychology, anthropology, creativity and sociology to study a newly curious topic: why people buy the stuff that they buy. It is not unfair to say that all modern advertising sprang from this groundbreaking work.

Cut to present day and we find that Malcolm Gladwell, author of runaway anthropological bestsellers Blink and The Tipping Point, has called for the abolition of focus groups. He contends that because human beings cannot rationalize initial impressions it is both misleading and dangerous to ask people to explain what they like and why they like it. Documented examples of this are far and wide, but the most convincing are the market research studies conducted prior to the launch of New Coke and Absolut Vodka. New Coke was considered an unequivocal slam dunk in taste tests against Classic Coke but was subsequently a failure of epic proportions when it went to market. Conversely, the Absolut Vodka bottle design was universally panned in focus groups. A courageous Michel Leroux, then the brand manager of the new vodka, pressed for its launch, and the brand set international sales records. The Absolut bottle is now considered a contemporary design icon. So when the demand for validating design solutions is so strong, yet research is seemingly so fraught with pitfalls, what is a designer to do?

There is a group of brand consultants and cultural anthropologists alike that believe now that it is not the actual research itself that is the problem. It is rather about how research is often misused, what type of design concepts and stimulus are tested, and how data is analyzed that is most often at fault. When used correctly, research shouldn't stifle creativity but rather offer designers stronger inspiration and focus.

The Golden Rules of Market Research

1. Focus on testing communication effectiveness vs. design appeal. Market research should be about perceptions, not preferences. And do not ask someone if a design piece will influence a purchase decision-no self-respecting person will admit that they are so superficial as to be influenced this way… though we all are!
2. When testing, make allowances for familiarity. We are generally more comfortable with what we know, and humans, as a species, tend to be frightened of change.
3. Market research is an art, not a science. Try to investigate emotional connections and design sensibilities. Avoid over-dependence on numerical imperatives.
4. Focus on what consumers like about the brand or product first. Then ask them to focus on what a design is communicating about the brand or product in general. Try to avoid asking consumers to explain why they like what they like in specific design terms.
5. More is definitely merrier. Do not test designs in isolation. Let consumers see designs alongside other designs or next to the competition. This makes it easier to respond through comparing and contrasting.
6. Never ask consumers how they would improve a design. They're not design experts and you want their reactions, not their solutions.

There are a wide variety of research techniques that can have merit for designers, but like any techniques, there are different tools, stimulus and surveying mechanisms that are appropriate for each technique. There is not, repeat not, one correct way to test design.

The following are some of the mainstays of modern market research. With each is included the advantages, the challenges and the bottom line. These days, when marketers are spending millions of dollars to facilitate a brand identity evolution or a new product introduction, getting Marta or their mother-in-law's seal of approval isn't enough. Nor is the “gut instinct” approach. As an old marketing professor of mine once warned me, “When selecting design, my instincts tell me one thing: not to go with my instincts.”

Ethnographic Research

Some of the best design research does not involve testing actual design at all. In fact, it is usually conducted before any conceptual work begins.

Ethnography is the branch of anthropology that provides scientific descriptions of individual human societies. Ethnographic research involves rigorous one-on-one conversations and observations with consumers in their everyday surroundings, be it at home, at work, shopping, in a bar and so forth. This approach helps marketers and designers understand consumers on a much deeper level than any other qualitative research technique. For a few hours (or days!) you get to see the world through the consumer's eyes; observe the context in which design operates in their lives and witness first-hand their current aesthetic sensibilities. Get an in-depth view of their home and their choices. Good ethnographers don't ask consumers to play a designer or brand expert, but to simply be themselves. This provides a complex and rich picture of creative sensibilities. Further, the insights gleaned are most valuable to developing a design that will emotionally connect and delight. Many marketers have embraced the ethnographic research technique both as a panacea to their frustrations with focus groups and because it gets much closer to consumer truths.

The Advantages of Ethnographic Research

• It brings you much closer to “reality.”
• It provides deeper profiling of consumers: lifestyles, brand relationships, design sensibilities, shopping dynamics.
• It unearths truths through observation as well as discussion.
• It is particularly good for sensitive topics.

The Challenges

• It is time consuming.
• You often get limited sample sizes.
• There is limited client involvement in the actual research.

The Bottom Line

• Ethnographic research gives you the ability to comprehend consumers on a deeper level, thus allowing for a better understanding of their imagination through design.

Focus Groups

Many marketers agree with Malcolm Gladwell that focus groups are dangerous, but this has less to do the forum itself and more because of how they are abused. Clients are often apt to say, “Let's let the focus groups tell us what consumers want.” Then a design strategy team is required to question a rogue set of amateur strangers to come to a consensus, in two hours, about a marketing and design strategy they know nothing about. It is fraught with difficulties. Put in another way, if the Absolut Vodka brand team had simply given consumers what they thought they wanted, the design icon would not have made it to market. Instead, the research findings taught them that the design they had was likely to disrupt and challenge the status quo—and that was in sync with the original strategy.

Focus groups are a most efficient forum in the effort to identify what design concepts are communicating on an intuitive level. But beware: this is where facilitators can sometimes force consumers to rationalize their responses or be the arbiter of what's right and wrong. Rather, their reactions should be used as one of the filters to help the designer and brand teams understand what designs will create the desired effect and delight the most.

The Advantages of Focus Groups

• They can give quick and controlled feedback.
• They are better used to explore broader conceptual themes than tight executions.
• Design can be utilized to stimulate emotions versus land at solutions.

The Challenges of Focus Groups

• The environment is usually unnatural (e.g., one-way mirrored room).
• Respondents often say what they think you want to hear, not what they really feel.
• The temptation is to treat consumers as if they were art directors and experts.
• The forum is often perceived as old-fashioned.

The Bottom Line

• Try to keep this stalwart technique fresh by experimenting with new techniques.

Quantitative Eye Tracking

Quantitative market research is the systematic attempt to define, measure and report on the relationships between various elements. In design it is used essentially to compare one design to another in a very specific environment. Eye tracking is simply the tracking of the pupil as it moves across an image. Eye tracking technology was pioneered over thirty years ago by Elliott Young at Perception Research Systems and is now the leading technology for measuring how humans “see” products. Eye tracking technology is highly effective at measuring the speed in which a design is seen in a competitive context and the way consumers navigate that design (i.e., what they look at first, what they pause at longer, what they go back to and study again).

It is also particularly good for testing impacts in packaging design. When testing a new design it is considered accurate as long as both the current and new design are being tested under exactly the same circumstances and with a significant number of consumers (at least 100–150 of any one target demographic). Eye tracking is also a useful measure of how a final design is going to stand out in a retail environment. The danger lies in the obvious: it becomes easy to evaluate these results in isolation and not consider the total context in which the design may be launched. For example, will the brand or product be supported by promotions, advertising, point-of-purchase? It is important to consider the results of this type of testing as one of the filters for decision making rather than the final arbiter.

The Advantages of Quantitative Eye Tracking

• Marketers often believe that which is not measured is not fully valued!
• Eye tracking is significantly more sophisticated than any other type of quantitative research in terms of:
  • Noting: who sees what
  • Speed of noting: how quickly they see what
  • Reexamination: who returns to look at something again
   
• You get multi-dimensional measurement: aesthetic appeal, product expectations, imagery, shelf impact, purchase intent, etc.
• It yields numerical and projectable ratings.
• It is an excellent test of impact and alienation.

The Challenges

• It is sometimes used as a “go/no-go” decision maker versus as a diagnostic tool to guide a final decision, and this can be shortsighted.
• Quant testing cannot realistically project sales impact.

The Bottom Line

• Educate your researchers, as design is small part of their business.
• Use Quant testing to inform, not dictate.

Online Testing

Online testing provides feedback from a large group of people relatively quickly and cost-effectively. But like all research techniques, it has drawbacks. There is considerably less control over how respondents can react to the aesthetic quality of what they are seeing and there is limited ability for real-time dialogue with consumers. There is also a dependency on respondents to be honest about whom they “say” they are (watch out, you might have 18-year-old male miscreants logging on to your test for menopausal hormone replacements!). But for a quick read on design, it will provide volumes of information fast.

The Advantages of Online Testing

• It casts wide net.
• It is time and cost efficient as well as flexible.
• It is kind of “cool.”

The Challenges

• There is often a misconception about speed—it is not really that fast.
• There is a limit to how long respondents will stay involved online.
• There is no way to ensure the purity of polling sample.
• There are inherent issues with quality control; design subtleties can be lost on screen.

The Bottom Line

• This is a mode of testing that has yet to be truly embraced by the design community (rightly so).

There's almost nothing more demoralizing for a designer than witnessing a bunch of non-designer consumers ripping apart their hard work. But if you are able to step back for a moment and watch the spontaneous enthusiasm and delight that great design can elicit in a consumer, research can ignite passion in a truly profound and deeply resonating way.

Ultimately, it is important to remember this: market research does not determine good design. Designers must design the work before it goes to research. And in today's risk adverse corporate world, it is unlikely that market research will be going away anytime soon. But take comfort in the quantifiable fact that when used correctly, the insight it provides can be an amazing springboard for creativity.