Tag Archives: data

Making Fieldwork Easier and More Productive

Posted by

0

Advertisers, marketers and designers have long held the role of creating materials that reflect the lives of customers. Traditionally, this has relied on market research that is gathered in something of a vacuum, or reflects the beliefs and practices of the researcher more than the consumer.  People’s preferences all too often are neatly, if unimaginatively, packaged and handed off to a team tasked with creating new design applications, be the application a new product or a new brand platform.

Quantitative methods such as surveys demographics data provide wonderful snapshots of a large population but give little insight into what matters most to people and why it matters.  Usability tests and affinity diagrams provide information about the acceptability of new design concept and prototype, allowing designers to adapt and alter the message of a brand, campaign or product according to people’s stated preferences (which may or may not reflect their true beliefs).

From the qualitative side, focus groups and group interviews provide more qualitative feedback on product concepts,  messages and, to a lesser extent, explore unmet needs. The problem is that focus groups often reflect exaggerated responses and how important it is for humans to feel clever in front of perfect strangers. Additionally, these methods rely on people’s awareness and descriptive ability away from the context in which they would normally be thinking about a topic. In other words, they make things up, usually subconsciously but sometimes intentionally, in order to give an answer to a question. The result for design is mediocrity at its best and radically failed messaging at its worst.

Direct observation combined with interviewing (ethnography and ethnography-lite) is perhaps a more compelling method of coming to understand what people say, think and do.  It has certainly become a fixture in many organizations in recent years. And from a design standpoint it gives both researchers and designers a richer understanding of the issues, practices, and peculiarities of shoppers and the consumer, providing a more complete picture to work from when developing a brand or campaign. The problem is that while the depth of information uncovered is rich and insightful, it often stops short of any real observational depth that can be crafted into something truly meaningful. Surface-level findings are just that – surface level. If fieldwork is to be genuinely inspiring it needs to dig deep.  And researchers need to begin recognizing that their work is a creative, interpretive process.  That means that we needn’t fixate on getting the “right” answer, but that we get an interpretation of data that provides a “creative” answer.  Doing that means rethinking how we conduct research.  Here are 5 tips to making observational research relevant.

Start a conversation. When entering into fieldwork researchers tend to immediately jump into asking questions. The problem is that the abruptness and intrusiveness of these questions often changes behavior, resulting in semi-meaningful answers. To prevent this, begin with conversation and observation. Yes, that means allowing yourself time to get to know your participants as people. Let questions emerge as activities unfold. The simple fact is that we frequently don’t know what we should really be looking for until we’ve had time to immerse ourselves in the surroundings.  Simply put, relax and take your time rather than buffeting people with questions.

Look for patterns. Behavior does not happen in a vacuum.  People are individual organisms, yes, but they work within a social and cultural framework.  That means that activities and statements are always part of a larger pattern of meaning and practice.  Don’t take statements at face value.  Always look for patterns and connections between what people say, believe, and do.

Record information in their terms. Record what the participant says in their own terms rather than paraphrasing. Word choices, inflection, cadence, and non-verbal cues carry meaning that is lost when we try to simplify.  Facial expressions and body language convey a great deal of information. Simplicity will come out of the analysis – don’t do it when you are gathering information. 

EVERYTHING is data. Seemingly unimportant details are often the pieces that are the most important.  Environment and context have a huge influence on what people say and do.  Therefore, it is crucial when gathering information to include as much as possible in the interpretive process.  It may seem overwhelming, but everything is potential data for the analytical and creative mill.

Relax and embrace a range of perspectives.  Research should not be a list of facts and observations if the goal is to generate insights and innovation.  Research is a creative and interpretive act, no matter how much we may try obscure that fact.  As such, research is most effective when a wide array of disciplines are engaged in fieldwork.  Turn off your “scientist” sign and include a range of perspectives both in data collection and in analysis.

Customer research is only as powerful as its outcome.  Generating volumes of consumer insights and observations means very little if those insights and observations can’t be readily translated into something tangible, be it a brand platform, an ad campaign, or a new product offering.   While fieldwork can and does yield powerful insights, it means little if we forget that we are in a creative field that works best when a wide range of skills and perspectives come together.  Both in the field and out.

Ethnographers vs. Moderators: Know What You Are Buying

Posted by

6

The other day I was speaking with someone about ethnography and was informed by the person in question that she too was a “moderator.” She, of course, practiced ethnography, such as it is, and informed me she had been “moderating ethnographies” for years.  Yes, it made my skin crawl. Not because someone was crossing disciplinary boundaries, but because the choice of words told that ethnography was indeed the last thing she practiced, but had no doubt sold her self-defined ethnographic prowess into many a company. And unfortunately, this is precisely what continues to water down and cheapen the methodology and its use in business settings.

Let me state that I am not a moderator, I am an ethnographer and an anthropologist. And while both moderators and ethnographers speak to people, they are hardly one and the same.  On the surface it no doubt seems like I’m splitting hairs, but this isn’t just a simple matter of differing opinions or semantic variation, it is at the heart of how practitioners execute their work and how they practice ethnography.

A moderator is defined as a presenter, or host.  A moderator is a person or organization responsible for running an event.  A moderator is a person given special powers to enforce the rules of a collective event, be it a focus group, a forum, a blog, etc.  Moderation is the process of eliminating or lessening extremes. It is used to ensure normality throughout the medium on which it is being conducted. In other words, moderators assume control and direct. They maintain power and tease out information that is essentially qualitative hypothesis testing. Understand, I have no problem with moderation and moderators, but the practice of moderation is anything but ethnographic.

Ethnography is a qualitative research method aimed to learn and understand cultural phenomena which reflect the knowledge and system of meanings guiding the life of a cultural group.  Data collection methods are meant to capture the social meanings and ordinary activities of people in naturally occurring settings that are commonly referred to as “the field.” The goal is to collect data in such a way that the researcher does not impose any of their own bias on the data. Multiple methods of data collection may be employed to facilitate a relationship that allows for a more personal and in-depth portrait of the informants and their community. These can include participant observation, field notes, interviews, and surveys.  In order to accomplish a neutral observation a great deal of reflexivity on the part of the researcher is required. Reflexivity asks us to explore the ways in which a researcher’s involvement with a particular study influences, acts upon and informs such research.  The goal is to minimize the power structure and allow people, our participants, to inform and guide the researcher according to what matters most to them, be it spoken or unspoken. In other words, we are not moderating, we are learning and exploring.

So why does any of this matter? It matters because when a client chooses to do ethnographic research, they need to know that they are getting what they paid for – people who understand the theoretical models governing cultural behavior and the training to tease out information and opportunities that traditional methods do not yield.  Ethnography’s strength comes from the ability to work fluidly with participants as opposed to moderating a setting or social interaction. The researcher who refers to him or herself as a moderator of ethnography, through his or her choice of words, is indicating how they will do fieldwork, how they will interpret findings and how they subconsciously see their role in the field. And again, while there is nothing wrong with “moderating”, selling it as ethnography or assuming that the word “moderator” is synonymous with “ethnographer” is like saying that because I can do basic money management I can now call myself an accountant. Or because I own a copy of The Lotus Sutra I am an expert in Buddhism. Or because I can change my oil I am a mechanic.  You get the point.  Not only is it a disservice to the discipline, it is a disservice to the client.

Simply put, if you’re going to hire an ethnographer, it isn’t enough to ask what markets they will work in or how big the sample population will be. If you’re going to spend the money, the time and the effort ask the obvious question: “What do you call yourself.” Then get them to articulate not only their methods, but the rationale behind them. It’s your money. Be sure you are paying for what you have commissioned.

Mobile Design and What the Numbers DON’T Tell Us

Posted by

0

I, like so much of the Western Hemisphere, spent part of my weekend shopping for dolls, pajamas and the latest electronic gadget.  And like so many others, I turned to my mobile phone for support on more than one occasion.  I am far from unique in this regard.  What caught my eye was the number of children I saw with a cell phone in hand – not their own (although there were no doubt some pre-tween kids among the throngs who did indeed have the pleasure of owning a very expensive smart phone, though I hesitate to think they were the norm), but a parent’s phone, which they used to play games, watch YouTube and generally make the tedium of shopping less pronounced.  What struck me as relevant is that when asked in surveys, the data frequently comes back saying that parent’s rarely give over their phones to the kiddies while shopping.  Observations in the field would imply quite the opposite is true.  Mobile phone use in a shopping environment is not just about the owner of the phone, it is about the parent/child dynamic and the underlying practices that go beyond procurement of goods.

Why does it matter?  It matters because while we have plenty of data about what people say they do, we have precious few insights about what’s really going on.  We are still in the wild west mentality of mobile design and need to get a better understanding of the range of contexts as we design mobile experiences for shoppers.  One size does not fit all and the numbers, while compelling, mean little if we don’t address the bigger questions under the surface.  It matters because a good mobile design and a creative mobile strategy can mean the difference between a useful application and a waste of millions of dollars in development, reduced brand equity, etc.

As another example, look at the numbers around the Hispanic market.  The numbers show that Hispanics are generally younger and more technologically savvy. AOL’s Hispanic Cyberstudy reports 46% of Hispanics who are actively online are under the age of 35.  32% of Hispanics access the Internet through their smartphones, compared to 20% of the general market. With roughly $1.3 trillion in buying it’s no wonder that Hispanics are a significant target for marketers.  But the numbers don’t address the bigger questions.

First, which “Hispanics” are we talking about?  Do the numbers refer to 3rd generation Cuban Americans with money or 1st generation Guatemalan farm workers? While or bias would probably lead us to assume the latter isn’t using a smart phone to shop, there is neither qualitative nor quantitative evidence to support this; is simply a matter of our own prejudices and preconceived notions about immigrants.  Second, Hispanics skew younger as a whole when compared against the total US population, so of course they are accessing the web through their phones more frequently than the general market – the data show that this is the case for all younger shoppers, so the difference between Hispanics and everyone else is misleading.  Finally, simply being online via a mobile device doesn’t necessarily mean ecommerce is taking place or that it is even desired.  Is the phone being used to supplement computer-based online interaction or is it a surrogate?  Do the numbers even reflect  use or do they reflect self perceptions, desires, the search for status, etc.? The point is that while we can infer quite a bit from the numbers, we are filtering them through our own biases.  Until we rethink the questions a bit we are designing based on potentially false assumptions.

Regardless of the populations to which we want to cater, designing a good mobile experience should entail getting on the ground and spending time learning what’s really going on.  And sometimes that means getting inventive about how we gather insights.  For example, if you want to understand how a good mobile banking site should operate, it isn’t enough to know the numbers of people using financial apps.  You need to understand how they conceive of money, when they do their banking (e.g. work, at home once everyone is in bed, on the train, etc.), and how they view their bank (many people hate their bank, but the cost of switching doesn’t outweigh the pain).  If you want to understand what the ramification of something going wrong are, spend time looking at mobile transactions in a place like Afghanistan, where banking can be a deadly affair – whatever people are doing in a place where bad mobile design can get you killed will probably shed some light on what is and isn’t necessary the world over.

The point is simple.  Good mobile design means getting your hands dirty and learning how context shapes how people use a given app or mobile site.  If all you have are statistics, you’re bound to create a solution that addresses the wrong problem.

Stats Are B.S. 9 out of 10 Times.

Posted by

0

Nowadays, every company and organization within a company has an agenda of some sort and has taken to throwing out statistics, all in an effort to convince naive people into embracing that agenda. Rarely are the statistics meaningful in a holistic sense. Rather they are meant to bolster an opinion that has no real depth or substance.  Indeed, it more dangerous than the “gut feel” information ethnographers are accused of practicing (by people lacking real knowledge of ethnography).  Why? Because they are absolutes grounded in a singular worldview – they lack any collaborative checks and balances.

The truth be told, statistics are absolutely worthless when it comes to proving anything at all. Statistics are only as good as the method used to come by them, and the way they are presented.  And they are only as good as the minds that construct and apply them.  When numbers are involved, arrogance becomes more dangerous because it holds a weapon the uninformed tend to fear.  In other words, stats are often more about the person talking than anything meaningful.

So the next time someone throws statistics at you, keep an open mind. You do not want to become a victim of misinformation by statistics.

Data Just Ain’t Enough, Folks

Posted by

0

As me emerge slowly from the last recession, retailers are fixating on the “data.” They ask, “How can retailers leverage their in-store customer data for online purchase and preference sharing?” They want t know how in-store data can translate into a greater share of wallet and a greater share of preference? Completely understandable.  The problem is that it oversimplifies the shopping process, reducing how, where and why people shop to a series or series of numbers and gross assumptions about what those numbers mean. IN OTHER WORDS, THEY DON’T CONECT THE DOTS.  Or more accurately, they don’t connect the dots correctly.

If a retailer wants to get past reifying numbers and making assumptions that lead to wasted money, space and time they need to rethink the figures they have and start to contextualize the shopping process. They can recognize that a single purchase in store is part of a complex system of behavior that can translate into unique partnerships, product offerings and promotions that can be adapted to contexts that they may have never considered. Perhaps it makes sense to provide QR codes at a concert.  Perhaps it makes sense for location-based specials and promotions. The point is that the data gleaned from the in-store purchase and the online purchase signal things about each other.

Again, the line between the in-store and out-of-store experience is blurred for consumers and shoppers.  The statistics we gather are useful, to be sure, but they reflect only a single element of why people shop. And if you understand the “what” but not the “why” then you have lost THE opportunity. If you understand the motivations for being in the store and in the greater shopping milieu, that information can be used to tailor digital messages, retail design promotional offerings, etc. that fit the context of the people you want to engage.

A Better Way of Sampling

Posted by

0

As I’m sure many researchers have experienced, there is an obsession with defining research samples in exact, often painfully rigid ways.  We slice, dice and cram people into individual categories and then find ourselves baffled when people don’t do what they’re supposed to do. But people don’t act as individuals.  Or more precisely, people don’t act, think, or do much of anything else without it being wrapped in the social and cultural models that define their worldview.  We are social creatures and act within a system of meaning that is shared and reconstructed through interaction with others. Rather than examining the system, many researchers look at elements.  They rely on data that fit their needs (please the boss, make a product look more successful, etc.) rather than looking at the connections, which is where real opportunities lie. While ethnography doesn’t solve every problem, it does put human meaning and interaction into perspective, something traditional methods can’t do.

The key point in ethnography is that the unit of analysis is not the individual, but the culture in which people operate.  As such, it is intrinsic to understanding ethnography’s value to comprehend that the study of a culture involves exploring two levels of consciousness and meaning: the explicit and the implicit. Explicit culture is what we see and hear people articulate: social mores, tool, basic interactions, etc.  It is that level of shared knowledge people can typically communicate easily, or those aspects of material culture that are readily identifiable. Implicit culture is comprised of those things, which are simply “known” and usually either unspoken or difficult to articulate. It is that space where culture is not just trappings and customs, but rather meanings, symbols, and practices. The implicit side of culture is the domain of meanings takes shape, and it is here where the ethnographic understanding finds its true value.

_g_

The Scientist-Artist

Posted by

0

Great innovations, powerful customer experiences and change are produced by someone in a state of exploration.  It is the state of childlike wonder.  It may seem overly poetic of trite, but it has been the driving force behind those things we see as great leaps forward, be it in art, science or business.  Taking advantage of tools that are available to you can inspire shifts in thought and connections that uncover patterns normally overlooked when viewing a problem from a single vantage point.  Science and innovation are creative acts, not the recitation of data.  When we fixate on solving a problem, we often ignore or overlook elements of the puzzle that inform the bigger picture.

If you do that, you aren’t willing to consider the impossible.  Understanding context and looking for systems rather than identifying elements of a system, ripped from their context, obfuscates.  It does not inform.  This is why the “outlier” is important – it articulates the norm and sheds light on consumer/user/shopper behavior, uncovering the meanings that we are often prone to ignore which explain why we do what we do. A constricted view is great if you’re trying to figure out the optimal screen size for a cell phone, but it means little if you’re trying to figure out what comes next.  Game changing innovation comes when you are part scientist, part artist and part child.

By Gavin

Poetry and Research

Posted by

0

Commenting on something I had written about storytelling, someone said, “Maybe stories are just data with a soul.”

Poetic. And I would say research needs more poetry. Indeed, there are many researchers who would agree. Data need not be dry. Besides, “data” has a richer definition and history than we like to admit in an age that fetishizes numbers. Stories are the symbolic launchpad of metaphor, the defining element of the human condition.

Gavin