Tag Archives: data collection

Making Fieldwork Easier and More Productive

Posted by

0

Advertisers, marketers and designers have long held the role of creating materials that reflect the lives of customers. Traditionally, this has relied on market research that is gathered in something of a vacuum, or reflects the beliefs and practices of the researcher more than the consumer.  People’s preferences all too often are neatly, if unimaginatively, packaged and handed off to a team tasked with creating new design applications, be the application a new product or a new brand platform.

Quantitative methods such as surveys demographics data provide wonderful snapshots of a large population but give little insight into what matters most to people and why it matters.  Usability tests and affinity diagrams provide information about the acceptability of new design concept and prototype, allowing designers to adapt and alter the message of a brand, campaign or product according to people’s stated preferences (which may or may not reflect their true beliefs).

From the qualitative side, focus groups and group interviews provide more qualitative feedback on product concepts,  messages and, to a lesser extent, explore unmet needs. The problem is that focus groups often reflect exaggerated responses and how important it is for humans to feel clever in front of perfect strangers. Additionally, these methods rely on people’s awareness and descriptive ability away from the context in which they would normally be thinking about a topic. In other words, they make things up, usually subconsciously but sometimes intentionally, in order to give an answer to a question. The result for design is mediocrity at its best and radically failed messaging at its worst.

Direct observation combined with interviewing (ethnography and ethnography-lite) is perhaps a more compelling method of coming to understand what people say, think and do.  It has certainly become a fixture in many organizations in recent years. And from a design standpoint it gives both researchers and designers a richer understanding of the issues, practices, and peculiarities of shoppers and the consumer, providing a more complete picture to work from when developing a brand or campaign. The problem is that while the depth of information uncovered is rich and insightful, it often stops short of any real observational depth that can be crafted into something truly meaningful. Surface-level findings are just that – surface level. If fieldwork is to be genuinely inspiring it needs to dig deep.  And researchers need to begin recognizing that their work is a creative, interpretive process.  That means that we needn’t fixate on getting the “right” answer, but that we get an interpretation of data that provides a “creative” answer.  Doing that means rethinking how we conduct research.  Here are 5 tips to making observational research relevant.

Start a conversation. When entering into fieldwork researchers tend to immediately jump into asking questions. The problem is that the abruptness and intrusiveness of these questions often changes behavior, resulting in semi-meaningful answers. To prevent this, begin with conversation and observation. Yes, that means allowing yourself time to get to know your participants as people. Let questions emerge as activities unfold. The simple fact is that we frequently don’t know what we should really be looking for until we’ve had time to immerse ourselves in the surroundings.  Simply put, relax and take your time rather than buffeting people with questions.

Look for patterns. Behavior does not happen in a vacuum.  People are individual organisms, yes, but they work within a social and cultural framework.  That means that activities and statements are always part of a larger pattern of meaning and practice.  Don’t take statements at face value.  Always look for patterns and connections between what people say, believe, and do.

Record information in their terms. Record what the participant says in their own terms rather than paraphrasing. Word choices, inflection, cadence, and non-verbal cues carry meaning that is lost when we try to simplify.  Facial expressions and body language convey a great deal of information. Simplicity will come out of the analysis – don’t do it when you are gathering information. 

EVERYTHING is data. Seemingly unimportant details are often the pieces that are the most important.  Environment and context have a huge influence on what people say and do.  Therefore, it is crucial when gathering information to include as much as possible in the interpretive process.  It may seem overwhelming, but everything is potential data for the analytical and creative mill.

Relax and embrace a range of perspectives.  Research should not be a list of facts and observations if the goal is to generate insights and innovation.  Research is a creative and interpretive act, no matter how much we may try obscure that fact.  As such, research is most effective when a wide array of disciplines are engaged in fieldwork.  Turn off your “scientist” sign and include a range of perspectives both in data collection and in analysis.

Customer research is only as powerful as its outcome.  Generating volumes of consumer insights and observations means very little if those insights and observations can’t be readily translated into something tangible, be it a brand platform, an ad campaign, or a new product offering.   While fieldwork can and does yield powerful insights, it means little if we forget that we are in a creative field that works best when a wide range of skills and perspectives come together.  Both in the field and out.

Video and Analysis

Posted by

2

It is not uncommon for a researcher using film in data collection to run into people concerned with the validity of the method.  Sometimes the concerns revolve around whether film and video are art or science.  Because of its interpretive, creative, impressionistic, and emotional attributes, art is sometimes assumed to be in direct conflict with an objective, value-free “science”—apparently creating an unavoidable conflict between the goals of film as art and user research as science. Consequently, people—academics and professionals alike—often assume limited possibilities for film.  The status of film as a serious analytical resource has remained fairly marginal.

Film is sometimes seen as a humanistic pastime, not significant scientific work. It is meant to appeal to the audience’s emotional pliability.  Ultimately, the producer of the final visual document is seen as selectively building subjectively constituted data and constructing a piece that reflects his/her interpretation rather than “the facts”.  However, the same can be said for any written document, particularly when behavioral research methods are applied to data collection for a specific task or client need.  A logo-centric culture prevents researchers from benefiting from the full breadth of insight and information available, treating video as if has less validity than the written word.  However, written reports often have pictures, films often use written narratives, subtitles or intertitles.  They always have accompanying written material.  The reality is that while the film-focused researcher does indeed run the risk of compromising the complex realities of a particular behavior or series of behaviors, the risk is no greater than that of the researcher relying primarily on the written word.

Typically, film is accepted most openly is when it is considered to fit the documentary archetype.  This stems from the widely held belief that film is a mirror for the world.  The argument is that the camera is a device for scientifically recording data about human behavior that is more objective than other types of information because of the mechanical nature of the collection device.  While this may be true, it probably is not.  However, given the context of the work (time limitations and constraints imposed by the nature of contractual research), the footage supplied by the camera may be as close as we can get to a check of objectivity.  The reality of research purchased by a company is such that it assumes, even demands, a final product that is easily used, applies to a wide range of internal needs, and can be readily disseminated.

For some, manipulation of the footage (editing it into a film, altering, etc.) destroys its “scientific value.”  The model is that teams go into the field to film material, the scientist studies the footage, and the filmmaker transforms into art.  In actuality, this fantasy is never realized.  The footage is indeed dissected and analyzed by the researcher, typically transformed into a product the client will readily consume, but by its very nature qualitative research always has a degree of subjectivity.  In fact, any and all research, be it in the field and interpretive or in the laboratory and highly controlled, involves degrees of subjectivity and personal biasing.  This hardly invalidates the work or the means by which data are captured and displayed.  Validity and reliability are not necessarily one and the same.

If researchers are supposed to make films intelligible to client audiences, they must learn what common sense, such as it is, dictates as constituting a good documentary film, that is, they should emulate the aesthetic conventions of documentary realism.   Pieces of the puzzle are, of course, missing from any documentary film, but the most important themes and primary informational pieces remain for consumption by a wide range of viewers.  The pieces selected for a final edit do indeed play to the emotions of the client, but without that emotional impact clients are likely to forego the deeper issues entirely, unwilling or unable to sift through the informational tome so often presented by researchers.  By communicating customer needs, reactions, behaviors, etc., film spurs viewers to delve deeper into the research findings and examine the totality of the research in greater detail.  Film can be used to access a level of emotional response and personal identification or conflict which is difficult within the lexical constraints of writing.  By a series of movements in a sequence, films can communicate in concrete and specific terms what in written words would be abstract expressions.

Another argument against video documentation as a primary means of disseminating findings is that because prior consent is always sought, there is always some degree engagement by the participant with the camera and therefore the findings are inaccurate.  However, the very fact that participants are recruited for any study by definition means that there is some degree of awareness and engagement.

Consequently, whether the awareness and engagement take place with the researcher exclusively or with the researcher and camera together, the authenticity of an activity, context, or behavior should not be in dispute.  After all, typically, the camera is soon forgotten, but the person asking questions and watching over the shoulder remains.

The Case of Cell Phones, Youth, and Japan

We applied video field data gathering and mini-documentary reporting during a recent study for a wireless communications provider who wanted to understand mobile phone usage in Japan.

A company had contracted with the consulting firm for whom I was working at the time in the hope of gaining a better understanding of how  portable information devices (such as PDAs) and internet-ready cellular phones were used in the context of daily life.  They were interested in uncovering what characteristics other than image quality, sound quality, and functionality were determinate in the decision to purchase a PDA or cellular phone in urban centers of Japan, and why those “peripheral” issues were important.  The term “peripheral” is the term used by executives to describe how they viewed the work – they were skeptical of the notion that culture impacts perceptions and uses of technology.  So, while the team was ensured work, there was little guarantee that the findings would be implemented.  In addition, the researchers were given half the time to conduct the research that they had originally requested.  Gaining the attention and interest of primary decision makers became in order to conduct further, more in-depth research at a later date became almost as important as the findings.  Without continued research, the researchers feared that the company would act without consideration to the needs and cultural patterns of the population.

The team was asked to identify some of the behavioral and cultural motivators in the purchasing decisions of young (16 – 30 years old) Japanese from middle-income homes.  The research took place in several locations in Japan to provide a range of cultural practices.  However, because the researchers (two ethnographers and one social psychologist) were out of touch most of the time but needed at the end of the project to build a single, cohesive series of conclusions, they needed to capture the participant observation sessions on video for later shared analysis and review.  Added to this was the fact that only two of the researchers spoke Japanese well enough to effectively communicate.   The other had to rely on interpreters or the language skills of the informant.  The researchers decided it was imperative to capture on video exactly what was said for later analysis and translation.

Because of time constraints and the limited language skills of the researchers, the goal of the research centered greatly on material culture, display,  and overt patterns of interaction.  Consequently, activities, objects, spaces, and moments of interaction needed to be captured on video so that the researchers could return to the tapes later to catalogue patterns.  Without the video footage, much of the information would have been overlooked or misinterpreted – video allowed the team to accurately assess their assumptions, catalogue use patterns and artifacts, and check for validity.

By returning to the video over a two-week period, the researchers were able to determine with some accuracy what designs were preferred and why, what levels of functionality were important, what was most significant in terms of brand and image, and what patterns of interaction were taking place.  It also allowed them to demonstrate what they did not know and thus get buy-in to conduct more extensive research.  The final video presented to the company ensured that business planners and designers would be sensitive to cultural aspects of products to be used in Japan.

Analytical Basics of Photo and Video Journaling

Posted by

2

Increasingly, research involves elements of visual journaling. The advent of the smart phone and access to cheap, small digital camcorders and cameras has made it easier to involve participants in the process of data collection through time.  The question is, what to do with all the video and images you collect as participants essentially become part of the research team? Step one is to simply organize what it is that comes in from a visual standpoint. Step two is to categorize the data based along lines of what the participants tell you about it. I specifically look at the linguistic component here.

Initial analysis should identify the level of reflection. What are the basic cognitive frames around which the language is centered. While there are any number of levels the researcher can devise, I typically subscribe to the levels of reflection developed in Chabon & Lee-Wilkerson (2006):

  • Level 1 – Descriptive: When describing the image or video, the participant provided evidence that new knowledge was obtained, which allowed him/her to make sense of new experiences or make links between old and new knowledge (what one used to think/did vs. what was learned and how it affected him/her).
  • Level 2 – Emphatic: When describing the image or video, the participant expressed thoughts or emotions about others and self. He/she reflects their experience (emotions, attitudes, beliefs) onto how future participants may feel or react to the experience. Participant empathizes with those around him/her, projecting expectations and empathy.
  • Level 3 – Analytic: When describing the image or video, the participant demonstrates the application of learning to a broader context of personal life. The photo journal entry provides evidence of learning/growth in order to contrast, compare, or plan for new actions or responses. Participant also notice unexpected positive or negative outcomes related to the image or video – greater attention is paid to connecting the research topic with other ideas, issues, etc.
  • Level 4 – Metacognitive: When describing the image or video, the participant demonstrate examination of the learning process, showing what learning occurred during the journaling, how learning occurred, and how newly acquired knowledge or learning altered existing knowledge, practices or beliefs. The participant plans to change future behavior based on the project experience and its outcome(s) on his/her life.

To help diminish bias, entries are coded independently by two individuals unfamiliar with the topic, and then compared for differences which are to be resolved by consensus.  Once the data are coded and the relevant themes identified, they can then be tied to the business objectives of the project, resulting in better strategies, greater innovation and increased revenue.

Doing Rapid Ethnography

Posted by

0

The hallmark of ethnographic research is field work done in natural settings, where it can yield a broad picture and provide a more complete context of activity. But, ethnography can, at times, scare our clients.  Because of this depth, it is often seen as slow, expensive and inclined to produce more information than can easily be translated into action.  And to be fair, that assessment can be true.  However, an ethnographic approach need not always be so.

Due to budget constraints and time demands, a “ rapid” approach to ethnography can be both more practical and still yield findings and insights that can produce highly actionable results.  Is it always appropriate? No, but it is often better than no research and may actually be more beneficial depending on the goals of the research. In a rapid ethnography model, researchers can lessen time demands taking short focused studies to rapidly gain understanding of the brand, the product and the actions/meanings surrounding them. The trick is remembering design the processes around tighter focus, interactivity with participants and collaborative data analysis, not only with the researchers but members of the client team.

First, focus can be more difficult to achieve than we think.  A central tenet of good ethnography is that you don’t go into the field with your answers running – you’re goal is to learn about context and then determine how a product fits into the system.  So focusing too early can mean loading the front end of a project with too many preconceived notions. So focus in the sense we’re talking about here means having research teams identify the general area of interest and identifying specific questions that need to be answered by the fieldwork.  This means identifying the “why” behind the questions rather than simply recreating a context-based survey.

This means developing both a concise field guide (things to look for) and a field book (consistent, shared mode of documentation) before the fieldwork begins that are specific enough to target and isolate key behaviors and activities, but open enough to let the participant serve as the guide.  Constructing the field guide and field book in this way will help direct what research teams attend to during the data collection process and how they frame the field analysis. This is also a good place to consider using liminal members of a group and/or outliers. Because they are on the periphery of the subject in question, they often have ken insights on what others are doing and why they’re doing it.  Sometimes the best insights come from those least inclined to interact with a product or brand.

Another consideration in conducting a successful rapid ethnography is for researchers to use multiple techniques to increase the likelihood of discovering new concepts, interesting behavior, etc.  As an example, using art work or writing in the process can yield symbolic associations that wouldn’t necessarily come out immediately in the context of traditional ethnography.  Asking people to create and construct changes the nature of the inquiry and produces results that can then be compared against both the interview and the observations.  It is another way of quickly triangulating data. Other techniques might include resource flow documentation, defining activity valleys and peaks, or using cross-participant interviews (participants interview each other).

The third point to stress is using collaborative analytical methods.  Computer assisted analysis is always an option, but requires added expense and can be time consuming to learn  However, there are alternatives to ATLASti and other such tools.  Simply having a secure networking site where field notes, insights and observations can be shared between team members at the end of the day is extremely helpful.  Of course, the risk is that people might start jumping to conclusions too soon, but that can be mitigated through dialog. The point is that this allows researchers to collaboratively understand the ever-expanding field data and modify or refine the research in real time.  Another technique is to use metaphor and concept mapping in a shared system to help researchers align the underlying meanings of what they’re finding in the field rather than waiting until the end of the fieldwork to tease out insights. The creative side, insights-driven side of the research is essentially done in tandem with the fieldwork.

Is rapid ethnography always an ideal approach?  Certainly not. But it is a useful tool when budgets and time are limited.  And increasingly, it is simply part of the tool kit we have.