
FORMATIVE RESEARCH
By Margaret Walsh-Buhi, Ph.D., MPH
&
John P. Elder, Ph.D., MPH
Introduction
Research is integral to all aspects of determining the best mix of communication inputs, as through research we address all of the key questions we have regarding the "what," "how," "who," and "to whom" of our health communication efforts. For example, who would be the best source for our communication? What media or channels should we use? What should our message look like in terms of both feel and content or heat and light? What does our audience look, think, and act like? What subgroups, or segments, compose our audience? Who in the audience has already adopted the behavior or social change, and who is resistant to it? How can we use audience members and their social networks to inform one another? How can we maintain a behavioral, community, or social change once we get it initiated?
​
These are all questions that need to be researched as we plan a communication campaign. They are not addressed simultaneously, but instead in a sequential fashion that corrects and improves our questions and our communication materials as we present them to people who represent our potential audience.
Part One: Reliability, Validity, Triangulation, and N
First, two quick definitions. Reliability means getting the same answer consistently, over time and often by two different observers. It does not ensure that the answer is correct, but instead, that it measurements yield consistent results. Validity refers to the extent to which our measures reflect the true situation or condition. In other words, validity means that we are actually measuring what we say we are measuring. The relationship between the two is that a measure can be reliable without being valid; that is, we can consistently get the wrong answer. But a measure cannot be valid if it is not reliable.
​
In quantitative research, there are specific ways in which we determine the reliability and validity of our measures, and we are able to apply specific numerical coefficients to these variables. The qualitative research counterparts to reliability and validity are credibility, transferability, dependability, and confirmability (Trochim, 2006). A form of convergent validity, credibility simply refers to the congruence of our findings with "reality," at least as we know it. Transferability, a form of construct validity, refers to our ability to fully describe all the contextual factors that relate to the inquiry. For instance, if we are assessing general political values, we will have different results during the peak of an election season than those we might end up with in a non-election year. Dependability comprises the process by which we report our methods in detail so that other researchers working in different communities would be able to gain the same results in the same way, allowing us to compare results. Finally, confirmability helps us ensure that our findings are the results of the experiences and ideas of the informants, rather than of our own conscious or subconscious biases.
​
How do we establish the specific convergent validity of our qualitative and quantitative measures? Denzin et al. (1978) [1] and Patton (1999) [2] identify four types of triangulation, or the determination of whether different forms of research yield congruent and complementary information. The process of triangulation derives from the importance of combining our qualitative research (such as through in-depth interviews or focus groups) and quantitative data. Triangulation can take a variety of forms, such as “methods triangulation” (determining the findings generated by different data collection methods), “source triangulation” (determining the consistency of data from different sources, perhaps at different points in time or in public versus private settings; or from different [3] people with different points of view), “analysis triangulation” (determining the congruence of interpretations that we and other researchers may have), and “theory triangulation” (determining whether we perceive the results of our study to be consistent from different theoretical points of view).
​
The process of triangulation will assist us in determining the N: the number of individuals we need to interview/focus groups we need to conduct (invoking concepts very similar to the statistical determination of central tendencies, variance, and power). Eventually, you will likely start getting redundant information from additional individuals or groups (i.e., you’ve established the central tendency). At this point, you may have reached "saturation"; in other words, a) there is not much new information to be gleaned from further qualitative research, at least until the program is farther along in terms of its development or implementation; or b) you are in the program evaluation phase, when you want to know what worked, what did not, and why. Finally, qualitative data analysis may be as straightforward and simple or as complex as the situation calls for (and as the moderator and their organization have the capacity for). Essentially, we look for patterns and unique responses both in focus group and individual interview responses. We then consolidate these responses into the more typical and atypical responses, and proceed with program planning accordingly (e.g., the development of social media or video materials) or to prepare to actually launch a campaign. There are far more sophisticated ways to analyze qualitative data, including many software programs that range from interview analyses to “big data” and web analytics (see the Social Media module). Health communication professionals are increasingly turning to mixed methods research, which combines quantitative and qualitative data to come up with optimal program planning and evaluation approaches.​
Part Two: Focus Groups and In-Depth Interviews
Focus groups and individual (either short or longer, in-depth) interviews may take a variety of forms, such as: a) informal and conversational, b) guided generally by an interview guide, c) standardized but with open-ended questions, and/or d) standardized but with closed-ended, fixed-response questions. The quality of these data depends largely on the skills of the researcher/moderator. What are the characteristics of a good qualitative researcher? Check out this humorous and dated-but-still-relevant video on qualitative research.
​
Focus groups are good method to obtain rich qualitative, formative data, especially if one is looking for a range of ideas or feelings people have about health and social issues. They also are convenient if you're trying to understand differences and perspectives between groups or categories of people. In focus groups, you will want to uncover factors that influence behavioral patterns, motivation, and other characteristics that provide insight into complicated topics. Focus groups are a good format for allowing ideas to emerge from the group rather than what you could achieve with individual interviews, as the participants present not only their own opinions, but then react to those of others as well. Focus groups can be used to pilot-test concepts, oral presentations, videos, posters, and other materials. Focus groups can also take on more complex issues, such as campaigns, policy changes, and legislation. Both focus groups and individual interviews can set the stage for subsequent quantitative research, for example, by helping optimize the clarity of questionnaires. Focus groups especially can help the researcher with specific language variations and the use of slang.
​
Focus groups, however, are not the correct format for every research challenge. Participants in focus groups may be influenced by more vocal and dominant individuals, or all members may work to arrive at an harmonic consensus that masks true attitudes (i.e., the information is reliable but not valid). Nor are focus groups necessarily appropriate for asking sensitive information (e.g., sexual practices, drug use) that individuals may be willing to share only on a one-on-one basis with a professional. Participants are aware that focus group information is not necessarily purely confidential, as fellow participants in the group may share what they've heard with others once they leave the group. If the focus group topic is especially emotionally charged, conflicts can arise among the members and disrupt the group. Focus groups can be difficult to arrange logistically contrasted to individual interviews.
​
Planning
In planning the in-depth interview or focus group study, the researcher or research team must first be very clear about the broad purpose of the study, or its “destination”. The following questions must also be taken into consideration, and shared by the researcher with the participants:
​
1. What is the problem or issue that the study will address?
2. What led up to the decision to do the study?
3. What is the specific purpose of the study?
4. What kinds of information do you want (e.g., wording for quantitative questions, responses to health communication materials)?
5. Who is it that wants the information, and/or to whom will you give the information?
6. What are the ethical, legal, economic, and social considerations that may lead you to conduct research and to choose one method over the other?
​
Although these questions may seem obvious, going through them will ensure that your research meets the standards of being efficient, reliable, valid and ethical.
Research Activity 1
Think about a health or social issue that is a priority for you personally or professionally, and that others may have varying opinions or attitudes about. Arrange a small group of individuals for a focus group or one or two individuals who might sit with you for a one-on-one, in-depth interview and conduct some formative research around this issue. Prepare a guide for yourself so that you stay on track, and don't violate any of the principles set forth above. Given that this is just an exercise, feel free to use friends and relatives. Ask them to role-play as though they were strangers you did not know before. They can even be given a role to play (such as a parent, college student, construction worker, etc.). Ask them if you can record the session. Transcribe your notes, and then erase the recording. Remember to maintain participants’ anonymity and confidentiality.
Developing Individual and Focus Group Interview Questions
There's a big difference between good and bad interview questions. The following guidelines will improve the quality of your research effort:
​
1. Input: Get feedback from other experts and even people who resemble your audience as to how clear and interesting your questions are.
2. Language: Use words and phrases that participants commonly use (and, conversely, avoid technical terms or slang that would not be understood very well by the participants).
3. Clear and short: Questions should be open-ended to the greatest extent possible, but structure potential responses into succinct rather than rambling answers.
4. One question at a time, please: Ask one-dimensional questions rather than double-barreled questions (i.e., embedding two distinct questions within one phrase). Double-barreled questions result in answers that we cannot rely on. For example, rather than asking “did the news sadden and concern you?”, break this down into two distinct questions, since it is possible for respondents to say “yes” or “no” to both or one of the emotions mentioned.
5. Pilot: Test your questions and repeat the above steps as needed.
6. Manage your time: Focus groups should last approximately two hours with about 10 minutes each per 12 or so questions. In-depth interviews can be much shorter, given that you're not engaging more than one person in a broader conversation. Sequence your questions to maximize clarity and efficiency. Don’t allow responses to go much over allocated time.
Phrasing and Flow
To the greatest extent possible, use open-ended questions. Give respondents a time frame to think through, such as: “over the past week, what have you thought about the dangers of opioid use? How does that compare to how you looked at it last year?” Make sure to avoid reinforcing individuals selectively for certain types of responses, especially those that you may agree with personally and, conversely, disapproving of other responses when they are inconsistent with your own personal values or behavioral patterns. In other words: do everything possible to avoid being judgmental.
When sensitive topics come up, individual participants, especially in focus groups (contrasted to one-on-one, in-depth interviews), may be very reluctant to give their own experiences as they may find their personal information to be embarrassing. In such cases, change the wording of your question to the third person voice (i.e., from “what do you believe/think/do?” to “what do people like you in general believe?”).
Make sure your questions are appropriately sequenced. There typically are five categories of questions that would be presented in this order: opening questions, introductory questions, transition questions, key questions, and ending questions. Opening questions may include ‘icebreakers,’ such as asking participants to say something humorous about a family or class reunion they recently attended. Introductory questions introduce the theme of the focus group interview and give participants the broad concept of the health or social issues to be discussed. Transition questions then get people to start drilling down into some of their relevant behavioral and emotional experience with the theme. Hopefully by then the group will be ready to deal with the key questions related to the theme. Finally, ending questions will be a way to conclude, debrief, and ease any tensions that may have arisen during the group, and summarize the purpose of the group. You then conclude by thanking the participants for their valuable information and saying farewell.
Putting Your Group Together
How do we find participants for our groups? There is an extensive array of techniques [4], and here are a few of them. Certainly, advertising through PSAs, flyers, posters, and the Internet (including social media networks) are effective ways to recruit participants. In certain cases, nominations from your colleagues or other individuals in the target audience might assist you. Piggybacking onto already established groups (e.g., getting permission to ask a few questions at the end of a Parent Teacher Association, or PTA, meeting for parents) may be an efficient way to find focus group participants (or even ‘ready-made’ groups, e.g., a PTA), as would finding individuals at a location where you can ask them to spend a short amount of time with you. In all cases, you need to be ready to tell the participants what is expected of them in terms of time and effort, and what can they expect in terms of reward.
​
If you are able to select from a large number of participants, make sure you set specific inclusion and exclusion criteria (e.g., must be over 18 but under 35, must have a college education, must have equal numbers of whites, African-Americans, Latinos, and Asian-Americans, etc.). And if you're interviewing individuals of parenting age, offer childcare as well. Transportation may be an issue for focus groups since not everyone will live near where you want them to be, especially if you are recruiting economically disadvantaged individuals. In all cases, incentives should be as generous as allowable by IRBs and should be sufficient to attract individual participants without influencing them in terms of the type of responses they will give you.
​
Sampling
In terms of finding the most representative participants for a formative research effort, a subset of the population is selected, just as it is for any given study. In formative research, however, we simply do our best to find representative individuals who can give us information that we can generalize to a broad enough sample to at least begin a health communication effort or develop a more substantive research approach. Therefore, if we're interested in all undergraduates on a large campus, we would want to sample our interviewees more or less according to the sex, racial/ethnic, and age make-up of the campus. There are many different scientific and non-scientific approaches to sampling. Three different sampling approaches are used in formative research: purposive sampling, quota sampling, and snowball sampling.
​
Purposive sampling is the least scientific approach to determining our participants in terms of their generalizability (although the relative scientific merit of various approaches is debated). Essentially, this involves researchers using their own judgment to determine how best to select a representation or subset of individuals. This may often be a more efficient approach, but it does not allow one to make strong generalizations to broader populations. An example would be to walk up to a few students coming out of the Student Union and try to get one or more of them to discuss attitudes about soft drinks, obesity, etc. This information may be used for a news article, informal report, or even pilot data for the next phase of qualitative or quantitative research.
​
Quota sampling is used when the researcher wants to make sure that certain characteristics (e.g., race, gender identity, drivers vs. mass transit users) of a population will be represented to the extent determined by the investigator. For example, if we want to make sure that our survey participants match the class standing of a university (freshmen, sophomores, etc.), we would make sure that we sample individuals who fall into each of these specific categories in the exact proportions they exist across campus. For instance, if at the end of our first effort we find that only 40% of our sample are male while it’s 50% campus-wide, would need to go out and find enough males to fill that quota to make the samples equal, matching the university profile.
​
Snowball sampling (or “chain referral sampling”) is used when scientific or purposive samples may not get us to the people we want. For instance, we may want to conduct research into health protective behaviors among sex workers. Using confidentiality and anonymity, we may be able to assure individuals that they are free to speak with us, but we would have no way to find a large enough sample through standard techniques. Instead, we would ask our initial respondents to name a few individuals we could contact using their social networks and proceed accordingly. Therefore, the final sample will comprise a "snowball", developed with one individual serving as the core of the sample and using their networks and those of subsequent respondents to find others.
​
Composition of the Group
​It is probably not a good idea to include individuals who have distinct experiences or expertise with the theme of your research. For instance, if we're talking about educational challenges, one would not necessarily have a group mixture of adults with college education, and others who have limited formal education. The researcher should also control to the extent possible participants’ level of commitment to or passion about sensitive issues, such as reproductive rights (unless it is the very tension and conflict about the issue that is of specific interest to the researcher).
​
One approach to group composition is the case-control design borrowed from epidemiology, presented as well in the audience module. If you're interested in how to get people to adopt a behavior or engage in social change, consider running separate groups of early adopters vs. non-adopters. This will allow you to learn why some individuals have already engaged in the change that interests you (and what tips they might have for a communication campaign). At the same time, the ‘controls’ can inform us of why they have not adopted a certain behavior or action, and what it might take to get them to do so. It would not be a good idea to mix these groups, as one subset may intimidate the other or conflicts may arise between them.
Analysis Activity 1
Educational Foundations and Research of the University of North Dakota developed a video to demonstrate how focus groups can produce meaningless results. Cheryl Hunter, the moderator of this focus group (purposefully) makes several mistakes in running her group. Watch the video and see how many of these you can catch. (The answers are presented by the producers on the subsequent link; please do your own work first and then check it with their answers—they may have missed some other mistakes!).
Analysis Activity 2
Now check out this real life video. Is this truly a focus group? Why or why not? Regardless of your answer, how could it be improved?
Concluding Remarks
This module has presented a cursory introduction to formative health communication research. This topic could easily comprise a course in itself (as could, for that matter, other modules’ topics in the Global Communication Project). As a health communication professional, you may be conducting your own formative research. If you study this module and the information in its links, you should be able to do so effectively. On the other hand, should resources allow and topic complexity necessitate, you might consider contracting with a business that specializes in this methodology. The basic information in this module will help you select the right person or firm to assist you. We trust that the basic skills presented in this module will guide you in making and implementing those decisions.
References and Footnotes
[1] (Note of course that even the best quantitative research does not always succeed in making correct generalizations to entire populations, as we witnessed in the opinion polls in the 2016 presidential election in the USA).
​
[2] Denzin, Norman K. (1978): The Research Act: A Theoretical Introduction to Sociological Methods. New York: McGraw-Hill
​
[3] Patton, MQ. (1999). "Enhancing the quality and credibility of qualitative analysis." HSR: Health Services Research. 34 (5) Part II. pp. 1189-1208.
​
[4] Namageyo-Funa, A., Rimando, M., Brace, A. M., Christiana, R. W., Fowles, T. L., Davis, T. L., Martinez, L. M., & Sealy, D. (2014). Recruitment in Qualitative Public Health Research: Lessons Learned During Dissertation Sample Recruitment. The Qualitative Report, 19(4), 1-17.