Another set of thematic analysis papers

Braun and Clarke (2021); Massey, 2011; Trainor and Bundon (2021); Byrne (2022); Peel (2020).

One size fits all? What counts as quality practice in (reflexive) thematic analysis?

Braun and Clarke (2021)

Following procedure is not a guarantor for doing ‘good TA (p.329)

TA does – we believe – offer a distinct way of working with qualitative data, and that, although it shares some features in common with other approaches that seek to identify ‘patterns’ in data (e.g., grounded theory, interpretative phenomenological analysis [IPA] or qualitative content analysis), it is nonetheless a method (or cluster of methods) in its own right. (p.330)

Through this writing, developing a TA website,1 and teaching, our understanding of the (evolving) landscape of TA (as we see it) has deepened, as has our clarification of where our approach ‘fits,’and what elements are most vital to quality (reflexive) TA, and why. In recent publications, we have more carefully articulated the assumptions and values around qualitative research that inform our approach to TA (e.g. Braun and Clarke 2019a) to demarcate what is distinct and different about our approach (e.g. Braun and Clarke 2019c; Braun et al. 2019a). We now call this approach reflexive TA (see Braun and Clarke 2019a, 2019b; Braun et al. 2019a; Terry et al. 2017).

This not only demarcates it as a particular TA approach, it emphasises the importance of the researcher’s subjectivity as analytic resource, and their reflexive engagement with theory, data and interpretation. (p.330)

 

Our original paper sought to provide accessible guidance for TA research that

retained flexibility. (p.330)

 

We acknowledge the limits of written guidance, and the potential for it to be (mis)interpreted as prescriptive. (p.331).

 

Furthermore, we aim to be clear that this phase approach is not intended to be followed rigidly. (p.331)

 

Researchers using reflexive TA inductively need to identify, and ideally articulate in their reporting, the theoretical assumptions informing their analysis. (p.331).

 

Using reflexive TA deductively means existing research and theory provide the lens through which we analyse and interpret data. (p.331).

 

Within reflexive TA, the coding process is integral to theme development, in the sense that themes are an ‘outcome’ of these coding and theme development processes, are developed through coding; coding is not – in general – a process for finding evidence for pre-conceptualised themes. The analytic process involves immersion in the data, reading, reflecting, questioning, imagining, wondering, writing, retreating, returning. It is far from mechanical and is a process that requires ‘headspace’ and time for inspiration to strike and insight to develop (Gough and Lyons 2016). (p.332).

 

continuously and rigorously reflect[ing] on their own taken for granted thinking’ (p. 1760) when researching the experiences.

 

Time and space (with the data) help develop the nuanced analyses that reflexive TA can deliver, producing rich, complex, non-obvious themes that could never have been anticipated in advance of analysis. (p.332).

 

Our failure to fully articulate the assumptions informing our approach to TA, and how our approach differs from the other approaches we cited (e.g. Boyatzis 1998), undoubtedly contributes to the confusions and misconceptions apparent in some TA research. (p.332).

 

As previously noted, TA refers not to a singular approach, but rather to a cluster of sometimes conflicting approaches, divergent both in procedure and underlying philosophy, but which share an interest in capturing patterns in data. Yet too often authors do not specify their particular orientation to TA, or indeed acknowledge the diversity of TA. (p.333)

 

Reflexive’ TA captures approaches that fully embrace qualitative research values and the subjective skills the researcher brings to the process (p.333).

 

Analysis, which can be more inductive or more theoretical/deductive, is a situated interpretative reflexive process. Coding is open and organic, with no use of any coding framework. Themes should be the final ‘outcome’ of data coding and iterative theme development. (p.333)

 

Demonstrating

coding reliability and the avoidance of ‘bias’ is illogical, incoherent and

ultimately meaningless in a qualitative paradigm and in reflexive TA, because

meaning and knowledge are understood as situated and contextual, and

researcher subjectivity is conceptualised as a resource for knowledge production, which inevitably sculpts the knowledge produced, rather than

a must-be-contained threat to credibility. (p.334)

 

We encourage TA researchers to clearly demarcate which TA approach they are using. Furthermore, if they cite authors from differentorientations to TA, to clearly specify what they are ‘taking’ from each and justify (well) any use of divergent criteria and practice. (p.335).

 

The take away . . . Researchers should always reflect on and specify the philosophical and theoretical assumptions informing their use of TA, even inductive TA. (p.338).

 

Experience is likewise often assumed to be accessible through

TA, and TA is commonly described as particularlycompatible with phenomenology

(e.g. Guest, MacQueen, and Namey 2012; Joffe 2012) or even as

a phenomenological method: ‘thematic analysis adopts a phenomenological

position to systematically identify themes’(Newton-John et al. 2017, 1822). (p.338).

 

Before IPA, TA was used as a phenomenological method in psychology (e.g.

Dapkus 1985), yet the proclamation that TA and phenomenology are aligned

is rarely explained. We speculate that this reflects an understanding of TA as

(only) compatible with broadly experiential approaches to qualitative research,

and the analysis of ‘subjective viewpoints’ (Flick 2014, 423) – research underpinned

by a reflective view of language and focused on exploring participants’

lived experience, sense-making, views, needs, practices and so on, through

a broadly ‘empathic’ lens (Braun and Clarke 2013; Willig 2013). This framing

is unnecessarily limited. (p.338).

 

We contend that even TA with a descriptive purpose is an

interpretative activity undertaken by a researcher who is situated in various

ways, and who reads data through the lenses of their particular social, cultural,

historical, disciplinary, political and ideological positionings. They edit and

evoke participant ‘voices’ but ultimately tell their story about the data: ‘social

research cast through voices typically involves carving out unacknowledged

pieces of narrative evidence that we select, edit, and deploy to border our

arguments’ (Fine 1992, 218). (p.339).

 

Our language use is never neutral, even in apparently descriptive reporting. (p.339)

 

Interpretative depth lies in the skill of the analyst, not the method. (p.340).

 

In reflexive TA,

a code is conceptualised as an analytic unit or tool, used by researcher to

develop (initial) themes. Here, codes can be thought of as entities that capture

(at least) one observation, display (usually just) one facet; themes, in contrast,

are like multi-faceted crystals – they capture multiple observations or facets

(occasionally, rich, complex and multifaceted codes might be ‘promoted’ to

themes [Charmaz 2006], a process called ‘subsumption’ in IPA [Smith,

Flowers, and Larkin 2009]). (p.340)

 

‘An account of themes “emerging” or being “discovered” is a passive account of the

process of analysis, and it denies the active role the researcher always plays in

identifying patterns/themes, selecting which are of interest, and reporting them to

the readers’ (p. 80). We quoted Ely et al. 1997, 205–6) who argued that ‘if themes“reside”anywhere, they reside in our heads from our thinking about our data and

creating links as we understand them.’ (p.343).

 

We acknowledge that our (initial) phrasing of the third phase of reflexive TA searching for themes’ –has likely contributed to confusion around the conceptualisation of themes as pre-existing entities that reside in data. For this reason, we have, for now, relabelled this phase ‘generating initial themes’ to highlight the active role of the researcher in theme creation and the provisionality of themes when first developed. (p.343).

 

We encourage researchers using reflexive TA to write about theme generation as a creative and active process, one they are central to, and to always avoid claiming that themes emerged. (p.343).

 

Yes! We encourage authors to explain and defend their research values, using the information provided in this paper and elsewhere to justify their challenges to requests or requirements from editors and reviewers. (p.344).

 

Table 1. A tool for evaluating thematic analysis (TA) manuscripts for publication: Twenty questions to guide assessment of TA research quality. (p.345).

 

To improve the quality of published TA, we encourage researchers to reflect on the

relationship between analytic practices, including quality practices, and

the ontological and epistemological foundations of their research, and to

use TA knowingly, deliberatively and reflexively. (p.346).

 

A proposed model for the (thematic) analysis and interpretation of focus groups in evaluation research
(Massey, 2011)

 

Evaluation researchers increasingly use both qualitative and quantitative methods in their evaluation efforts. Among the more common qualitative methods of obtaining data are focus group techniques. Focus groups have been described as a ‘‘carefully planned discussion designed to obtain perceptions on a defined area of interest in a permissive, non-threatening environment’’ (Krueger, 1994; p. 6). They combine elements of both interviewing and participant observation, and provide an opportunity to probe the participants’ cognitive and emotional responses while also observing underlying group dynamics (Vaughn, Schumm, & Sinagub, 1996). (p.21).

 

The group setting and the moderator’s ability to offer helpful prompts are designed to encourage an insightful discussion of the pertinent issues among the group members. The resulting data offers a robust alternative to more traditional survey methods when absolute numbers of respondents are less important than is a rich investigation of content. (p.21)

 

Focus groups are also unique in that they allow data both from the individual, and from the individual as part of a larger group. Some suggest that the group serves as the fundamental unit of analysis (Morgan, 1997), such that even when reporting a single response, it is being expressed in a larger social context (Hollander, 2004). Others suggest that the communication process among and across members is most important (Myers & Macnaghten, 2001). Hyden and Bulow (2003), and Kitzinger (1994), suggest that the data emerging from the group includes both individual elements and elements that emerge uniquely as members of a group. The interaction of group members produces something that is not reducible to individual members (Hyden & Bulow, 2003) nor group opinions (Albrecht, Johnson, & Walther, 1993).

 

To obtain data, most focus groups use a questioning route or discussion guide. These guides include a select group of questions or discussion points that are designed to both elicit conversation among participants and also guide their commentary to the most fruitful areas of discussion (Greenbaum, 2000; Myers & Macnaghten, 2001). The guide is designed to elicit the most compelling and telltale responses from participants. The discussion guide is often the foundation on which to base subsequent written reports (Greenbaum, 2000; Krueger, 1994).

 

At other times, the questions may be designed to stimulate discussion without directly interrogating the participants regarding the issues of interest (Hughes & DuMont, 1993; Morgan, 1997). This will include situations where the researcher has developed hypotheses or research questions that may be addressed without direct questioning (Massey, Armstrong, Boroughs, Henson, & McCash, 2005).

 

In evaluation research, focus groups have been shown to be an effective way to obtain a diverse range of information (Basch, 1987; Morgan, 1997). Focus groups may be used to answer the same type of questions as in-depth interviews, but in a social context (Armstrong & Massey, 2002; Boaz, Ziebland, Wyke, & Walker, 1998; Watson & Robertson, 1996). They are helpful in understanding how stakeholders regard specific experiences or incidents (Kitzinger & Barbour, 2001; Krueger, 1994; Wibeck, Dahlgren, & Oberg, 2007),

 

 

Three forms of qualitative analysis associated with focus groups include grounded theory, phenomenological approaches, and thematic analysis.

 

Within an evaluation framework, grounded theory and phenomenology lend themselves to action research models such as participatory and empowerment evaluation. The emphasis in these participatory approaches tend toward sharing the experiences and reality of the participants and empowering their role in partnership with evaluators (Andonian, 2008; Holte-McKenzie,

Forde, & Theobald, 2006; Nichols, 2002). The action research model includes continuing review and an emphasis on situational definitions and shared meanings (Kemmis & McTaggart, 2008; Morrison & Lilford, 2001; Trondsen & Sandaunet, 2009). As action research, the focus group participants are empowered through their mutual discovery of the meaning of their experiences. The perspectives include such principles as ownership, participation, and self-evaluation (Fetterman, 2001; Fetterman & Wandersman, 2005). Group members are active participants in discovering meaning and relevance as part of the group process. Stakeholder participation is critical, not simply as sources of collecting data, but also as sources of the meaning of data (Cousins & Whitmore, 1998; Holte-McKenzie et al., 2006).

 

A third approach, thematic analysis, offers a meaningful and common alternative for the analysis of evaluation oriented focus groups (Boyatzis, 1998; Frankland & Bloor, 2001; Webb & Kevern, 2001; Wiggins, 2004), when the intent is to understand the underlying themes and relationships that explain the organization, functioning, or impacts associated with a program (Krueger & Casey, 1998; Krueger & Casey, 2000).

 

In a review of published evaluation research using focus groups (Wiggins, 2004), thematic analysis was the most common approach to data analysis.

 

Data derived from the raw material of focus group transcripts are described as falling into three levels that fit the thematic approach to latent data analysis. These three levels of data are characterized as articulated, attributional, and emergent. This model is offered as a means to clarify the process of interpretation of focus group results and make analysis more transparent to evaluation research consumers. (p.23).

 

Articulated data is defined as that data that arises in direct response to the questions and prompts provided in the discussion guide. (p.23)

 

This data offers, in the participants’ own words, their descriptions, interpretations, and commentary on the topics of interest. At the second level, attributional data derives from comments and discussion that relate to a priori theories, operating hypotheses, or research questions that the evaluator brings to the study. This will include data applied to ‘the search for signals or indicators reflecting theories of interest’ (Boyatzis, 1998, p. 33), (p.23).

 

Finally, we reserve the term emergent data for that information that contributes to new insights and hypothesis formulation and is the unanticipated product of individual comments and exchanges among group members. (p.23).

 

It is proposed that each of these kinds of data is relevant and valuable for the evaluator and that each adds to our understanding of meaning from the group perspective. The value of distinguishing these three kinds of data lies in the evaluator’s obligation to be explicit and methodical (Patton, 1999; Ryan & Bernard, 2003) (p.23).

 

Articulated data, then, is defined as that information that is expressed in response to, or specifically addresses, the questions posed questions and probes by the moderator, as well as conversation that emerges among participants as they discuss these questions. (p.23).

 

Articulated data deals with attitudes, beliefs, observations, experiences, opinions and preferences that are all referents to the question posed by the researcher. Articulated data reflects the ‘‘language and concepts that participants use to structure their experiences’’ (Hughes & DuMont, 1993, p. 776), and reflects opportunities to ‘access the everyday language’ of participants (Bloor et al., 2001, p. 10). Articulated data will also include the participants’ comments and reactions to other members as they agree, qualify, or disagree, providing insight into the participants’ thinking (Ansay, Perkins, & Nelson, 2004). (p.23).

 

The researcher has the opportunity to consolidate, interpret and derive inferences from the articulated responses to specific questions. (p.23).

 

While surveys may address the degree to which certain standards have been met across the community, focus groups may provide clues as to what the standards are. (p.23).

 

The distinguishing feature of articulated data is its link to specific questions. The depth of analysis that is possible will depend on the quality and richness of the data that is available and on the capacity of the evaluator to make the case for the themes that are proposed.  (p.23).

 

One advantage of articulated data is the opportunity to explore the various interpretations of the questions by group members and further define or operationalize constructs (Kress & Shoffner, 2007).  The advantage of articulated data is in the depth of understanding that might be derived from the capacity of group participants to recast issues from their own perceptual framework and expand on that perspective in conversation with other intimates of the issues.  (p.24).

 

Finally, the interpretation of articulated data is the most easily defended. Questions are asked and answers provided.  Articulated data emerging from the group requires no dissembling; the purpose of the group is clearly described and reflected in the questions asked of participants, and the purpose of questioning and the aims of the study may be shared (cf. Mc Lafferty, 2004). Thus, the credibility of findings taken directly from participants is enhanced (Thayer & Fine, 2001). (p.24).

 

Quotations may be used to corroborate findings (Wiggins, 2004). The researcher may present discrete examples such as ‘‘the focus group question asked this . . .’’ and the ‘‘participant(s) said this . . .’’ The interpretive narration of the researcher should be explicit and generally defensible from the examples that are made available in the participants’ own words. (p.24).

 

Articulated data is dependent on the questions posed in the group. The value of articulated data rests with the value of questions asked. If questions are confusing or irrelevant, or if the moderator fails to provide useful and meaningful prompts, the focus group may fail to capture the participants’ meanings. Poor questions may fail to cultivate any new insights, leaving the evaluator scrambling to draw meaning from sparse communication. (p.24).

 

With emergent data we move from more ‘‘theory-driven to data-driven approaches’’ (Boyatzis, 1998, p. 29). The use of the term emergent data is reserved for that information related to group meanings, processes, and norms that add new insights and generate new hypothesis and is the unanticipated product of comments and exchanges of group members. (p.25).

 

These issues are ordinarily neither articulated through the questioning route, nor hypothesized for attribution, but rather arise as a new insight, borne not of what was asked, but what was either found to be important by the participants, or that emerges from the underlying individual, social, or cultural issues touched upon by what was asked. (p.25).

 

This data touches on the larger themes and unifying concepts that are invisible before the study begins but that offer explanatory power for events related by the group. Emergent themes will include those that have not been hypothesized, the unasked questions that seem to be addressed in the stories, anecdotes, explanations, and conversations among participants. A strength of the focus group method is the capacity to uncover the unique experiential data that determines the complexity of social situations. (p.25).

 

The strength of emergent data is the capacity it has to allow us to come to a greater understanding of those often unspoken social and normative values that underlie our attitudes, beliefs and behaviors. What we gain is the chance to explore those meanings and interpretations that move beyond our preconceived theories and propositions and that allow us to anchor knowledge of social and psychological processes in the norms and experiences of the population (Hughes & DuMont, 1993). Emergent data is also closely associated with the interactions among participants, and the unique aspects of the group process (Wibeck et al., 2007). (p.26).

 

One potential limitation of emergent data is its more tentative foundation. As we move from more theory-driven to more data driven, or inductive, processes, we decrease the likelihood of consistency in judgment (Boyatzis, 1998). At each level, it is incumbent on the researcher to establish the veracity of the interpretation by demonstrating the sources of data that lead to the researchers’ conclusions. (p.26).

 

On occasion, the focus group moderator may have the opportunity to explore themes that arise in the conversation in order to gain a greater understanding of the statements. (p.26)

 

emergent themes may only show themselves after the transcripts are analyzed. The difficulty is not in finding new themes in the expressions of the participants, but rather of finding sufficient and consistent data to justify drawing conclusions about these themes. (p.26).

 

success in interpreting emergent data depends on greater trust in the analysis (p.26).

 

The juxtaposition of information available at the three levels offers insight into the complex meaning of experiences. (p.26).

 

Second, the data should not be considered equivalent in its ease of interpretation. Articulated data is the most relevant for the questions asked and requires the least interpretation from the researcher. It is the least ambiguous data set, and has the greatest ‘face validity’ (Anastasi, 1988). If we have asked the right questions, (and in the right way), articulated data should provide the best answers to those questions. (p.26).

 

Attributional data is more difficult to interpret. Attributional data requires a more explicit description of propositions or hypotheses that may then be tested against evidence provided in the participants’ commentary. The evaluator has the advantage, however, of being able to control group membership, construct the discussion guide, and shape the conversation among participants, in order to increase opportunities for attributional data to emerge. (p.26).

 

Third, it is not proposed that articulated and attributional data are purely deductive, while emergent data is purely inductive. Attributional data is more likely to be deductive by design, as we will likely have some propositional framework by which to test our hypotheses. (p.26).

 

Thematic analysis of articulated and emergent data will likely be a blend of both inductive and deductive processes, with emergent data most clearly inductive. (p.27).

 

Evaluators have the opportunity to make their intentions explicit and thus identify the kind of data they anticipate will be most relevant for their purpose. (p.27).

 

It should be clear from the previous discussion that preparation for focus group analysis should not be an entirely post-hoc event. The simplest guideline is to plan the focus group based on the specific needs of the evaluation and where the most relevant data is likely to be found. The construction of the guide and the questions, probes, and prompts offered by the moderator should reflect the purpose of the effort. The composition and number of groups will be partly determined by any tests of attributional if then propositions. Questions and prompts must be developed to shape conversation to the questions and contrasts of interest for the evaluation. (p.27).

 

For example, if a primary concern is gaining a greater understanding of how participants experience stigma, then an emphasis on articulated data would suggest the use of a direct questioning (p.27)

 

An evaluator who wishes to test the hypothesis that family members experience stigma differently than consumers, must establish the format of the focus group(s) to allow these contrasts to occur. (p.27)

 

Emergent data may also be revealed in unspoken assumptions that arise as part of other conversation. (p.27).

 

Focus groups offer an opportunity to obtain significant insight regarding the experiences, observations and opinions of group members. (p.27).

 

If a participant articulates an attitude, belief, or opinion, makes an observation, or relates an experience as part of a focus group discussion, there is something of value in that articulation. The evaluator has information that was not available before. And the summary of the attitudes, beliefs, and opinions of the participants, taken in the larger context with the statements of others, can provide critical new insights and lead to a greater understanding of the topics of interest. (p.27).

 

The exposition of each of these levels of data allows the researcher greater clarity in relating the findings of focus group research, and allows the reader a better understanding of the kinds of conclusions and evidence that might be expected from such an analysis. It is suggested that distinguishing among three levels of data serve to increase the specificity and transparency of the data analysis process, and will serve to increase evaluation consumers understanding, appreciation, and faith in the conclusions drawn from such analysis. (p.27).