Methods that calculate saturation based on the proportion of new themes relative to the overall number of themes in a dataset (e.g., Guest et al. Quantitative data analysis. How both quantitative and qualitative data is collected and Analysed? For each resample, we calculated the proportion of new themes found in run lengths of two or three new events relative to a base size of four, five or six interviews. and Namey et al. In cases where most of the numbers are quite close to the average, the standard deviation will be relatively low. We use cookies to ensure that we give you the best experience on our website. How to Analyze Qualitative Data from UX Research: Thematic Analysis It is a method for identifying, analyzing, organizing, describing, and reporting themes found within a data set (Braun & Clarke, 2006). This study included 40 individual interviews with African American men in the Southeast US about their health seeking behaviors [29]. Since we had available the total number of codes identified in each dataset, we carried out one additional calculation as a way to provide another metric to understand how the median number of interviews to reach a new information threshold related to retrospectively-assessed degrees of saturation with the entire dataset. No, Is the Subject Area "Cross-cultural studies" applicable to this article? Get a 14-day fully-featured trial. Divide the number of new themes in this run (seven) by the number of unique themes in the base set (37). Thematic analysis describes an iterative process as to how to go from messy data to a map of the most important themes in the data. Thematic analysis is a method of analyzing qualitative data. What a quantitative researcher accepts, for example, as a large enough effect size or a small enough p-value is a subjective determination and based on convention in a particular field of study. This can be particularly useful when the data is complex or when the researcher is looking to uncover relationships between the data and other variables. For example, Onwuegbuzie and Johnson (2021) note "data analysis in mixed methods research [can be]the most difficult step of the mixed methods research process" (p. 1) and there is a "lack of methodological guidance in the extant literature on these topics" (p. 16). While methods of data collection and data analysis represent the core of research methods, you have to address a range of additional elements within the scope of your research. A key advantage is that the metrics are flexible, affording researchers the ability to choose different degrees of rigor by selecting different run lengths and/or new information thresholds. Samples and populations can both be represented by a qualitative variable and/or a quantitative variable, because the definition of sample and population does not include anything about the type of variables that can be used within it. In other words, what is the minimum number of data collection events (i.e., interviews) we should review/analyze to calculate the amount of information already gained? We offer personal insights and practical examples, while exploring issues of rigor and trustworthiness. During the past two decades, scholars have conducted empirical research and developed mathematical/statistical models designed to estimate the likely number of qualitative interviews needed to reach saturation for a given study. For example, a researcher may use thematic analysis to analyze a survey data set in order to identify themes in the responses. Revised on The interview guide contained 13 main questions, each with scripted sub-questions. Thematic analysis is one of the most frequently used qualitative analysis approaches. Calculation is simple. Thematic analysis: A practical guide - Melissa Forbes, 2022 - SAGE Journals The number of new themes in this first run is seven. They defined the term as the point at which no additional data are being found whereby the [researcher] can develop properties of the category (pg. The number of new themes found in the run defines the numerator in the saturation ratio. Summarizing a quantitative study is relatively clear: you scored 25% better than the competition, let's say. However, thematic analysis is a flexible method that can be adapted to many different kinds of research. Thematic analysis allows you a lot of flexibility in interpreting the data, and allows you to approach large data sets more easily by sorting them into broad themes. When assessing saturation, incoming information is weighed against the information already obtained. No, PLOS is a nonprofit 501(c)(3) corporation, #C2354500, based in San Francisco, California, US, Corrections, Expressions of Concern, and Retractions, https://doi.org/10.1371/journal.pone.0232076. Have a human editor polish your writing to ensure your arguments are judged on merit, not grammar errors. For more information about PLOS Subject Areas, click However, it also involves the risk of missing nuances in the data. Yes thematic analysis can be used in mixed method research. Abstract. This process was originally developed for psychology researchby Virginia Braun and Victoria Clarke. Citation: Guest G, Namey E, Chen M (2020) A simple method to assess and report thematic saturation in qualitative research. [The data used for each step are included in Fig 3, along with indication of the base, runs, and saturation points. Interested in ChatGPT For 1-on-1 Interviews? 2022 - 2023 Times Mojo - All Rights Reserved It is similar to a literature review, which is a critical survey and assessment of the existing research on your particular topic. The two branches of quantitative analysis, How to choose the right quantitative methods. No credit card is required. Following this process can also help you avoid confirmation bias when formulating your analysis. What qualitative and quantitative data have in common with one and another? Here again the unit of analysis is the data collection event; the items of analysis are unique codes. In this broader sense, saturation is often described as the point in data collection and analysis when new incoming data produces little or no new information to address the research question [4, 9, 1113]. The concept of saturation was first introduced into the field of qualitative research as theoretical saturation by Glaser and Strauss in their 1967 book The Discovery of Grounded Theory [10]. . The purpose of this article is to guide researchers using thematic analysis as a research method. PLoS ONE 15(5): PDF Answers to frequently asked questions about thematic analysis Thematic analysis is frequently used to analyse qualitative data in psychology, healthcare, social research and beyond. In our example, we might start combining codes into themes like this: At this stage, we might decide that some of our codes are too vague or not relevant enough (for example, because they dont appear very often in the data), so they can be discarded. Why Do Cross Country Runners Have Skinny Legs? Abstract. At a run length of three interviews, the median number of interviews required before a drop in new information was observed was seven. Thematic analysis helps researchers understand those aspects of a phenomenon that participants talk about frequently or in depth, and the ways in which those aspects of a phenomenon may be connected. Thematic analysis is often quite subjective and relies on the researchers judgement, so you have to reflect carefully on your own choices and interpretations. How do I undertake thematic analysis in systematic review? We should also include a methodology section, describing how we collected the data (e.g. Themes are generally broader than codes. First, it can help researchers to identify relationships between the data and other variables. The number of new themes evident across 812 interviews corresponded with a median degree of saturation of 62% to 71%. If our findings hold true in other contexts, it suggests that using a default base size of four interviews is sufficient. [26], Fugard & Potts [21], Galvin [20]) ignores the fact that most qualitative research employs non-probabilistic, purposive sampling suited to the nature and objectives of qualitative inquiry [28]. But they do provide a transparent way of presenting data saturation assessments that can be subsequently interpreted by other researchers. Height in feet, age in years, and weight in pounds are examples of quantitative data. Dataset 3. We could increase the run length to 3 (or an even larger number), and/or we could set a more stringent new information threshold of no new information. https://doi.org/10.1371/journal.pone.0232076.s001. The number of themes evident across 68 interviews corresponded with a median degree of saturation of 79% to 82%. The 5th and 95th percentiles provide a nonparametric 90% confidence interval for the number of transcripts needed to reach saturation as defined at these new information thresholds. They provide empirically-based guidance for approximating how many qualitative interviews might be needed for a given study and serve a role analogous to power calculations in quantitative research design (albeit in some case without the math and degree of precision). Our method also specifically applies to contexts in which an inductive thematic analysis [24] is used, where emergent themes are discovered in the data and then transformed into codes. You can use quantitative analysis to interpret data that was collected either: During an experiment. Likewise, while an odds ratio of 1.2 may be statistically significant, whether or not its meaningful in a real-world sense is entirely open to interpretation. Thematic analysis is a powerful tool that can be used in quantitative research. Across academic disciplines, and for about the past five decades, the answer to this question has usually revolved around reaching saturation [1, 59]. https://doi.org/10.1371/journal.pone.0232076.g002. Thematic analysis is used in qualitative research and focuses on examining themes or patterns of meaning within data. Can You Do Thematic Analysis In Quantitative Research? At the 0% new information threshold, the median number of interviews to indicate saturation were again consistent across bases sizes, varying only by the run length. We are advocating for similar flexibility and transparency in assessing and reporting on thematic saturation. Global Health, Population, and Nutrition, FHI 360, Durham, North Carolina, United States of America. Following a review of the empirical research on data saturation and sample size estimation in qualitative research, we propose an alternative way to evaluate saturation that overcomes the shortcomings and challenges associated with existing methods identified in our review. The process contains six steps: Familiarize yourself with your data. This is done by randomly resampling from the sample with replacement (i.e., an item may be selected more than once in a resample) many times in a way that mimics the original sampling scheme. The results section of the review will comprise thematic synthesis of quantitative studies, thematic synthesis of qualitative studies, and the aggregation of the two. Validation, We will prospectively calculate saturation using a base size of 4 interviews and run length of 2 interviews. A latent approach involves reading into the subtext and assumptions underlying the data. To choose the right statistical methods and techniques, you need to consider the, Qualitative Data Analysis Methods 101: The Big 6 Methods (Including Examples), Sampling Methods & Strategies 101: What You Need To Know, Qualitative Data Coding 101: Everything You Need To Know. For any method of assessing saturation, including ours, researchers need to decide at which level they will identify and include themes/codes. Here, we return to the data set and compare our themes against it. One may still raise two logical questions after reviewing the example process above. As one reason for the growth in qualitative synthesis is what they can add to quantitative reviews, it is not surprising that there is also growing interest in mixed methods reviews. Discrepancies in code application were resolved through discussion, resulting in consensus-coded documents. Powered by Speak Ai Inc. Made in Canada with. Their tool estimates, for example, that to have 80% power to detect two instances of a theme with a 10% prevalence in a population, 29 participants would be required. This is compounded by the fact that detailed descriptions of methods are often omitted from qualitative discussions. In this paper we present a way of assessing thematic saturation in inductive analysis of qualitative interviews. The interview was a follow-up qualitative inquiry into womens responses on a quantitative survey. At the 0% new information threshold saturation was indicated at the same points as in Dataset 1: 11+2 and 14+3, consistent across all base sizes. Qualitative research: A guide to design and implementation. Boyatzis (1998) described thematic analysis as a translator for those speaking the languages of qualitative and quantitative analysis . It is present in all qualitative research but, unfortunately, it is evident mainly by declaration [1]. Can research be done using both qualitative and quantitative data Why or why not? Here again, despite a different total number of themes in the overall dataset, the number of new themes evident across 1114 interviews corresponded with a median degree of saturation of 87% to 89%. We hope researchers find this method useful, and that others build on our work by empirically testing the method on different types of datasets drawn from diverse study populations and contexts. Qualitative Methods for Policy Analysis: Case Study Research Strategy Estimates are based on (specified) assumptions, and expectations regarding various elements in a particular study. Each time they find a word, they make a copy of it and its immediate context. However, the purpose and process of identifying themes may differ based not only on the methodology but also the research questions (Braun & Clarke, 2006). Roles An extract from one interview looks like this: In this extract, weve highlighted various phrases in different colors corresponding to different codes. Thematic analysis is an important tool for researchers to consider when conducting quantitative research. Across base sizes, for a run length of two, we would say that saturation was indicated at 6+2, while for a run length of three we would say saturation was observed at 7+3, both at the 5% new information level. [22]). [17] and Coenen et al. Yes Using a meta-analytic approach, Galvin [20] reviewed and statistically analyzedusing binomial logic54 qualitative studies. They further argue that the stopping point for an inductive study is typically determined by the judgement and experience of researchers. The results from the bootstrapping analyses are presented by dataset, in Tables 2, 3 and 4. How many qualitative interviews are enough? We want to create potential themes that tell us something helpful about the data for our purposes. Interested in Can Thematic Analysis Be Used In Quantitative Research?? Results will aim to offer an account of current understandings of patient experiences and perspective regarding PICC, Hickman-type, and Port devices in the context of anti-cancer . Now we have to make sure that our themes are useful and accurate representations of the data. Yes The new information threshold selected affects the point at which saturation is indicated, as one would expect. When to use thematic analysis. Inductive probing was employed throughout all interviews. Data saturation is the most commonly employed concept for estimating sample sizes in qualitative research. [9] conducted a stepwise inductive thematic analysis of 60 in-depth interviews among female sex workers in West Africa and discovered that 70% of all 114 identified themes turned up in the first six interviews, and 92% were identified within the first 12 interviews. This means that relative to the total number of unique codes identified in the first four, five, or six interviews, the amount of new information contributed by interviews 7 and 8 was less than or equal to 5% of the total. Substantially more women from the Kenya sample were married and living with their partners (63% versus 3%) and were less likely to have completed at least some secondary education. The approach to analyze data in qualitative research cannot be the same as in the quantitative research. Some types of research questions you might use thematic analysis to answer: It can be done quickly and with no statistical expertise. Thematic analysis can be used to analyze quantitative data sets in order to identify patterns and themes. Inductive probing was employed throughout all interviews. https://doi.org/10.1371/journal.pone.0232076.t005. The metrics are flexible. We would annotate these two extra interviews (indicative of run length) by appending a superscript +2 to the interview number, to indicate a total of eight interviews were completed. Check out the dedicated article the Speak Ai team put together on ChatGPT For Academic Textbooks to learn more. Thematic analysis is a popular six-phased approach to analysing qualitative data; however, very few studies adopting this approach have explicitly demonstrated step-by-step and explained the whole . Scribbr. The interviews following Interview 12, though yielding four additional themes, remained at or below the 5% new information threshold. It is not a research method in itself but rather an analytic approach and synthesizing strategy used as part of the meaning-making process of many methods, including case study research. Each code describes the idea or feeling expressed in that part of the text. This body of work has advanced the evidence base for sample size estimation in qualitative inquiry during the design phase of a study, prior to data collection, but it does not provide qualitative researchers with a simple and reliable way to determine the adequacy of sample sizes during and/or after data collection. To provide the foundation for this approach, we define saturation and then review the work to date on estimating saturation and sample sizes for in-depth interviews. Mixed methods dissertations combine qualitative and quantitative approaches to research. to learn more. This can help the researcher to better understand the data and draw more meaningful conclusions. For the next run we add the new themes for the next two interviews, 6 and 7 (note the overlap of interview 6), resulting in a sum of four. It is usually applied to a set of texts, such as interview transcripts. comparing reflexive thematic analysis and other patternbased qualitative analytic approaches. Quantitative data is data that can be counted or measured in numerical values. Formal analysis, While these three studies offer diverse and analytically rigorous case studies, they provide limited generalizability. The quotient reveals 19% new information. The same can be said for how a researcher chooses to report and interpret statistical findings. How to achieve trustworthiness in thematic analysis. There are several benefits to using thematic analysis in quantitative research. Copyright: 2020 Guest et al. https://doi.org/10.1371/journal.pone.0232076.t002. Theres also the distinction between a semantic and a latent approach: Ask yourself: Am I interested in peoples stated opinions (semantic) or in what their statements reveal about their assumptions and social context (latent)? Their calculation incorporates: (1) the estimated prevalence of a theme within the population, (2) the number of desired instances of that theme, and (3) the desired power for a study. Protocol for a systematic review and thematic synthesis of patient Conceptualization, We then identified the number of transcripts needed to meet a new information threshold of 5% or 0%. Central to content analysis is the process of coding, which . EMMY NOMINATIONS 2022: Outstanding Limited Or Anthology Series, EMMY NOMINATIONS 2022: Outstanding Lead Actress In A Comedy Series, EMMY NOMINATIONS 2022: Outstanding Supporting Actor In A Comedy Series, EMMY NOMINATIONS 2022: Outstanding Lead Actress In A Limited Or Anthology Series Or Movie, EMMY NOMINATIONS 2022: Outstanding Lead Actor In A Limited Or Anthology Series Or Movie. At this stage, we want to be thorough: we go through the transcript of every interview and highlight everything that jumps out as relevant or potentially interesting. Thematic analysis is a method for analyzing qualitative data that entails searching across a data set to identify, analyze, and report repeated patterns (Braun and Clarke 2006). In other words, no new information was observed after a median of 11 interviews using a run-length of 2, nor after 14 interviews using a run length of 3. Not surprisingly, researchers have recently begun asking two follow up questions about data saturation that go beyond estimation: How can we better operationalize the concept of saturation? We have provided researchers with a method to easily calculate saturation during or after data collection. Base size refers to how we circumscribe the body of information already identified in a dataset to subsequently use as a denominator (similar to Francis et al.s initial analysis sample). A thematic analysis can uncover the major themes from your research. In practical terms, this implies that saturation should initially be assessed after six interviews (four in the base, and two in the run). Coding means highlighting sections of our text usually phrases or sentences and coming up with shorthand labels or codes to describe their content. Generate accurate APA, MLA, and Chicago citations for free with Scribbr's Citation Generator. We subsequently propose an alternative way of evaluating saturation and offer a relatively easy-to-use method of assessing and reporting on it during or after an inductive thematic analysis. When collecting and analyzing data, quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings. For this third run we add the number of new themes identified within interviews 7 and 8. https://doi.org/10.1371/journal.pone.0232076.t007. Which Teeth Are Normally Considered Anodontia? Can qualitative and quantitative variables be used to describe both samples and populations? Research Methods | Definitions, Types, Examples - Scribbr Empirical research to address this issue began appearing in the literature in the early 2000s. The two "branches" of quantitative analysis. As Morse pointed out more than 20 years ago, saturation is an important component of rigor. Framework analysis involves a systematic analysis of content summaries of participants views/experiences in the latter stage.