Quality and trustworthiness in qualitative research in counseling psychology. By way of a summary, the frequency with which these were deployed across the three journals is indicated in Table3. According to the original Grounded Theory texts, data collection should continue until there are no new discoveries (i.e., data saturation; Glaser & Strauss, 1967). Luborsky MR, Rubinstein RL. Gergen KJ, Josselson R, Freeman M. The promises of qualitative inquiry. A systematic analysis of single-interview-per-participant designs within three health-related journals from the disciplines of psychology, sociology and medicine, over a 15-year period, was conducted to examine whether and how sample sizes were justified and how sample size was characterised and discussed by authors. And whilst a large sample size was endorsed and valued by some qualitative researchers, within the psychological tradition of IPA, a large sample size was counter-normative and therefore needed to be justified. Res Nurs Health. However, 40% did not report the number of participants. Number of eligible articles published each year per journalFootnote. Other articles appeared to accept and acknowledge that their sample was flawed because of its small size (as well as other compositional deficits e.g. Other work has sought to examine practices of sample size reporting and sufficiency assessment across a range of disciplinary fields and research domains, from nutrition [34] and health education [32], to education and the health sciences [22, 27], information systems [30], organisation and workplace studies [33], human computer interaction [21], and accounting studies [24]. These IPA studies illustrate that the characterisation of sample size sufficiency can be a function of researchers theoretical and epistemological commitments rather than the result of an objective sample size assessment. L X kFAd F%%d.KhXZ0`qI$ Despite these limitations, the analysis of the characterisation of sample size and of the threats seen to accrue from insufficient sample size, enriches our understanding of sample size (in)sufficiency argumentation by linking it to other features of the research. Expert Rev Pharmacoecon Outcomes Res. Ritchie et al. Qual Quant. (I Cant get no) saturation: a simulation and guidelines for sample sizes in qualitative research. (BJHP13). Morse JM. Similarly, Britten [40] notes that large interview studies will often comprise of 50 to 60 people. These data may be seen as an opportunity to generate further hypotheses and are a unique application of the medicalisation framework. ACM. The material collected was considerable and, given the focused nature of the study, extremely detailed. Sobal J. data, theoretical, thematic saturation) (SHI04; SHI13; SHI30) whilst another four articles explicitly stated that they achieved saturation (SHI100; SHI125; SHI136; SHI137). BMJ. Data were saturated. Whilst the quantitative research community has established relatively straightforward statistics-based rules to set sample sizes precisely, the intricacies of qualitative sample size determination and assessment arise from the methodological, theoretical, epistemological, and ideological pluralism that characterises qualitative inquiry (for a discussion focused on the discipline of psychology see [12]). The present study sought to enrich existing systematic analyses of the customs and practices of sample size reporting and justification by focusing on qualitative research relating to health. Greenhalgh T, Annandale E, Ashcroft R, Barlow J, Black N, Bleakley A, et al. Similarly, BJHP04, which did not provide a sample size justification, argued that it targeted a larger sample size in order to eventually recruit a sufficient number of interviewees, due to anticipated low response rate. Sample sizes of published literature in the area of the subject matter under investigation (3.5% of all justifications) were used by 2 BMJ articles as guidance and a precedent for determining and defending their own sample size (BMJ08; BMJ15 see extract in section Pragmatic considerations). Lincoln YS, Guba EG. Given that older adults have been ignored in the research concerning suntanning, fair-skinned older adults are the most likely to experience skin cancer, and women privilege appearance over health when it comes to sunbathing practices, our study offers depth and richness of data in a demographic group much in need of research attention. Thousand Oaks, CA: Sage; 1998. 0000007440 00000 n How theory is used and articulated in qualitative research: development of a new typology. Determining sample size. Others investigated PhD qualitative studies [31] and grounded theory studies [35]. Our lack of epidemiological knowledge about healthcare incidents, however, means that determining an appropriate sample size continues to be difficult. 2001. Focusing on single-interview-per-participant qualitative designs, the present study aims to further contribute to the dialogue of sample size in qualitative research by offering empirical evidence around justification practices associated with sample size. In Proceedings of the 2016 CHI conference on human factors in computing systems. 0000034516 00000 n Only one article (SHI30) used the term transferability to argue for the potential of wider relevance of the results which was thought to be more the product of the composition of the sample (i.e. Determination of sample size so that it is in line with, and serves the requirements of, the research design (2.3% of all justifications) that the study adopted was another justification used by 2 BMJ papers (BMJ16; BMJ08 see extract in section In line with existing research). 30, 35], might be based on research that does not provide adequate evidence of sample size sufficiency. London: Sage; 2003. p. 77108. 2017;17(5):57188. The method of data collection had to be individual, synchronous qualitative interviews (i.e. Providing a sample size justification was not related to the number of interviews conducted, but it was associated with the journal that the article was published in, indicating the influence of disciplinary or publishing norms, also reported in prior research [30]. This sample size justification (8.4% of all justifications) was mainly employed by BJHP articles and referred to an intensive, idiographic and/or latently focused analysis, i.e. In the BMJ, two studies claimed that they achieved data saturation (BMJ17; BMJ18) and one article referred descriptively to achieving saturation without explicitly using the term (BMJ13). quantitative calculating [25] recommend that researchers present cumulative frequency graphs supporting their judgment that saturation was achieved. Three articles described a state of achieved saturation without using the term or specifying what sort of saturation they had achieved (i.e. On the basis of this analysis, the categories that expressed qualitatively different sample size justifications were developed. Often unjustified (and in two cases incongruent with their own claims of saturation) these findings imply that sample size in qualitative health research is often adversely judged (or expected to be judged) against an implicit, yet omnipresent, quasi-quantitative standpoint. 0000034683 00000 n They noted that their sample was relatively homogeneous, their research aims focused, so studies of more heterogeneous samples and with a broader scope would be likely to need a larger size to achieve saturation. The authors further proposed two main principles upon which specification of saturation be based: (a) researchers should a priori specify an initial analysis sample (e.g. Nonetheless, by the last few interviews, significant repetition of concepts was occurring, suggesting ample sampling. 1995;18(2):17983. respondents reproductive quantitative matrilineal Methodological studies in this area aim to provide guidance about saturation and develop a practical application of processes that operationalise and evidence saturation. When authors justified their sample size, our findings indicate that sufficiency was mostly appraised with reference to features that were intrinsic to the study, in agreement with general advice on sample size determination [4, 11, 36]. Thus qualitative health research constitutes an emblematic case that may help to unfold underlying philosophical and methodological differences across the scientific community that are crystallised in considerations of sample size. Despite this, although nuances within sub-themes were still emerging towards the end of data analysis, the themes themselves were being replicated indicating a level of completeness. https://doi.org/10.1186/s12874-018-0594-7, DOI: https://doi.org/10.1186/s12874-018-0594-7. In total, 214 articles 21 in the BMJ, 53 in the BJHP and 140 in the SHI were eligible for inclusion in the review. (BMJ15). This research was initially conceived of and partly conducted with financial support from the Multidisciplinary Assessment of Technology Centre for Healthcare (MATCH) programme (EP/F063822/1 and EP/G012393/1). One paper claimed theoretical saturation (BMJ06) conceived as being when no further recurring themes emerging from the analysis whilst another study argued that although the analytic categories were highly saturated, it was not possible to determine whether theoretical saturation had been achieved (BMJ04). (SHI81). Fusch PI, Ness LR. J Nutr Educ. London: Sage; 2006. the point at which no additional issues are identified) was achieved at 9 interviews, but meaning saturation (i.e. Similarly, fewer than 10% of articles in organisation and workplace studies provided a sample size justification relating to existing recommendations by methodologists, prior relevant work, or saturation [33], whilst only 17% of focus groups studies in health-related journals provided an explanation of sample size (i.e. Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group. Finally, SHI112 argued that once it had achieved saturation of discursive patterns, further sampling was decided and conducted to check for consistency of the findings. (BMJ08). Marshall MN. 0000014581 00000 n BJHP17 referred descriptively to a state of achieved saturation without specifically using the term. Examining additional disciplines (e.g. (SHI01). Google Scholar. 2015;25(5):5878. the point at which no further dimensions, nuances, or insights of issues are identified) required 1624 interviews. Further, these data do not need to be statistically generalisable for us to draw inferences that may advance medicalisation analyses (Charmaz 2014). This large-scale format of IPA has been used by other researchers in the field of false-positive research. : a review of qualitative interviews in IS research. pension quantitative easing OReilly M, Parker N. Unsatisfactory saturation: a critical exploration of the notion of saturated sample sizes in qualitative research. More specifically, six articles defended their sample size on the basis of an intensive analysis of transcripts and/or the idiographic focus of the study/analysis. Similarly, whilst methodological research that seeks to demonstrate the achievement of saturation is invaluable since it explicates the parameters upon which saturation is contingent and indicates when a research project is likely to require a smaller or a larger sample [e.g. Saunders MN, Townsend K. Reporting and justifying the number of interview participants in organization and workplace research. Saturation often under the terms of data or thematic saturation has diffused into several qualitative communities beyond its origins in grounded theory. Qualitative inquiry and research design: choosing among five approaches. sample size) conducted and the provision of a justification (rpb=.054, p=.433). In the BMJ, one article (BMJ15) appealed to pragmatic reasons, relating to time constraints and the difficulty to access certain study populations, to justify the determination of its sample size. number of focus groups), with saturation being the most frequently invoked argument, followed by published sample size recommendations and practical reasons [22]. JOAAG. 2nd ed. Whilst community norms and prior research can establish useful rules of thumb for estimating sample sizes [60] and reveal what sizes are more likely to be acceptable within research communities researchers should avoid adopting these norms uncritically, especially when such guidelines [e.g. 1995;5(2):1479. BMJ01, BJHP02, SHI03). 2010. Transforming qualitative information: thematic analysis and code development. respondents frequency (BJHP41). 2014;11(1):2541. non-representativeness, biases, self-selection) or anticipated that they might be criticized for their small sample size. A number of articles which characterised their sample size as small did so against an implicit or explicit quantitative framework of reference. (SHI35). Qual Res. Researchers have proposed 30 as an approximate or working number of interviews at which one could expect to be reaching theoretical saturation when using a semi-structured interview approach (Morse 2000), although this can vary depending on the heterogeneity of respondents interviewed and complexity of the issues explored. Ogden J, Cornwell D. The role of topic, interviewee, and question in predicting rich interview data in the field of health research. Are we there yet? In terms of principles, Lincoln and Guba [17] proposed that sample size determination be guided by the criterion of informational redundancy, that is, sampling can be terminated when no new information is elicited by sampling more units. Konstantina Vasileiou. National Centre for Research Methods Review Paper. This study could be criticised for using a small and unrepresentative sample. He is unaware of any direct conflict of interest with respect to this paper. Accessed 17 May 2018. Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Nevertheless, claims of saturation were never substantiated in relation to procedures conducted in the study itself, endorsing similar observations in the literature [25, 30, 47]. Qual Health Res. The funding body did not have any role in the study design, the collection, analysis and interpretation of the data, in the writing of the paper, and in the decision to submit the manuscript for publication. Manage cookies/Do not sell my data we use in the preference centre. Sandelowski M. One is the liveliest number: the case orientation of qualitative research. PLoS One. As the peer-review process becomes increasingly public, future research could usefully examine how reporting around sample size sufficiency and data adequacy might be influenced by the interactions between authors and reviewers. This gave a target sample of 75100 patients and 3550 caregivers overall. An examination of 83 qualitative interview studies in leading information systems journals [30] indicated little defence of sample sizes on the basis of recommendations by qualitative methodologists, prior relevant work, or the criterion of saturation. Spencer L, Ritchie J, Lewis J, Dillon L. Quality in qualitative evaluation: a framework for assessing research evidence. 0000012498 00000 n Similarly, BJHP38 (see extract in section Qualities of the analysis) claimed that its sample size was within the range of sample sizes of published studies that use its analytic approach. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Glenton C, Carlsen B, Lewin S, Munthe-Kaas H, Colvin CJ, Tunalp , et al. If an article was published in the BJHP, the odds of providing a justification were 4.8 times higher than if published in the SHI. Sample size in qualitative interview studies: guided by information power. As shown in Table2, the majority of articles which justified their sample size provided one justification (70% of articles); fourteen studies (25%) provided two distinct justifications; one study (1.7%) gave three justifications and two studies (3.3%) expressed four distinct justifications. 0000006193 00000 n The other two BJHP articles conducted thematic analysis (BJHP34; BJHP38). What is an adequate sample size? D#HtTZH+X$u & Qual Res. Expert Rev Med Devices. Once this sample had been analysed, one further interview from each age-stratified group was randomly chosen to check for consistency of the findings. Sample size in grounded theory cannot be determined a priori as it is contingent on the evolving theoretical categories. Field Methods.