characteristics of qualitative research design are

Qualitative Research: Characteristics, Design, Methods & Examples

Lauren McCall

MSc Health Psychology Graduate

MSc, Health Psychology, University of Nottingham

Lauren obtained an MSc in Health Psychology from The University of Nottingham with a distinction classification.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Qualitative research is a type of research methodology that focuses on gathering and analyzing non-numerical data to gain a deeper understanding of human behavior, experiences, and perspectives.

It aims to explore the “why” and “how” of a phenomenon rather than the “what,” “where,” and “when” typically addressed by quantitative research.

Unlike quantitative research, which focuses on gathering and analyzing numerical data for statistical analysis, qualitative research involves researchers interpreting data to identify themes, patterns, and meanings.

Qualitative research can be used to:

• Gain deep contextual understandings of the subjective social reality of individuals
• To answer questions about experience and meaning from the participant’s perspective
• To design hypotheses, theory must be researched using qualitative methods to determine what is important before research can begin. 

Examples of qualitative research questions include: 

• How does stress influence young adults’ behavior?
• What factors influence students’ school attendance rates in developed countries?
• How do adults interpret binge drinking in the UK?
• What are the psychological impacts of cervical cancer screening in women?
• How can mental health lessons be integrated into the school curriculum? 

Characteristics 

Naturalistic setting.

Individuals are studied in their natural setting to gain a deeper understanding of how people experience the world. This enables the researcher to understand a phenomenon close to how participants experience it. 

Naturalistic settings provide valuable contextual information to help researchers better understand and interpret the data they collect.

The environment, social interactions, and cultural factors can all influence behavior and experiences, and these elements are more easily observed in real-world settings.

Reality is socially constructed

Qualitative research aims to understand how participants make meaning of their experiences – individually or in social contexts. It assumes there is no objective reality and that the social world is interpreted (Yilmaz, 2013). 

The primacy of subject matter 

The primary aim of qualitative research is to understand the perspectives, experiences, and beliefs of individuals who have experienced the phenomenon selected for research rather than the average experiences of groups of people (Minichiello, 1990).

An in-depth understanding is attained since qualitative techniques allow participants to freely disclose their experiences, thoughts, and feelings without constraint (Tenny et al., 2022). 

Variables are complex, interwoven, and difficult to measure

Factors such as experiences, behaviors, and attitudes are complex and interwoven, so they cannot be reduced to isolated variables , making them difficult to measure quantitatively.

However, a qualitative approach enables participants to describe what, why, or how they were thinking/ feeling during a phenomenon being studied (Yilmaz, 2013). 

Emic (insider’s point of view)

The phenomenon being studied is centered on the participants’ point of view (Minichiello, 1990).

Emic is used to describe how participants interact, communicate, and behave in the research setting (Scarduzio, 2017).

Interpretive analysis

In qualitative research, interpretive analysis is crucial in making sense of the collected data.

This process involves examining the raw data, such as interview transcripts, field notes, or documents, and identifying the underlying themes, patterns, and meanings that emerge from the participants’ experiences and perspectives.

Collecting Qualitative Data

There are four main research design methods used to collect qualitative data: observations, interviews,  focus groups, and ethnography.

Observations

This method involves watching and recording phenomena as they occur in nature. Observation can be divided into two types: participant and non-participant observation.

In participant observation, the researcher actively participates in the situation/events being observed.

In non-participant observation, the researcher is not an active part of the observation and tries not to influence the behaviors they are observing (Busetto et al., 2020). 

Observations can be covert (participants are unaware that a researcher is observing them) or overt (participants are aware of the researcher’s presence and know they are being observed).

However, awareness of an observer’s presence may influence participants’ behavior. 

Interviews give researchers a window into the world of a participant by seeking their account of an event, situation, or phenomenon. They are usually conducted on a one-to-one basis and can be distinguished according to the level at which they are structured (Punch, 2013). 

Structured interviews involve predetermined questions and sequences to ensure replicability and comparability. However, they are unable to explore emerging issues.

Informal interviews consist of spontaneous, casual conversations which are closer to the truth of a phenomenon. However, information is gathered using quick notes made by the researcher and is therefore subject to recall bias. 

Semi-structured interviews have a flexible structure, phrasing, and placement so emerging issues can be explored (Denny & Weckesser, 2022).

The use of probing questions and clarification can lead to a detailed understanding, but semi-structured interviews can be time-consuming and subject to interviewer bias. 

Focus groups 

Similar to interviews, focus groups elicit a rich and detailed account of an experience. However, focus groups are more dynamic since participants with shared characteristics construct this account together (Denny & Weckesser, 2022).

A shared narrative is built between participants to capture a group experience shaped by a shared context. 

The researcher takes on the role of a moderator, who will establish ground rules and guide the discussion by following a topic guide to focus the group discussions.

Typically, focus groups have 4-10 participants as a discussion can be difficult to facilitate with more than this, and this number allows everyone the time to speak.

Ethnography

Ethnography is a methodology used to study a group of people’s behaviors and social interactions in their environment (Reeves et al., 2008).

Data are collected using methods such as observations, field notes, or structured/ unstructured interviews.

The aim of ethnography is to provide detailed, holistic insights into people’s behavior and perspectives within their natural setting. In order to achieve this, researchers immerse themselves in a community or organization. 

Due to the flexibility and real-world focus of ethnography, researchers are able to gather an in-depth, nuanced understanding of people’s experiences, knowledge and perspectives that are influenced by culture and society.

In order to develop a representative picture of a particular culture/ context, researchers must conduct extensive field work. 

This can be time-consuming as researchers may need to immerse themselves into a community/ culture for a few days, or possibly a few years.

Qualitative Data Analysis Methods

Different methods can be used for analyzing qualitative data. The researcher chooses based on the objectives of their study. 

The researcher plays a key role in the interpretation of data, making decisions about the coding, theming, decontextualizing, and recontextualizing of data (Starks & Trinidad, 2007). 

Grounded theory

Grounded theory is a qualitative method specifically designed to inductively generate theory from data. It was developed by Glaser and Strauss in 1967 (Glaser & Strauss, 2017).

This methodology aims to develop theories (rather than test hypotheses) that explain a social process, action, or interaction (Petty et al., 2012). To inform the developing theory, data collection and analysis run simultaneously. 

There are three key types of coding used in grounded theory: initial (open), intermediate (axial), and advanced (selective) coding. 

Throughout the analysis, memos should be created to document methodological and theoretical ideas about the data. Data should be collected and analyzed until data saturation is reached and a theory is developed. 

Content analysis

Content analysis was first used in the early twentieth century to analyze textual materials such as newspapers and political speeches.

Content analysis is a research method used to identify and analyze the presence and patterns of themes, concepts, or words in data (Vaismoradi et al., 2013). 

This research method can be used to analyze data in different formats, which can be written, oral, or visual. 

The goal of content analysis is to develop themes that capture the underlying meanings of data (Schreier, 2012). 

Qualitative content analysis can be used to validate existing theories, support the development of new models and theories, and provide in-depth descriptions of particular settings or experiences.

The following six steps provide a guideline for how to conduct qualitative content analysis.

• Define a Research Question : To start content analysis, a clear research question should be developed.
• Identify and Collect Data : Establish the inclusion criteria for your data. Find the relevant sources to analyze.
• Define the Unit or Theme of Analysis : Categorize the content into themes. Themes can be a word, phrase, or sentence.
• Develop Rules for Coding your Data : Define a set of coding rules to ensure that all data are coded consistently.
• Code the Data : Follow the coding rules to categorize data into themes.
• Analyze the Results and Draw Conclusions : Examine the data to identify patterns and draw conclusions in relation to your research question.

Discourse analysis

Discourse analysis is a research method used to study written/ spoken language in relation to its social context (Wood & Kroger, 2000).

In discourse analysis, the researcher interprets details of language materials and the context in which it is situated.

Discourse analysis aims to understand the functions of language (how language is used in real life) and how meaning is conveyed by language in different contexts. Researchers use discourse analysis to investigate social groups and how language is used to achieve specific communication goals.

Different methods of discourse analysis can be used depending on the aims and objectives of a study. However, the following steps provide a guideline on how to conduct discourse analysis.

• Define the Research Question : Develop a relevant research question to frame the analysis.
• Gather Data and Establish the Context : Collect research materials (e.g., interview transcripts, documents). Gather factual details and review the literature to construct a theory about the social and historical context of your study.
• Analyze the Content : Closely examine various components of the text, such as the vocabulary, sentences, paragraphs, and structure of the text. Identify patterns relevant to the research question to create codes, then group these into themes.
• Review the Results : Reflect on the findings to examine the function of the language, and the meaning and context of the discourse. 

Thematic analysis

Thematic analysis is a method used to identify, interpret, and report patterns in data, such as commonalities or contrasts. 

Although the origin of thematic analysis can be traced back to the early twentieth century, understanding and clarity of thematic analysis is attributed to Braun and Clarke (2006).

Thematic analysis aims to develop themes (patterns of meaning) across a dataset to address a research question. 

In thematic analysis, qualitative data is gathered using techniques such as interviews, focus groups, and questionnaires. Audio recordings are transcribed. The dataset is then explored and interpreted by a researcher to identify patterns. 

This occurs through the rigorous process of data familiarisation, coding, theme development, and revision. These identified patterns provide a summary of the dataset and can be used to address a research question.

Themes are developed by exploring the implicit and explicit meanings within the data. Two different approaches are used to generate themes: inductive and deductive. 

An inductive approach allows themes to emerge from the data. In contrast, a deductive approach uses existing theories or knowledge to apply preconceived ideas to the data.

Phases of Thematic Analysis

Braun and Clarke (2006) provide a guide of the six phases of thematic analysis. These phases can be applied flexibly to fit research questions and data. 

Template analysis

Template analysis refers to a specific method of thematic analysis which uses hierarchical coding (Brooks et al., 2014).

Template analysis is used to analyze textual data, for example, interview transcripts or open-ended responses on a written questionnaire.

To conduct template analysis, a coding template must be developed (usually from a subset of the data) and subsequently revised and refined. This template represents the themes identified by researchers as important in the dataset. 

Codes are ordered hierarchically within the template, with the highest-level codes demonstrating overarching themes in the data and lower-level codes representing constituent themes with a narrower focus.

A guideline for the main procedural steps for conducting template analysis is outlined below.

• Familiarization with the Data : Read (and reread) the dataset in full. Engage, reflect, and take notes on data that may be relevant to the research question.
• Preliminary Coding : Identify initial codes using guidance from the a priori codes, identified before the analysis as likely to be beneficial and relevant to the analysis.
• Organize Themes : Organize themes into meaningful clusters. Consider the relationships between the themes both within and between clusters.
• Produce an Initial Template : Develop an initial template. This may be based on a subset of the data.
• Apply and Develop the Template : Apply the initial template to further data and make any necessary modifications. Refinements of the template may include adding themes, removing themes, or changing the scope/title of themes. 
• Finalize Template : Finalize the template, then apply it to the entire dataset. 

Frame analysis

Frame analysis is a comparative form of thematic analysis which systematically analyzes data using a matrix output.

Ritchie and Spencer (1994) developed this set of techniques to analyze qualitative data in applied policy research. Frame analysis aims to generate theory from data.

Frame analysis encourages researchers to organize and manage their data using summarization.

This results in a flexible and unique matrix output, in which individual participants (or cases) are represented by rows and themes are represented by columns. 

Each intersecting cell is used to summarize findings relating to the corresponding participant and theme.

Frame analysis has five distinct phases which are interrelated, forming a methodical and rigorous framework.

• Familiarization with the Data : Familiarize yourself with all the transcripts. Immerse yourself in the details of each transcript and start to note recurring themes.
• Develop a Theoretical Framework : Identify recurrent/ important themes and add them to a chart. Provide a framework/ structure for the analysis.
• Indexing : Apply the framework systematically to the entire study data.
• Summarize Data in Analytical Framework : Reduce the data into brief summaries of participants’ accounts.
• Mapping and Interpretation : Compare themes and subthemes and check against the original transcripts. Group the data into categories and provide an explanation for them.

Preventing Bias in Qualitative Research

To evaluate qualitative studies, the CASP (Critical Appraisal Skills Programme) checklist for qualitative studies can be used to ensure all aspects of a study have been considered (CASP, 2018).

The quality of research can be enhanced and assessed using criteria such as checklists, reflexivity, co-coding, and member-checking. 

Co-coding 

Relying on only one researcher to interpret rich and complex data may risk key insights and alternative viewpoints being missed. Therefore, coding is often performed by multiple researchers.

A common strategy must be defined at the beginning of the coding process  (Busetto et al., 2020). This includes establishing a useful coding list and finding a common definition of individual codes.

Transcripts are initially coded independently by researchers and then compared and consolidated to minimize error or bias and to bring confirmation of findings. 

Member checking

Member checking (or respondent validation) involves checking back with participants to see if the research resonates with their experiences (Russell & Gregory, 2003).

Data can be returned to participants after data collection or when results are first available. For example, participants may be provided with their interview transcript and asked to verify whether this is a complete and accurate representation of their views.

Participants may then clarify or elaborate on their responses to ensure they align with their views (Shenton, 2004).

This feedback becomes part of data collection and ensures accurate descriptions/ interpretations of phenomena (Mays & Pope, 2000). 

Reflexivity in qualitative research

Reflexivity typically involves examining your own judgments, practices, and belief systems during data collection and analysis. It aims to identify any personal beliefs which may affect the research. 

Reflexivity is essential in qualitative research to ensure methodological transparency and complete reporting. This enables readers to understand how the interaction between the researcher and participant shapes the data.

Depending on the research question and population being researched, factors that need to be considered include the experience of the researcher, how the contact was established and maintained, age, gender, and ethnicity.

These details are important because, in qualitative research, the researcher is a dynamic part of the research process and actively influences the outcome of the research (Boeije, 2014). 

Reflexivity Example

Who you are and your characteristics influence how you collect and analyze data. Here is an example of a reflexivity statement for research on smoking. I am a 30-year-old white female from a middle-class background. I live in the southwest of England and have been educated to master’s level. I have been involved in two research projects on oral health. I have never smoked, but I have witnessed how smoking can cause ill health from my volunteering in a smoking cessation clinic. My research aspirations are to help to develop interventions to help smokers quit.

Establishing Trustworthiness in Qualitative Research

Trustworthiness is a concept used to assess the quality and rigor of qualitative research. Four criteria are used to assess a study’s trustworthiness: credibility, transferability, dependability, and confirmability.

1. Credibility in Qualitative Research

Credibility refers to how accurately the results represent the reality and viewpoints of the participants.

To establish credibility in research, participants’ views and the researcher’s representation of their views need to align (Tobin & Begley, 2004).

To increase the credibility of findings, researchers may use data source triangulation, investigator triangulation, peer debriefing, or member checking (Lincoln & Guba, 1985). 

2. Transferability in Qualitative Research

Transferability refers to how generalizable the findings are: whether the findings may be applied to another context, setting, or group (Tobin & Begley, 2004).

Transferability can be enhanced by giving thorough and in-depth descriptions of the research setting, sample, and methods (Nowell et al., 2017). 

3. Dependability in Qualitative Research

Dependability is the extent to which the study could be replicated under similar conditions and the findings would be consistent.

Researchers can establish dependability using methods such as audit trails so readers can see the research process is logical and traceable (Koch, 1994).

4. Confirmability in Qualitative Research

Confirmability is concerned with establishing that there is a clear link between the researcher’s interpretations/ findings and the data.

Researchers can achieve confirmability by demonstrating how conclusions and interpretations were arrived at (Nowell et al., 2017).

This enables readers to understand the reasoning behind the decisions made. 

Audit Trails in Qualitative Research

An audit trail provides evidence of the decisions made by the researcher regarding theory, research design, and data collection, as well as the steps they have chosen to manage, analyze, and report data. 

The researcher must provide a clear rationale to demonstrate how conclusions were reached in their study.

A clear description of the research path must be provided to enable readers to trace through the researcher’s logic (Halpren, 1983).

Researchers should maintain records of the raw data, field notes, transcripts, and a reflective journal in order to provide a clear audit trail. 

Discovery of unexpected data

Open-ended questions in qualitative research mean the researcher can probe an interview topic and enable the participant to elaborate on responses in an unrestricted manner.

This allows unexpected data to emerge, which can lead to further research into that topic. 

The exploratory nature of qualitative research helps generate hypotheses that can be tested quantitatively (Busetto et al., 2020).

Flexibility

Data collection and analysis can be modified and adapted to take the research in a different direction if new ideas or patterns emerge in the data.

This enables researchers to investigate new opportunities while firmly maintaining their research goals. 

Naturalistic settings

The behaviors of participants are recorded in real-world settings. Studies that use real-world settings have high ecological validity since participants behave more authentically. 

Limitations

Time-consuming .

Qualitative research results in large amounts of data which often need to be transcribed and analyzed manually.

Even when software is used, transcription can be inaccurate, and using software for analysis can result in many codes which need to be condensed into themes. 

Subjectivity 

The researcher has an integral role in collecting and interpreting qualitative data. Therefore, the conclusions reached are from their perspective and experience.

Consequently, interpretations of data from another researcher may vary greatly. 

Limited generalizability

The aim of qualitative research is to provide a detailed, contextualized understanding of an aspect of the human experience from a relatively small sample size.

Despite rigorous analysis procedures, conclusions drawn cannot be generalized to the wider population since data may be biased or unrepresentative.

Therefore, results are only applicable to a small group of the population. 

While individual qualitative studies are often limited in their generalizability due to factors such as sample size and context, metasynthesis enables researchers to synthesize findings from multiple studies, potentially leading to more generalizable conclusions.

By integrating findings from studies conducted in diverse settings and with different populations, metasynthesis can provide broader insights into the phenomenon of interest.

Extraneous variables

Qualitative research is often conducted in real-world settings. This may cause results to be unreliable since extraneous variables may affect the data, for example:

• Situational variables : different environmental conditions may influence participants’ behavior in a study. The random variation in factors (such as noise or lighting) may be difficult to control in real-world settings.
• Participant characteristics : this includes any characteristics that may influence how a participant answers/ behaves in a study. This may include a participant’s mood, gender, age, ethnicity, sexual identity, IQ, etc.
• Experimenter effect : experimenter effect refers to how a researcher’s unintentional influence can change the outcome of a study. This occurs when (i) their interactions with participants unintentionally change participants’ behaviors or (ii) due to errors in observation, interpretation, or analysis. 

What sample size should qualitative research be?

The sample size for qualitative studies has been recommended to include a minimum of 12 participants to reach data saturation (Braun, 2013).

Are surveys qualitative or quantitative?

Surveys can be used to gather information from a sample qualitatively or quantitatively. Qualitative surveys use open-ended questions to gather detailed information from a large sample using free text responses.

The use of open-ended questions allows for unrestricted responses where participants use their own words, enabling the collection of more in-depth information than closed-ended questions.

In contrast, quantitative surveys consist of closed-ended questions with multiple-choice answer options. Quantitative surveys are ideal to gather a statistical representation of a population.

What are the ethical considerations of qualitative research?

Before conducting a study, you must think about any risks that could occur and take steps to prevent them. Participant Protection : Researchers must protect participants from physical and mental harm. This means you must not embarrass, frighten, offend, or harm participants. Transparency : Researchers are obligated to clearly communicate how they will collect, store, analyze, use, and share the data. Confidentiality : You need to consider how to maintain the confidentiality and anonymity of participants’ data.

What is triangulation in qualitative research?

Triangulation refers to the use of several approaches in a study to comprehensively understand phenomena. This method helps to increase the validity and credibility of research findings. 

Types of triangulation include method triangulation (using multiple methods to gather data); investigator triangulation (multiple researchers for collecting/ analyzing data), theory triangulation (comparing several theoretical perspectives to explain a phenomenon), and data source triangulation (using data from various times, locations, and people; Carter et al., 2014).

Why is qualitative research important?

Qualitative research allows researchers to describe and explain the social world. The exploratory nature of qualitative research helps to generate hypotheses that can then be tested quantitatively.

In qualitative research, participants are able to express their thoughts, experiences, and feelings without constraint.

Additionally, researchers are able to follow up on participants’ answers in real-time, generating valuable discussion around a topic. This enables researchers to gain a nuanced understanding of phenomena which is difficult to attain using quantitative methods.

What is coding data in qualitative research?

Coding data is a qualitative data analysis strategy in which a section of text is assigned with a label that describes its content.

These labels may be words or phrases which represent important (and recurring) patterns in the data.

This process enables researchers to identify related content across the dataset. Codes can then be used to group similar types of data to generate themes.

What is the difference between qualitative and quantitative research?

Qualitative research involves the collection and analysis of non-numerical data in order to understand experiences and meanings from the participant’s perspective.

This can provide rich, in-depth insights on complicated phenomena. Qualitative data may be collected using interviews, focus groups, or observations.

In contrast, quantitative research involves the collection and analysis of numerical data to measure the frequency, magnitude, or relationships of variables. This can provide objective and reliable evidence that can be generalized to the wider population.

Quantitative data may be collected using closed-ended questionnaires or experiments.

What is trustworthiness in qualitative research?

Trustworthiness is a concept used to assess the quality and rigor of qualitative research. Four criteria are used to assess a study’s trustworthiness: credibility, transferability, dependability, and confirmability. 

Credibility refers to how accurately the results represent the reality and viewpoints of the participants. Transferability refers to whether the findings may be applied to another context, setting, or group.

Dependability is the extent to which the findings are consistent and reliable. Confirmability refers to the objectivity of findings (not influenced by the bias or assumptions of researchers).

What is data saturation in qualitative research?

Data saturation is a methodological principle used to guide the sample size of a qualitative research study.

Data saturation is proposed as a necessary methodological component in qualitative research (Saunders et al., 2018) as it is a vital criterion for discontinuing data collection and/or analysis. 

The intention of data saturation is to find “no new data, no new themes, no new coding, and ability to replicate the study” (Guest et al., 2006). Therefore, enough data has been gathered to make conclusions.

Why is sampling in qualitative research important?

In quantitative research, large sample sizes are used to provide statistically significant quantitative estimates.

This is because quantitative research aims to provide generalizable conclusions that represent populations.

However, the aim of sampling in qualitative research is to gather data that will help the researcher understand the depth, complexity, variation, or context of a phenomenon. The small sample sizes in qualitative studies support the depth of case-oriented analysis.

What is narrative analysis?

Narrative analysis is a qualitative research method used to understand how individuals create stories from their personal experiences.

There is an emphasis on understanding the context in which a narrative is constructed, recognizing the influence of historical, cultural, and social factors on storytelling.

Researchers can use different methods together to explore a research question.

Some narrative researchers focus on the content of what is said, using thematic narrative analysis, while others focus on the structure, such as holistic-form or categorical-form structural narrative analysis. Others focus on how the narrative is produced and performed.

Boeije, H. (2014). Analysis in qualitative research. Sage.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology , 3 (2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Brooks, J., McCluskey, S., Turley, E., & King, N. (2014). The utility of template analysis in qualitative psychology research. Qualitative Research in Psychology , 12 (2), 202–222. https://doi.org/10.1080/14780887.2014.955224

Busetto, L., Wick, W., & Gumbinger, C. (2020). How to use and assess qualitative research methods. Neurological research and practice , 2 (1), 14-14. https://doi.org/10.1186/s42466-020-00059-z 

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology nursing forum , 41 (5), 545–547. https://doi.org/10.1188/14.ONF.545-547

Critical Appraisal Skills Programme. (2018). CASP Checklist: 10 questions to help you make sense of a Qualitative research. https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018_fillable_form.pdf Accessed: March 15 2023

Clarke, V., & Braun, V. (2013). Successful qualitative research: A practical guide for beginners. Successful Qualitative Research , 1-400.

Denny, E., & Weckesser, A. (2022). How to do qualitative research?: Qualitative research methods. BJOG : an international journal of obstetrics and gynaecology , 129 (7), 1166-1167. https://doi.org/10.1111/1471-0528.17150 

Glaser, B. G., & Strauss, A. L. (2017). The discovery of grounded theory. The Discovery of Grounded Theory , 1–18. https://doi.org/10.4324/9780203793206-1

Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18 (1), 59-82. doi:10.1177/1525822X05279903

Halpren, E. S. (1983). Auditing naturalistic inquiries: The development and application of a model (Unpublished doctoral dissertation). Indiana University, Bloomington.

Hammarberg, K., Kirkman, M., & de Lacey, S. (2016). Qualitative research methods: When to use them and how to judge them. Human Reproduction , 31 (3), 498–501. https://doi.org/10.1093/humrep/dev334

Koch, T. (1994). Establishing rigour in qualitative research: The decision trail. Journal of Advanced Nursing, 19, 976–986. doi:10.1111/ j.1365-2648.1994.tb01177.x

Lincoln, Y., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. BMJ, 320(7226), 50–52.

Minichiello, V. (1990). In-Depth Interviewing: Researching People. Longman Cheshire.

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic Analysis: Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods, 16 (1). https://doi.org/10.1177/1609406917733847

Petty, N. J., Thomson, O. P., & Stew, G. (2012). Ready for a paradigm shift? part 2: Introducing qualitative research methodologies and methods. Manual Therapy , 17 (5), 378–384. https://doi.org/10.1016/j.math.2012.03.004

Punch, K. F. (2013). Introduction to social research: Quantitative and qualitative approaches. London: Sage

Reeves, S., Kuper, A., & Hodges, B. D. (2008). Qualitative research methodologies: Ethnography. BMJ , 337 (aug07 3). https://doi.org/10.1136/bmj.a1020

Russell, C. K., & Gregory, D. M. (2003). Evaluation of qualitative research studies. Evidence Based Nursing, 6 (2), 36–40.

Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018). Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & quantity , 52 (4), 1893–1907. https://doi.org/10.1007/s11135-017-0574-8

Scarduzio, J. A. (2017). Emic approach to qualitative research. The International Encyclopedia of Communication Research Methods, 1–2 . https://doi.org/10.1002/9781118901731.iecrm0082

Schreier, M. (2012). Qualitative content analysis in practice / Margrit Schreier.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 , 63–75.

Starks, H., & Trinidad, S. B. (2007). Choose your method: a comparison of phenomenology, discourse analysis, and grounded theory. Qualitative health research , 17 (10), 1372–1380. https://doi.org/10.1177/1049732307307031

Tenny, S., Brannan, J. M., & Brannan, G. D. (2022). Qualitative Study. In StatPearls. StatPearls Publishing.

Tobin, G. A., & Begley, C. M. (2004). Methodological rigour within a qualitative framework. Journal of Advanced Nursing, 48, 388–396. doi:10.1111/j.1365-2648.2004.03207.x

Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & health sciences , 15 (3), 398-405. https://doi.org/10.1111/nhs.12048

Wood L. A., Kroger R. O. (2000). Doing discourse analysis: Methods for studying action in talk and text. Sage.

Yilmaz, K. (2013). Comparison of Quantitative and Qualitative Research Traditions: epistemological, theoretical, and methodological differences. European journal of education , 48 (2), 311-325. https://doi.org/10.1111/ejed.12014

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

4.2 Definitions and Characteristics of Qualitative Research

Qualitative research aims to uncover the meaning and understanding of phenomena that cannot be broken down into measurable elements. It is based on naturalistic, interpretative and humanistic notions. 5 This research method seeks to discover, explore, identify or describe subjective human experiences using non-statistical methods and develops themes from the study participants’ stories. 5 Figure 4.1 depicts major features/ characteristics of qualitative research. It utilises exploratory open-ended questions and observations to search for patterns of meaning in collected data (e.g. observation, verbal/written narrative data, photographs, etc.) and uses inductive thinking (from specific observations to more general rules) to interpret meaning. 6 Participants’ voice is evident through quotations and description of the work. 6 The context/ setting of the study and the researcher’s reflexivity (i.e. “reflection on and awareness of their bias”, the effect of the researcher’s experience on the data and interpretations) are very important and described as part of data collection. 6 Analysis of collected data is complex, often involves inductive data analysis (exploration, contrasts, specific to general) and requires multiple coding and development of themes from participant stories. 6

Reflexivity- avoiding bias/Role of the qualitative researcher

Qualitative researchers generally begin their work with the recognition that their position (or worldview) has a significant impact on the overall research process. 7 Researcher worldview shapes the way the research is conducted, i.e., how the questions are formulated, methods are chosen, data are collected and analysed, and results are reported. Therefore, it is essential for qualitative researchers to acknowledge, articulate, reflect on and clarify their own underlying biases and assumptions before embarking on any research project. 7 Reflexivity helps to ensure that the researcher’s own experiences, values, and beliefs do not unintentionally bias the data collection, analysis, and interpretation. 7 It is the gold standard for establishing trustworthiness and has been established as one of the ways qualitative researchers should ensure rigour and quality in their work. 8 The following questions in Table 4.1 may help you begin the reflective process. 9

Table 4.1: Questions to aid the reflection process

Philosophical underpinnings to qualitative research

Qualitative research uses an inductive approach and stems from interpretivism or constructivism and assumes that realities are multiple, socially constructed, and holistic. 10 According to this philosophical viewpoint, humans build reality through their interactions with the world around them. 10 As a result, qualitative research aims to comprehend how individuals make sense of their experiences and build meaning in their lives. 10 Because reality is complex/nuanced and context-bound, participants constantly construct it depending on their understanding. Thus, the interactions between the researcher and the participants are considered necessary to offer a rich description of the concept and provide an in-depth understanding of the phenomenon under investigation. 11

An Introduction to Research Methods for Undergraduate Health Profession Students Copyright © 2023 by Faith Alele and Bunmi Malau-Aduli is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Research Design | Types, Guide & Examples

What Is a Research Design | Types, Guide & Examples

Published on June 7, 2021 by Shona McCombes . Revised on November 20, 2023 by Pritha Bhandari.

A research design is a strategy for answering your   research question  using empirical data. Creating a research design means making decisions about:

• Your overall research objectives and approach
• Whether you’ll rely on primary research or secondary research
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, other interesting articles, frequently asked questions about research design.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

• Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships
• Descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

• Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study , your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question .

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews .

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.

Operationalization

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method , you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method , it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method , how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymize and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability ).

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarize your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A research design is a strategy for answering your   research question . It defines your overall approach and determines how you will collect and analyze data.

A well-planned research design helps ensure that your methods match your research aims, that you collect high-quality data, and that you use the right kind of analysis to answer your questions, utilizing credible sources . This allows you to draw valid , trustworthy conclusions.

Quantitative research designs can be divided into two main categories:

• Correlational and descriptive designs are used to investigate characteristics, averages, trends, and associations between variables.
• Experimental and quasi-experimental designs are used to test causal relationships .

Qualitative research designs tend to be more flexible. Common types of qualitative design include case study , ethnography , and grounded theory designs.

The priorities of a research design can vary depending on the field, but you usually have to specify:

• Your research questions and/or hypotheses
• Your overall approach (e.g., qualitative or quantitative )
• The type of design you’re using (e.g., a survey , experiment , or case study )
• Your data collection methods (e.g., questionnaires , observations)
• Your data collection procedures (e.g., operationalization , timing and data management)
• Your data analysis methods (e.g., statistical tests  or thematic analysis )

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Research Design | Types, Guide & Examples. Scribbr. Retrieved August 30, 2024, from https://www.scribbr.com/methodology/research-design/

Is this article helpful?

Shona McCombes

Other students also liked, guide to experimental design | overview, steps, & examples, how to write a research proposal | examples & templates, ethical considerations in research | types & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Types of qualitative research designs

Last updated

20 February 2023

Reviewed by

Jean Kaluza

Researchers often conduct these studies to gain a detailed understanding of a particular topic through a small, focused sample. Qualitative research methods delve into understanding why something is happening in a larger quantitative study. 

To determine whether qualitative research is the best choice for your study, let’s look at the different types of qualitative research design.

Analyze all your qualitative research

Analyze qualitative data faster and surface more actionable insights

• What are qualitative research designs?

Qualitative research designs are research methods that collect and analyze non-numerical data. The research uncovers why or how a particular behavior or occurrence takes place. The information is usually subjective and in a written format instead of numerical.

Researchers may use interviews, focus groups , case studies , journaling, and open-ended questions to gather in-depth information. Qualitative research designs can determine users' concepts, develop a hypothesis , or add context to data from a quantitative study.

• Characteristics of qualitative research design

Most often, qualitative data answers how or why something occurs. Certain characteristics are usually present in all qualitative research designs to ensure accurate data. 

The most common characteristics of qualitative research design include the following:

Natural environment

It’s best to collect qualitative research as close to the subject’s original environment as possible to encourage natural behavior and accurate insights.

Empathy is key

Qualitative researchers collect the best data when they’re in sync with their users’ concerns and motivations. They can play into natural human psychology by combining open-ended questioning and subtle cues.

They may mimic body language, adopt the users’ terminology, and use pauses or trailing sentences to encourage their participants to fill in the blanks. The more empathic the interviewer, the purer the data.

Participant selection

Qualitative research depends on the meaning obtained from participants instead of the meaning conveyed in similar research or studies. To increase research accuracy, you choose participants randomly from carefully chosen groups of potential participants.

Different research methods or multiple data sources

To gain in-depth knowledge, qualitative research designs often rely on multiple research methods within the same group. 

Emergent design

Qualitative research constantly evolves, meaning the initial study plan might change after you collect data. This evolution might result in changes in research methods or the introduction of a new research problem.

Inductive reasoning

Since qualitative research seeks in-depth meaning, you need complex reasoning to get the right results. Qualitative researchers build categories, patterns, and themes from separate data sets to form a complete conclusion.

Interpretive data

Once you collect the data, you need to read between the lines rather than just noting what your participant said. Qualitative research is unique as we can attach actions to feedback. 

If a user says they love the look of your design but haven’t completed any tasks, it’s up to you to interpret this as a failed test, even with their positive sentiments.  

Holistic account

To paint a large picture of an issue and potential solutions, a qualitative researcher works to develop a complex description of the research problem. You can avoid a narrow cause-and-effect perspective by describing the problem’s wider perspectives. 

• When to use qualitative research design

Qualitative research aims to get a detailed understanding of a particular topic. To accomplish this, you’ll typically use small focus groups to gather in-depth data from varied perspectives. 

This approach is only effective for some types of study. For instance, a qualitative approach wouldn’t work for a study that seeks to understand a statistically relevant finding.

When determining if a qualitative research design is appropriate, remember the goal of qualitative research is understanding the “ why .” 

Qualitative research design gathers in-depth information that stands on its own. It can also answer the “why” of a quantitative study or be a precursor to forming a hypothesis. 

You can use qualitative research in these situations:

Developing a hypothesis for testing in a quantitative study

Identifying customer needs

Developing a new feature

Adding context to the results of a quantitative study

Understanding the motivations, values, and pain points that guide behavior

Difference between qualitative and quantitative research design

Qualitative and quantitative research designs gather data, but that's where the similarities end. Consider the difference between quality and quantity. Both are useful in different ways.

Qualitative research gathers in-depth information to answer how or why . It uses subjective data from detailed interviews, observations, and open-ended questions. Most often, qualitative data is thoughts, experiences, and concepts.

In contrast, quantitative research designs gather large amounts of objective data that you can quantify mathematically. You typically express quantitative data in numbers or graphs, and you use it to test or confirm hypotheses.

Qualitative research designs generally have the same goals. However, there are various ways to achieve these goals. Researchers may use one or more of these approaches in qualitative research.

Historical study

This is where you use extensive information about people and events in the past to draw conclusions about the present and future.

Phenomenology

Phenomenology investigates a phenomenon, activity, or event using data from participants' perspectives. Often, researchers use a combination of methods.

Grounded theory

Grounded theory uses interviews and existing data to build a theory inductively.

Ethnography

Researchers immerse themselves in the target participant's environments to understand goals, cultures, challenges, and themes with ethnography .

A case study is where you use multiple data sources to examine a person, group, community, or institution. Participants must share a connection to the research question you’re studying.

• Advantages and disadvantages of qualitative research

All qualitative research design types share the common goal of obtaining in-depth information. Achieving this goal generally requires extensive data collection methods that can be time-consuming. As such, qualitative research has advantages and disadvantages. 

Natural settings

Since you can collect data closer to an authentic environment, it offers more accurate results.  

The ability to paint a picture with data

Quantitative studies don't always reveal the full picture. With multiple data collection methods, you can expose the motivations and reasons behind data.

Flexibility

Analysis processes aren't set in stone, so you can adapt the process as ideas or patterns emerge.

Generation of new ideas

Using open-ended responses can uncover new opportunities or solutions that weren't part of your original research plan.

Small sample sizes

You can generate meaningful results with small groups.

Disadvantages

Potentially unreliable.

A natural setting can be a double-edged sword. The inability to attach findings to anything statistically relevant can make data more difficult to quantify. 

Subjectivity

Since the researcher plays a vital role in collecting and interpreting data, qualitative research is subject to the researcher's skills. For example, they may miss a cue that changes some of the context of the quotes they collected.

Labor-intensive

You generally collect qualitative data through manual processes like extensive interviews, open-ended questions, and case studies.

Qualitative research designs allow researchers to provide an in-depth analysis of why specific behavior or events occur. It can offer fresh insights, generate new ideas, or add context to statistics from quantitative studies. Depending on your needs, qualitative data might be a great way to gain the information your organization needs to move forward.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 22 August 2024

Last updated: 5 February 2023

Last updated: 16 August 2024

Last updated: 9 March 2023

Last updated: 30 April 2024

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 4 July 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next, log in or sign up.

Get started for free

Research Design Review

A discussion of qualitative & quantitative research design, 10 distinctive qualities of qualitative research.

Researchers conduct qualitative research because they acknowledge the human condition and want to learn more, and think differently, about a research issue than what is usual from mostly numerical quantitative survey research data.  Not surprisingly, the unique nature of qualitative inquiry is characterized by a distinctive set of attributes, all of which impact the design of qualitative research one way or the other.  The 10 unique attributes of qualitative research* are the:

• Absence of “truth” With all the emphasis in qualitative research on reality and the human condition, it might be expected that qualitative inquiry is in the business of garnering “the truth” from participants.  Instead of “truth,” the qualitative researcher collects information from which some level of knowledge can be gained.  The researcher does not acquire this information and knowledge in a vacuum but rather in a context and, in this way, the research data are a product of various situational factors.  For this reason, qualitative researchers do not talk about the “truth” of their findings but rather the “plausibility” of their interpretations. Plausibility is derived from achieving accuracy in the data collection process , accuracy in the absence of absolute “truth.”
• Importance of context A relevant factor in the elusiveness of “truth” is the central and significant role context plays in qualitative research.  Whether it be the physical environment or mode by which an in-depth interview (IDI), group discussion, or observation is conducted the outcomes in qualitative research hinge greatly on the contexts from which we obtain this data.
• Importance of meaning Although the goal of all research is to draw meaning from the data, qualitative research is unique in the dimensionality of this effort.  Qualitative researchers derive meaning from the data by way of multiple sources, evaluating any number of variables such as: the context, the language, the impact of the participant-researcher relationship, the potential for participant bias, and the potential for researcher bias. Several articles in Research Design Review discuss the importance of meaning, including “Words Versus Meanings.”
• Researcher-as-instrument Along with the emphases on context, meaning, and the potential for researcher subjectivity, qualitative research is distinguished by the fact it places the researcher at the center of the data-gathering phase and, indeed, the researcher is the instrument by which information is collected.  The closeness of the researcher to the research participants and subject matter instills an in-depth understanding which can prove beneficial to a thorough analysis and interpretation of the outcomes; however, this intimacy heightens concerns regarding the researcher’s ability to collect (and interpret) data in an objective, unbiased manner. Mitigating these effects is discussed here .
• Participant-researcher relationship Closely associated with the idea that the researcher is the tool by which data are gathered is the important function of the participant-researcher relationship in qualitative research and its impact on research outcomes.  This relationship is at the core of IDIs, group discussions, and participant observation, where participants and researchers share the “research space” within which certain conventions for communicating (knowingly or not) may be formed and which, in turn, shapes the reality the researcher is capturing in the data. A discussion of this attribute along with two other unique attributes — importance of context and importance of meaning — can be found here .
• Skill set required of the researcher Qualitative research requires a unique set of skills from the researcher, skills that go beyond the usual qualities of organization, attention to detail, and analytical abilities that are necessary for all researchers.  Techniques to build rapport with participants and active listening skills are only two examples.  Qualitative researchers also need a special class of analytical skills that can meet the demands of contextual, multilayered analysis (see below) in qualitative inquiry where context, social interaction, and numerous other inter-connected variables contribute to the realities researchers take away from the field. Qualitative research involving multiple methods requires a special set of skills, as discussed in “Working with Multiple Methods in Qualitative Research: 7 Unique Researcher Skills.”
• Flexibility of the research design A defining characteristic of qualitative research is the flexibility built into the research design .  For instance, it is not until a focus group moderator is actually in a group discussion that he or she understands which topical areas to pursue more than others or the specific follow-up (probing) questions to interject.  And, a participant observer has little control over the activities of the observed and, indeed, the goal of the observer is to be as unobtrusive and flexible as possible in order to capture the reality of the observed events.
• Types of issues or questions effectively addressed by qualitative research Qualitative research is uniquely suited to address research issues or questions that might be difficult, if not impossible, to investigate under more structured, less flexible research designs.  Qualitative inquiry effectively tackles: sensitive or personal issues such as domestic violence and sexual dysfunction; intricate topics such as personal life histories; nebulous questions such as “Is the current school leadership as effective as it could be?”; and contextual issues such as in-the-moment decision-making.  Similarly, qualitative research is useful at gaining meaningful information from hard-to-reach or underserved populations such as children of all ages, subcultures, and deviant groups.
• Contextual, multilayered analysis Without a doubt, the analysis of qualitative data does not follow a straight line, where point ‘A’ leads to point ‘B’, but rather is a multilayered, involved process that continually builds upon itself until a meaningful, contextually-derived, and verifiable interpretation is achieved.  The interconnections, possible inconsistencies, and interwoven contextual input reaped in qualitative research demand that researchers embrace the tangles of their data from many sources.  A large contributor to the complexity of the analytical process is the inductive method.  Qualitative researchers typically analyze their outcomes from the inside out, deriving their interpretations from the themes they construct from the data gathered. Qualitative Data Analysis is a compilation of 16 articles discussing various facets of qualitative analysis.
• Unique capabilities of online and mobile qualitative research Online and mobile technology offer unique enhancements to qualitative research design.  In large part, this technology has shifted the balance of power from the researcher to the online or mobile participant who is given greater control of the research process by way of more flexibility, convenience, and ways to respond in greater detail and depth to the researcher’s questions.

* Adapted from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller, M. R. & Lavrakas, P. J., 2015. New York: Guilford Press).

Share this:

• Click to share on Reddit (Opens in new window)
• Click to share on Twitter (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on Facebook (Opens in new window)
• Click to share on Tumblr (Opens in new window)
• Click to email a link to a friend (Opens in new window)
• Click to print (Opens in new window)

37 comments

• Pingback: NURS 8200 Do you think there is one type of research (quantitative or qualitative) that is inherently more rigorous than the other? – Nursing Assignment Gurus
• Pingback: Why Qualitative Research Matters: Understanding People And Their Behaviors – TheSelfImprovement
• Pingback: The Issues & Questions Uniquely Suited For Qualitative Research | Research Design Review
• Pingback: Elevating Qualitative Design to Maximize Research Integrity | Research Design Review
• Pingback: Qualitative Research in APA Style | Research Design Review
• Pingback: Shared Constructs in Research Design: Part 1 — Sampling | Research Design Review
• Pingback: Quality Frameworks in Qualitative Research | Research Design Review
• Pingback: Qualitative Research: A Call for Collective Action | Research Design Review
• Pingback: Sample Size in Qualitative Research & the Risk of Relying on Saturation | Research Design Review

Would like know more about Qualitative research design

For an alternative approach to qualitative research design, see my book–JA Maxwell, Qualitative Research Design: An Interactive Approach (SAGE publications, 3rd ed., 2011).

For more on qualitative research design, see my book Qualitative Research Design: An Interactive Approach (3rd ed., Joseph Maxwell, SAGE Publications, 2013); there are reviews on Amazon.com.

• Pingback: Cognitive Interviewing: A Few Best Practices | Research Design Review
• Pingback: Applying the TQF Credibility Component: An IDI Case Study | Research Design Review
• Pingback: What is Your Meaning of “Qualitative Research”? | Research Design Review
• Pingback: Educational Research: The Qualitative Path – Nursing Education Network
• Pingback: The Three Dominant Qualities of Qualitative Research | Research Design Review
• Pingback: The “Real Ethnography” of Michael Agar | Research Design Review
• Pingback: Questions and more questions
• Pingback: Qualitative Research: A Collection of Articles from 2016 | Research Design Review
• Pingback: Paying Attention to Bias in Qualitative Research: A Message to Marketing Researchers (& Clients) | Research Design Review
• Pingback: December 12 | kusp fyi

Thank you…it really helps me a lot in understanding of qualitative research which I am embarking now.

• Pingback: Qualitative Data: Achieving Accuracy in the Absence of “Truth” | Research Design Review
• Pingback: 2. What Are The Characteristics Of Good Research Design – Bestperfeed.info
• Pingback: Mitigating Researcher-as-instrument Effects | Research Design Review
• Pingback: Qualitative Research “Participants” Are Not “Respondents” (& Other Misplaced Concepts From Quantitative Research) | Research Design Review
• Pingback: Lessons in Best Practices from Qualitative Research with Distinct Cultures | Research Design Review
• Pingback: Helping Survey Data “Line Up”: Qualitative Lends a Hand | Research Design Review
• Pingback: Social Constructionism & Quality in Qualitative Research Design | Research Design Review

This is a really nice summary. The one point I would add is that qualitative research is much better than quantitative research at identifying the processes by which events or outcomes occur. Quantitative research is very good at showing WHETHER A influenced B, but can tell us very little about HOW it did so. Qualitative methods can get inside the “black box” of experimental and statistical designs and reveal the mechanisms (mental as well as physical) that caused the result. See J.Maxwell, “Causal explanation, qualitative research, and scientific inquiry in education,” Educational Researcher 33(2), 3-11, March 2004.

Thank you for this addition, and my apologies for not responding sooner. I have read your paper in Educational Researcher as well as your very good book “A Realist Approach for Qualitative Research.” I agree wholeheartedly with your discussions of a process approach to causation, and the idea that social and cultural contexts are essential to understanding “causal mechanisms.” This is the important role that qualitative methods play and, as you say, why we are comfortable identifying causation from single case studies.

Let me add that I also appreciate your long discussion of validity in your 2012 book, including the three distinctions you make, e.g., separating accounts of phenomena from the accounts of meaning that can only come from individuals’ “conceptual framework.”

Thanks, again. And, again, sorry for the delay.

• Pingback: Qualitative Research Design: 13 Articles from RDR in 2013 | Research Design Review
• Pingback: Dissertation10110 Distinctive Qualities of Qualitative Research » Dissertation101

It is good I liked it, keep posting more on qualitative research method and practices

Reblogged this on Anthropologizing .

Reblogged this on Elodie Crespel .

Leave a comment Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed .

• Already have a WordPress.com account? Log in now.
• Subscribe Subscribed
• Copy shortlink
• Report this content
• View post in Reader
• Manage subscriptions
• Collapse this bar

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Characteristics of Qualitative Descriptive Studies: A Systematic Review

MSN, CRNP, Doctoral Candidate, University of Pennsylvania School of Nursing

Justine S. Sefcik

MS, RN, Doctoral Candidate, University of Pennsylvania School of Nursing

Christine Bradway

PhD, CRNP, FAAN, Associate Professor of Gerontological Nursing, University of Pennsylvania School of Nursing

Qualitative description (QD) is a term that is widely used to describe qualitative studies of health care and nursing-related phenomena. However, limited discussions regarding QD are found in the existing literature. In this systematic review, we identified characteristics of methods and findings reported in research articles published in 2014 whose authors identified the work as QD. After searching and screening, data were extracted from the sample of 55 QD articles and examined to characterize research objectives, design justification, theoretical/philosophical frameworks, sampling and sample size, data collection and sources, data analysis, and presentation of findings. In this review, three primary findings were identified. First, despite inconsistencies, most articles included characteristics consistent with limited, available QD definitions and descriptions. Next, flexibility or variability of methods was common and desirable for obtaining rich data and achieving understanding of a phenomenon. Finally, justification for how a QD approach was chosen and why it would be an appropriate fit for a particular study was limited in the sample and, therefore, in need of increased attention. Based on these findings, recommendations include encouragement to researchers to provide as many details as possible regarding the methods of their QD study so that readers can determine whether the methods used were reasonable and effective in producing useful findings.

Qualitative description (QD) is a label used in qualitative research for studies which are descriptive in nature, particularly for examining health care and nursing-related phenomena ( Polit & Beck, 2009 , 2014 ). QD is a widely cited research tradition and has been identified as important and appropriate for research questions focused on discovering the who, what, and where of events or experiences and gaining insights from informants regarding a poorly understood phenomenon. It is also the label of choice when a straight description of a phenomenon is desired or information is sought to develop and refine questionnaires or interventions ( Neergaard et al., 2009 ; Sullivan-Bolyai et al., 2005 ).

Despite many strengths and frequent citations of its use, limited discussions regarding QD are found in qualitative research textbooks and publications. To the best of our knowledge, only seven articles include specific guidance on how to design, implement, analyze, or report the results of a QD study ( Milne & Oberle, 2005 ; Neergaard, Olesen, Andersen, & Sondergaard, 2009 ; Sandelowski, 2000 , 2010 ; Sullivan-Bolyai, Bova, & Harper, 2005 ; Vaismoradi, Turunen, & Bondas, 2013 ; Willis, Sullivan-Bolyai, Knafl, & Zichi-Cohen, 2016 ). Furthermore, little is known about characteristics of QD as reported in journal-published, nursing-related, qualitative studies. Therefore, the purpose of this systematic review was to describe specific characteristics of methods and findings of studies reported in journal articles (published in 2014) self-labeled as QD. In this review, we did not have a goal to judge whether QD was done correctly but rather to report on the features of the methods and findings.

Features of QD

Several QD design features and techniques have been described in the literature. First, researchers generally draw from a naturalistic perspective and examine a phenomenon in its natural state ( Sandelowski, 2000 ). Second, QD has been described as less theoretical compared to other qualitative approaches ( Neergaard et al., 2009 ), facilitating flexibility in commitment to a theory or framework when designing and conducting a study ( Sandelowski, 2000 , 2010 ). For example, researchers may or may not decide to begin with a theory of the targeted phenomenon and do not need to stay committed to a theory or framework if their investigations take them down another path ( Sandelowski, 2010 ). Third, data collection strategies typically involve individual and/or focus group interviews with minimal to semi-structured interview guides ( Neergaard et al., 2009 ; Sandelowski, 2000 ). Fourth, researchers commonly employ purposeful sampling techniques such as maximum variation sampling which has been described as being useful for obtaining broad insights and rich information ( Neergaard et al., 2009 ; Sandelowski, 2000 ). Fifth, content analysis (and in many cases, supplemented by descriptive quantitative data to describe the study sample) is considered a primary strategy for data analysis ( Neergaard et al., 2009 ; Sandelowski, 2000 ). In some instances thematic analysis may also be used to analyze data; however, experts suggest care should be taken that this type of analysis is not confused with content analysis ( Vaismoradi et al., 2013 ). These data analysis approaches allow researchers to stay close to the data and as such, interpretation is of low-inference ( Neergaard et al., 2009 ), meaning that different researchers will agree more readily on the same findings even if they do not choose to present the findings in the same way ( Sandelowski, 2000 ). Finally, representation of study findings in published reports is expected to be straightforward, including comprehensive descriptive summaries and accurate details of the data collected, and presented in a way that makes sense to the reader ( Neergaard et al., 2009 ; Sandelowski, 2000 ).

It is also important to acknowledge that variations in methods or techniques may be appropriate across QD studies ( Sandelowski, 2010 ). For example, when consistent with the study goals, decisions may be made to use techniques from other qualitative traditions, such as employing a constant comparative analytic approach typically associated with grounded theory ( Sandelowski, 2000 ).

Search Strategy and Study Screening

The PubMed electronic database was searched for articles written in English and published from January 1, 2014 to December 31, 2014, using the terms, “qualitative descriptive study,” “qualitative descriptive design,” and “qualitative description,” combined with “nursing.” This specific publication year, “2014,” was chosen because it was the most recent full year at the time of beginning this systematic review. As we did not intend to identify trends in QD approaches over time, it seemed reasonable to focus on the nursing QD studies published in a certain year. The inclusion criterion for this review was data-based, nursing-related, research articles in which authors used the terms QD, qualitative descriptive study, or qualitative descriptive design in their titles or abstracts as well as in the main texts of the publication.

All articles yielded through an initial search in PubMed were exported into EndNote X7 ( Thomson Reuters, 2014 ), a reference management software, and duplicates were removed. Next, titles and abstracts were reviewed to determine if the publication met inclusion criteria; all articles meeting inclusion criteria were then read independently in full by two authors (HK and JS) to determine if the terms – QD or qualitative descriptive study/design – were clearly stated in the main texts. Any articles in which researchers did not specifically state these key terms in the main text were then excluded, even if the terms had been used in the study title or abstract. In one article, for example, although “qualitative descriptive study” was reported in the published abstract, the researchers reported a “qualitative exploratory design” in the main text of the article ( Sundqvist & Carlsson, 2014 ); therefore, this article was excluded from our review. Despite the possibility that there may be other QD studies published in 2014 that were not labeled as such, to facilitate our screening process we only included articles where the researchers clearly used our search terms for their approach. Finally, the two authors compared, discussed, and reconciled their lists of articles with a third author (CB).

Study Selection

Initially, although the year 2014 was specifically requested, 95 articles were identified (due to ahead of print/Epub) and exported into the EndNote program. Three duplicate publications were removed and the 20 articles with final publication dates of 2015 were also excluded. The remaining 72 articles were then screened by examining titles, abstracts, and full-texts. Based on our inclusion criteria, 15 (of 72) were then excluded because QD or QD design/study was not identified in the main text. We then re-examined the remaining 57 articles and excluded two additional articles that did not meet inclusion criteria (e.g., QD was only reported as an analytic approach in the data analysis section). The remaining 55 publications met inclusion criteria and comprised the sample for our systematic review (see Figure 1 ).

Flow Diagram of Study Selection

Of the 55 publications, 23 originated from North America (17 in the United States; 6 in Canada), 12 from Asia, 11 from Europe, 7 from Australia and New Zealand, and 2 from South America. Eleven studies were part of larger research projects and two of them were reported as part of larger mixed-methods studies. Four were described as a secondary analysis.

Quality Appraisal Process

Following the identification of the 55 publications, two authors (HK and JS) independently examined each article using the Critical Appraisal Skills Programme (CASP) qualitative checklist ( CASP, 2013 ). The CASP was chosen to determine the general adequacy (or rigor) of the qualitative studies included in this review as the CASP criteria are generic and intend to be applied to qualitative studies in general. In addition, the CASP was useful because we were able to examine the internal consistency between study aims and methods and between study aims and findings as well as the usefulness of findings ( CASP, 2013 ). The CASP consists of 10 main questions with several sub-questions to consider when making a decision about the main question ( CASP, 2013 ). The first two questions have reviewers examine the clarity of study aims and appropriateness of using qualitative research to achieve the aims. With the next eight questions, reviewers assess study design, sampling, data collection, and analysis as well as the clarity of the study’s results statement and the value of the research. We used the seven questions and 17 sub-questions related to methods and statement of findings to evaluate the articles. The results of this process are presented in Table 1 .

CASP Questions and Quality Appraisal Results (N = 55)

Note . The CASP questions are adapted from “10 questions to help you make sense of qualitative research,” by Critical Appraisal Skills Programme, 2013, retrieved from http://media.wix.com/ugd/dded87_29c5b002d99342f788c6ac670e49f274.pdf . Its license can be found at http://creativecommons.org/licenses/by-nc-sa/3.0/

Once articles were assessed by the two authors independently, all three authors discussed and reconciled our assessment. No articles were excluded based on CASP results; rather, results were used to depict the general adequacy (or rigor) of all 55 articles meeting inclusion criteria for our systematic review. In addition, the CASP was included to enhance our examination of the relationship between the methods and the usefulness of the findings documented in each of the QD articles included in this review.

Process for Data Extraction and Analysis

To further assess each of the 55 articles, data were extracted on: (a) research objectives, (b) design justification, (c) theoretical or philosophical framework, (d) sampling and sample size, (e) data collection and data sources, (f) data analysis, and (g) presentation of findings (see Table 2 ). We discussed extracted data and identified common and unique features in the articles included in our systematic review. Findings are described in detail below and in Table 3 .

Elements for Data Extraction

Data Extraction and Analysis Results

Note . NR = not reported

Quality Appraisal Results

Justification for use of a QD design was evident in close to half (47.3%) of the 55 publications. While most researchers clearly described recruitment strategies (80%) and data collection methods (100%), justification for how the study setting was selected was only identified in 38.2% of the articles and almost 75% of the articles did not include any reason for the choice of data collection methods (e.g., focus-group interviews). In the vast majority (90.9%) of the articles, researchers did not explain their involvement and positionality during the process of recruitment and data collection or during data analysis (63.6%). Ethical standards were reported in greater than 89% of all articles and most articles included an in-depth description of data analysis (83.6%) and development of categories or themes (92.7%). Finally, all researchers clearly stated their findings in relation to research questions/objectives. Researchers of 83.3% of the articles discussed the credibility of their findings (see Table 1 ).

Research Objectives

In statements of study objectives and/or questions, the most frequently used verbs were “explore” ( n = 22) and “describe” ( n = 17). Researchers also used “identify” ( n = 3), “understand” ( n = 4), or “investigate” ( n = 2). Most articles focused on participants’ experiences related to certain phenomena ( n = 18), facilitators/challenges/factors/reasons ( n = 14), perceptions about specific care/nursing practice/interventions ( n = 11), and knowledge/attitudes/beliefs ( n = 3).

Design Justification

A total of 30 articles included references for QD. The most frequently cited references ( n = 23) were “Whatever happened to qualitative description?” ( Sandelowski, 2000 ) and “What’s in a name? Qualitative description revisited” ( Sandelowski, 2010 ). Other references cited included “Qualitative description – the poor cousin of health research?” ( Neergaard et al., 2009 ), “Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research” ( Pope & Mays, 1995 ), and general research textbooks ( Polit & Beck, 2004 , 2012 ).

In 26 articles (and not necessarily the same as those citing specific references to QD), researchers provided a rationale for selecting QD. Most researchers chose QD because this approach aims to produce a straight description and comprehensive summary of the phenomenon of interest using participants’ language and staying close to the data (or using low inference).

Authors of two articles distinctly stated a QD design, yet also acknowledged grounded-theory or phenomenological overtones by adopting some techniques from these qualitative traditions ( Michael, O'Callaghan, Baird, Hiscock, & Clayton, 2014 ; Peacock, Hammond-Collins, & Forbes, 2014 ). For example, Michael et al. (2014 , p. 1066) reported:

The research used a qualitative descriptive design with grounded theory overtones ( Sandelowski, 2000 ). We sought to provide a comprehensive summary of participants’ views through theoretical sampling; multiple data sources (focus groups [FGs] and interviews); inductive, cyclic, and constant comparative analysis; and condensation of data into thematic representations ( Corbin & Strauss, 1990 , 2008 ).

Authors of four additional articles included language suggestive of a grounded-theory or phenomenological tradition, e.g., by employing a constant comparison technique or translating themes stated in participants’ language into the primary language of the researchers during data analysis ( Asemani et al., 2014 ; Li, Lee, Chen, Jeng, & Chen, 2014 ; Ma, 2014 ; Soule, 2014 ). Additionally, Li et al. (2014) specifically reported use of a grounded-theory approach.

Theoretical or Philosophical Framework

In most (n = 48) articles, researchers did not specify any theoretical or philosophical framework. Of those articles in which a framework or philosophical stance was included, the authors of five articles described the framework as guiding the development of an interview guide ( Al-Zadjali, Keller, Larkey, & Evans, 2014 ; DeBruyn, Ochoa-Marin, & Semenic, 2014 ; Fantasia, Sutherland, Fontenot, & Ierardi, 2014 ; Ma, 2014 ; Wiens, Babenko-Mould, & Iwasiw, 2014 ). In two articles, data analysis was described as including key concepts of a framework being used as pre-determined codes or categories ( Al-Zadjali et al., 2014 ; Wiens et al., 2014 ). Oosterveld-Vlug et al. (2014) and Zhang, Shan, and Jiang (2014) discussed a conceptual model and underlying philosophy in detail in the background or discussion section, although the model and philosophy were not described as being used in developing interview questions or analyzing data.

Sampling and Sample Size

In 38 of the 55 articles, researchers reported ‘purposeful sampling’ or some derivation of purposeful sampling such as convenience ( n = 10), maximum variation ( n = 8), snowball ( n = 3), and theoretical sampling ( n = 1). In three instances ( Asemani et al., 2014 ; Chan & Lopez, 2014 ; Soule, 2014 ), multiple sampling strategies were described, for example, a combination of snowball, convenience, and maximum variation sampling. In articles where maximum variation sampling was employed, “variation” referred to seeking diversity in participants’ demographics ( n = 7; e.g., age, gender, and education level), while one article did not include details regarding how their maximum variation sampling strategy was operationalized ( Marcinowicz, Abramowicz, Zarzycka, Abramowicz, & Konstantynowicz, 2014 ). Authors of 17 articles did not specify their sampling techniques.

Sample sizes ranged from 8 to 1,932 with nine studies in the 8–10 participant range and 24 studies in the 11–20 participant range. The participant range of 21–30 and 31–50 was reported in eight articles each. Six studies included more than 50 participants. Two of these articles depicted quite large sample sizes (N=253, Hart & Mareno, 2014 ; N=1,932, Lyndon et al., 2014 ) and the authors of these articles described the use of survey instruments and analysis of responses to open-ended questions. This was in contrast to studies with smaller sample sizes where individual interviews and focus groups were more commonly employed.

Data Collection and Data Sources

In a majority of studies, researchers collected data through individual ( n = 39) and/or focus-group ( n = 14) interviews that were semistructured. Most researchers reported that interviews were audiotaped ( n = 51) and interview guides were described as the primary data collection tool in 29 of the 51 studies. In some cases, researchers also described additional data sources, for example, taking memos or field notes during participant observation sessions or as a way to reflect their thoughts about interviews ( n = 10). Written responses to open-ended questions in survey questionnaires were another type of data source in a small number of studies ( n = 4).

Data Analysis

The analysis strategy most commonly used in the QD studies included in this review was qualitative content analysis ( n = 30). Among the studies where this technique was used, most researchers described an inductive approach; researchers of two studies analyzed data both inductively and deductively. Thematic analysis was adopted in 14 studies and the constant comparison technique in 10 studies. In nine studies, researchers employed multiple techniques to analyze data including qualitative content analysis with constant comparison ( Asemani et al., 2014 ; DeBruyn et al., 2014 ; Holland, Christensen, Shone, Kearney, & Kitzman, 2014 ; Li et al., 2014 ) and thematic analysis with constant comparison ( Johansson, Hildingsson, & Fenwick, 2014 ; Oosterveld-Vlug et al., 2014 ). In addition, five teams conducted descriptive statistical analysis using both quantitative and qualitative data and counting the frequencies of codes/themes ( Ewens, Chapman, Tulloch, & Hendricks, 2014 ; Miller, 2014 ; Santos, Sandelowski, & Gualda, 2014 ; Villar, Celdran, Faba, & Serrat, 2014 ) or targeted events through video monitoring ( Martorella, Boitor, Michaud, & Gelinas, 2014 ). Tseng, Chen, and Wang (2014) cited Thorne, Reimer Kirkham, and O’Flynn-Magee (2004)’s interpretive description as the inductive analytic approach. In five out of 55 articles, researchers did not specifically name their analysis strategies, despite including descriptions about procedural aspects of data analysis. Researchers of 20 studies reported that data saturation for their themes was achieved.

Presentation of Findings

Researchers described participants’ experiences of health care, interventions, or illnesses in 18 articles and presented straightforward, focused, detailed descriptions of facilitators, challenges, factors, reasons, and causes in 15 articles. Participants’ perceptions of specific care, interventions, or programs were described in detail in 11 articles. All researchers presented their findings with extensive descriptions including themes or categories. In 25 of 55 articles, figures or tables were also presented to illustrate or summarize the findings. In addition, the authors of three articles summarized, organized, and described their data using key concepts of conceptual models ( Al-Zadjali et al., 2014 ; Oosterveld-Vlug et al., 2014 ; Wiens et al., 2014 ). Martorella et al. (2014) assessed acceptability and feasibility of hand massage therapy and arranged their findings in relation to pre-determined indicators of acceptability and feasibility. In one longitudinal QD study ( Kneck, Fagerberg, Eriksson, & Lundman, 2014 ), the researchers presented the findings as several key patterns of learning for persons living with diabetes; in another longitudinal QD study ( Stegenga & Macpherson, 2014 ), findings were presented as processes and themes regarding patients’ identity work across the cancer trajectory. In another two studies, the researchers described and compared themes or categories from two different perspectives, such as patients and nurses ( Canzan, Heilemann, Saiani, Mortari, & Ambrosi, 2014 ) or parents and children ( Marcinowicz et al., 2014 ). Additionally, Ma (2014) reported themes using both participants’ language and the researcher’s language.

In this systematic review, we examined and reported specific characteristics of methods and findings reported in journal articles self-identified as QD and published during one calendar year. To accomplish this we identified 55 articles that met inclusion criteria, performed a quality appraisal following CASP guidelines, and extracted and analyzed data focusing on QD features. In general, three primary findings emerged. First, despite inconsistencies, most QD publications had the characteristics that were originally observed by Sandelowski (2000) and summarized by other limited available QD literature. Next, there are no clear boundaries in methods used in the QD studies included in this review; in a number of studies, researchers adopted and combined techniques originating from other qualitative traditions to obtain rich data and increase their understanding of the phenomenon under investigation. Finally, justification for how QD was chosen and why it would be an appropriate fit for a particular study is an area in need of increased attention.

In general, the overall characteristics were consistent with design features of QD studies described in the literature ( Neergaard et al., 2009 ; Sandelowski, 2000 , 2010 ; Vaismoradi et al., 2013 ). For example, many authors reported that study objectives were to describe or explore participants’ experiences and factors related to certain phenomena, events, or interventions. In most cases, these authors cited Sandelowski (2000) as a reference for this particular characteristic. It was rare that theoretical or philosophical frameworks were identified, which also is consistent with descriptions of QD. In most studies, researchers used purposeful sampling and its derivative sampling techniques, collected data through interviews, and analyzed data using qualitative content analysis or thematic analysis. Moreover, all researchers presented focused or comprehensive, descriptive summaries of data including themes or categories answering their research questions. These characteristics do not indicate that there are correct ways to do QD studies; rather, they demonstrate how others designed and produced QD studies.

In several studies, researchers combined techniques that originated from other qualitative traditions for sampling, data collection, and analysis. This flexibility or variability, a key feature of recently published QD studies, may indicate that there are no clear boundaries in designing QD studies. Sandelowski (2010) articulated: “in the actual world of research practice, methods bleed into each other; they are so much messier than textbook depictions” (p. 81). Hammersley (2007) also observed:

“We are not so much faced with a set of clearly differentiated qualitative approaches as with a complex landscape of variable practice in which the inhabitants use a range of labels (‘ethnography’, ‘discourse analysis’, ‘life history work’, narrative study’, ……, and so on) in diverse and open-ended ways in order to characterize their orientation, and probably do this somewhat differently across audiences and occasions” (p. 293).

This concept of having no clear boundaries in methods when designing a QD study should enable researchers to obtain rich data and produce a comprehensive summary of data through various data collection and analysis approaches to answer their research questions. For example, using an ethnographical approach (e.g., participant observation) in data collection for a QD study may facilitate an in-depth description of participants’ nonverbal expressions and interactions with others and their environment as well as situations or events in which researchers are interested ( Kawulich, 2005 ). One example found in our review is that Adams et al. (2014) explored family members’ responses to nursing communication strategies for patients in intensive care units (ICUs). In this study, researchers conducted interviews with family members, observed interactions between healthcare providers, patients, and family members in ICUs, attended ICU rounds and family meetings, and took field notes about their observations and reflections. Accordingly, the variability in methods provided Adams and colleagues (2014) with many different aspects of data that were then used to complement participants’ interviews (i.e., data triangulation). Moreover, by using a constant comparison technique in addition to qualitative content analysis or thematic analysis in QD studies, researchers compare each case with others looking for similarities and differences as well as reasoning why differences exist, to generate more general understanding of phenomena of interest ( Thorne, 2000 ). In fact, this constant comparison analysis is compatible with qualitative content analysis and thematic analysis and we found several examples of using this approach in studies we reviewed ( Asemani et al., 2014 ; DeBruyn et al., 2014 ; Holland et al., 2014 ; Johansson et al., 2014 ; Li et al., 2014 ; Oosterveld-Vlug et al., 2014 ).

However, this flexibility or variability in methods of QD studies may cause readers’ as well as researchers’ confusion in designing and often labeling qualitative studies ( Neergaard et al., 2009 ). Especially, it could be difficult for scholars unfamiliar with qualitative studies to differentiate QD studies with “hues, tones, and textures” of qualitative traditions ( Sandelowski, 2000 , p. 337) from grounded theory, phenomenological, and ethnographical research. In fact, the major difference is in the presentation of the findings (or outcomes of qualitative research) ( Neergaard et al., 2009 ; Sandelowski, 2000 ). The final products of grounded theory, phenomenological, and ethnographical research are a generation of a theory, a description of the meaning or essence of people’s lived experience, and an in-depth, narrative description about certain culture, respectively, through researchers’ intensive/deep interpretations, reflections, and/or transformation of data ( Streubert & Carpenter, 2011 ). In contrast, QD studies result in “a rich, straight description” of experiences, perceptions, or events using language from the collected data ( Neergaard et al., 2009 ) through low-inference (or data-near) interpretations during data analysis ( Sandelowski, 2000 , 2010 ). This feature is consistent with our finding regarding presentation of findings: in all QD articles included in this systematic review, the researchers presented focused or comprehensive, descriptive summaries to their research questions.

Finally, an explanation or justification of why a QD approach was chosen or appropriate for the study aims was not found in more than half of studies in the sample. While other qualitative approaches, including grounded theory, phenomenology, ethnography, and narrative analysis, are used to better understand people’s thoughts, behaviors, and situations regarding certain phenomena ( Sullivan-Bolyai et al., 2005 ), as noted above, the results will likely read differently than those for a QD study ( Carter & Little, 2007 ). Therefore, it is important that researchers accurately label and justify their choices of approach, particularly for studies focused on participants’ experiences, which could be addressed with other qualitative traditions. Justifying one’s research epistemology, methodology, and methods allows readers to evaluate these choices for internal consistency, provides context to assist in understanding the findings, and contributes to the transparency of choices, all of which enhance the rigor of the study ( Carter & Little, 2007 ; Wu, Thompson, Aroian, McQuaid, & Deatrick, 2016 ).

Use of the CASP tool drew our attention to the credibility and usefulness of the findings of the QD studies included in this review. Although justification for study design and methods was lacking in many articles, most authors reported techniques of recruitment, data collection, and analysis that appeared. Internal consistencies among study objectives, methods, and findings were achieved in most studies, increasing readers’ confidence that the findings of these studies are credible and useful in understanding under-explored phenomenon of interest.

In summary, our findings support the notion that many scholars employ QD and include a variety of commonly observed characteristics in their study design and subsequent publications. Based on our review, we found that QD as a scholarly approach allows flexibility as research questions and study findings emerge. We encourage authors to provide as many details as possible regarding how QD was chosen for a particular study as well as details regarding methods to facilitate readers’ understanding and evaluation of the study design and rigor. We acknowledge the challenge of strict word limitation with submissions to print journals; potential solutions include collaboration with journal editors and staff to consider creative use of charts or tables, or using more citations and less text in background sections so that methods sections are robust.

Limitations

Several limitations of this review deserve mention. First, only articles where researchers explicitly stated in the main body of the article that a QD design was employed were included. In contrast, articles labeled as QD in only the title or abstract, or without their research design named were not examined due to the lack of certainty that the researchers actually carried out a QD study. As a result, we may have excluded some studies where a QD design was followed. Second, only one database was searched and therefore we did not identify or describe potential studies following a QD approach that were published in non-PubMed databases. Third, our review is limited by reliance on what was included in the published version of a study. In some cases, this may have been a result of word limits or specific styles imposed by journals, or inconsistent reporting preferences of authors and may have limited our ability to appraise the general adequacy with the CASP tool and examine specific characteristics of these studies.

Conclusions

A systematic review was conducted by examining QD research articles focused on nursing-related phenomena and published in one calendar year. Current patterns include some characteristics of QD studies consistent with the previous observations described in the literature, a focus on the flexibility or variability of methods in QD studies, and a need for increased explanations of why QD was an appropriate label for a particular study. Based on these findings, recommendations include encouragement to authors to provide as many details as possible regarding the methods of their QD study. In this way, readers can thoroughly consider and examine if the methods used were effective and reasonable in producing credible and useful findings.

Acknowledgments

This work was supported in part by the John A. Hartford Foundation’s National Hartford Centers of Gerontological Nursing Excellence Award Program.

Hyejin Kim is a Ruth L. Kirschstein NRSA Predoctoral Fellow (F31NR015702) and 2013–2015 National Hartford Centers of Gerontological Nursing Excellence Patricia G. Archbold Scholar. Justine Sefcik is a Ruth L. Kirschstein Predoctoral Fellow (F31NR015693) through the National Institutes of Health, National Institute of Nursing Research.

Conflict of Interest Statement

The Authors declare that there is no conflict of interest.

Contributor Information

Hyejin Kim, MSN, CRNP, Doctoral Candidate, University of Pennsylvania School of Nursing.

Justine S. Sefcik, MS, RN, Doctoral Candidate, University of Pennsylvania School of Nursing.

Christine Bradway, PhD, CRNP, FAAN, Associate Professor of Gerontological Nursing, University of Pennsylvania School of Nursing.

• Adams JA, Anderson RA, Docherty SL, Tulsky JA, Steinhauser KE, Bailey DE., Jr Nursing strategies to support family members of ICU patients at high risk of dying. Heart & Lung. 2014; 43 (5):406–415. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Ahlin J, Ericson-Lidman E, Norberg A, Strandberg G. Care providers' experiences of guidelines in daily work at a municipal residential care facility for older people. Scandinavian Journal of Caring Sciences. 2014; 28 (2):355–363. [ PubMed ] [ Google Scholar ]
• Al-Zadjali M, Keller C, Larkey L, Evans B. GCC women: causes and processes of midlife weight gain. Health Care for Women International. 2014; 35 (11–12):1267–1286. [ PubMed ] [ Google Scholar ]
• Asemani O, Iman MT, Moattari M, Tabei SZ, Sharif F, Khayyer M. An exploratory study on the elements that might affect medical students' and residents' responsibility during clinical training. Journal of Medical Ethics and History of Medicine. 2014; 7 :8. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Atefi N, Abdullah KL, Wong LP, Mazlom R. Factors influencing registered nurses perception of their overall job satisfaction: a qualitative study. International Nursing Review. 2014; 61 (3):352–360. [ PubMed ] [ Google Scholar ]
• Ballangrud R, Hall-Lord ML, Persenius M, Hedelin B. Intensive care nurses' perceptions of simulation-based team training for building patient safety in intensive care: a descriptive qualitative study. Intensive and Critical Care Nursing. 2014; 30 (4):179–187. [ PubMed ] [ Google Scholar ]
• Benavides-Vaello S, Katz JR, Peterson JC, Allen CB, Paul R, Charette-Bluff AL, Morris P. Nursing and health sciences workforce diversity research using. PhotoVoice: a college and high school student participatory project. Journal of Nursing Education. 2014; 53 (4):217–222. [ PubMed ] [ Google Scholar ]
• Bernhard C, Zielinski R, Ackerson K, English J. Home birth after hospital birth: women's choices and reflections. Journal of Midwifery and Women's Health. 2014; 59 (2):160–166. [ PubMed ] [ Google Scholar ]
• Borbasi S, Jackson D, Langford RW. Navigating the maze of nursing research: An interactive learning adventure. 2nd. New South Wales, Australia: Mosby/Elsevier; 2008. [ Google Scholar ]
• Bradford B, Maude R. Fetal response to maternal hunger and satiation - novel finding from a qualitative descriptive study of maternal perception of fetal movements. BMC Pregnancy and Childbirth. 2014; 14 :288. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Burns N, Grove SK. The practice of nursing research: Conduct, critique, & utilization. 5th. Philadelphia, PA: Elsevier/Saunders; 2005. [ Google Scholar ]
• Canzan F, Heilemann MV, Saiani L, Mortari L, Ambrosi E. Visible and invisible caring in nursing from the perspectives of patients and nurses in the gerontological context. Scandinavian Journal of Caring Sciences. 2014; 28 (4):732–740. [ PubMed ] [ Google Scholar ]
• Carter SM, Littler M. Justifying knowledge, justifying methods, taking action: Epistemologies, methodologies, and methods in qualitative research. Qualitative Health Research. 2007; 17 (10):1316–1328. [ PubMed ] [ Google Scholar ]
• Critical Appraisal Skills Programme (CASP 2013) 10 questions to help you make sense of qualitative research. Oxford: CASP; 2013. Retrieved from http://media.wix.com/ugd/dded87_29c5b002d99342f788c6ac670e49f274.pdf . [ Google Scholar ]
• Chan CW, Lopez V. A qualitative descriptive study of risk reduction for coronary disease among the Hong Kong Chinese. Public Health Nursing. 2014; 31 (4):327–335. [ PubMed ] [ Google Scholar ]
• Chen YJ, Tsai YF, Lee SH, Lee HL. Protective factors against suicide among young-old Chinese outpatients. BMC Public Health. 2014; 14 :372. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Cleveland LM, Bonugli R. Experiences of mothers of infants with neonatal abstinence syndrome in the neonatal intensive care unit. Journal of Obstetric Gynecologic, & Neonatal Nursing. 2014; 43 (3):318–329. [ PubMed ] [ Google Scholar ]
• Corbin J, Strauss A. Basics of qualitative research: Techniques and procedures for developing grounded theory. 3rd. Thousand Oaks, CA: Sage Publications; 2008. [ Google Scholar ]
• Corbin JM, Strauss A. Grounded theory research: Procedures, canons and evaluation criteria. Qualitative Sociology. 1990; 13 (1):3–21. [ Google Scholar ]
• DeBruyn RR, Ochoa-Marin SC, Semenic S. Barriers and facilitators to evidence-based nursing in Colombia: perspectives of nurse educators, nurse researchers and graduate students. Investigación y Educación en Enfermería. 2014; 32 (1):9–21. [ PubMed ] [ Google Scholar ]
• Denzin NK, Lincoln YS. The Discipline and practice of qualitative research. In: Denzin NK, Lincoln YS, editors. Handbook of qualitative research. 2nd. Thousand Oaks, CA: Sage Publications; 2000. pp. 1–28. [ Google Scholar ]
• Ewens B, Chapman R, Tulloch A, Hendricks JM. ICU survivors' utilisation of diaries post discharge: a qualitative descriptive study. Australian Critical Care. 2014; 27 (1):28–35. [ PubMed ] [ Google Scholar ]
• Fantasia HC, Sutherland MA, Fontenot H, Ierardi JA. Knowledge, attitudes and beliefs about contraceptive and sexual consent negotiation among college women. Journal of Forensic Nursing. 2014; 10 (4):199–207. [ PubMed ] [ Google Scholar ]
• Friman A, Wahlberg AC, Mattiasson AC, Ebbeskog B. District nurses' knowledge development in wound management: ongoing learning without organizational support. Primary Health Care Research & Development. 2014; 15 (4):386–395. [ PubMed ] [ Google Scholar ]
• Gaughan V, Logan D, Sethna N, Mott S. Parents' perspective of their journey caring for a child with chronic neuropathic pain. Pain Management Nursing. 2014; 15 (1):246–257. [ PubMed ] [ Google Scholar ]
• Hammersley M. The issue of quality in qualitative research. International Journal of Research & Method in Education. 2007; 30 (3):287–305. [ Google Scholar ]
• Hart PL, Mareno N. Cultural challenges and barriers through the voices of nurses. Journal of Clinical Nursing. 2014; 23 (15–16):2223–2232. [ PubMed ] [ Google Scholar ]
• Hasman K, Kjaergaard H, Esbensen BA. Fathers' experience of childbirth when non-progressive labour occurs and augmentation is established. A qualitative study. Sexual & Reproductive HealthCare. 2014; 5 (2):69–73. [ PubMed ] [ Google Scholar ]
• Higgins I, van der Riet P, Sneesby L, Good P. Nutrition and hydration in dying patients: the perceptions of acute care nurses. Journal of Clinical Nursing. 2014; 23 (17–18):2609–2617. [ PubMed ] [ Google Scholar ]
• Holland ML, Christensen JJ, Shone LP, Kearney MH, Kitzman HJ. Women's reasons for attrition from a nurse home visiting program. Journal of Obstetric, Gynecologic, & Neonatal Nursing. 2014; 43 (1):61–70. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Johansson M, Hildingsson I, Fenwick J. 'As long as they are safe--birth mode does not matter' Swedish fathers' experiences of decision-making around caesarean section. Women and Birth. 2014; 27 (3):208–213. [ PubMed ] [ Google Scholar ]
• Kao MH, Tsai YF. Illness experiences in middle-aged adults with early-stage knee osteoarthritis: findings from a qualitative study. Journal of Advanced Nursing. 2014; 70 (7):1564–1572. [ PubMed ] [ Google Scholar ]
• Kawulich BB. Participant observation as a data collection method. Forum: Qualitative Social Research. 2005; 6 (2) Art. 43. Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/466/997 . [ Google Scholar ]
• Kerr D, McKay K, Klim S, Kelly AM, McCann T. Attitudes of emergency department patients about handover at the bedside. Journal of Clinical Nursing. 2014; 23 (11–12):1685–1693. [ PubMed ] [ Google Scholar ]
• Kneck A, Fagerberg I, Eriksson LE, Lundman B. Living with diabetes - development of learning patterns over a 3-year period. International Journal of Qualitative Studies on Health and Well-being. 2014; 9 :24375. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Krippendorf K. Content analysis: An introduction to its methodology. 2nd. Thousand Oaks, CA: Sage Publications; 2004. [ Google Scholar ]
• Larocque N, Schotsman C, Kaasalainen S, Crawshaw D, McAiney C, Brazil E. Using a book chat to improve attitudes and perceptions of long-term care staff about dementia. Journal of Gerontological Nursing. 2014; 40 (5):46–52. [ PubMed ] [ Google Scholar ]
• Li IC, Lee SY, Chen CY, Jeng YQ, Chen YC. Facilitators and barriers to effective smoking cessation: counselling services for inpatients from nurse-counsellors' perspectives--a qualitative study. International Journal of Environmental Research and Public Health. 2014; 11 (5):4782–4798. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lux KM, Hutcheson JB, Peden AR. Ending disruptive behavior: staff nurse recommendations to nurse educators. Nurse Education in Practice. 2014; 14 (1):37–42. [ PubMed ] [ Google Scholar ]
• Lyndon A, Zlatnik MG, Maxfield DG, Lewis A, McMillan C, Kennedy HP. Contributions of clinical disconnections and unresolved conflict to failures in intrapartum safety. Journal of Obstetric, Gynecologic, & Neonatal Nursing. 2014; 43 (1):2–12. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Ma F, Li J, Liang H, Bai Y, Song J. Baccalaureate nursing students' perspectives on learning about caring in China: a qualitative descriptive study. BMC Medical Education. 2014; 14 :42. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Ma L. A humanbecoming qualitative descriptive study on quality of life with older adults. Nursing Science Quarterly. 2014; 27 (2):132–141. [ PubMed ] [ Google Scholar ]
• Marcinowicz L, Abramowicz P, Zarzycka D, Abramowicz M, Konstantynowicz J. How hospitalized children and parents perceive nurses and hospital amenities: A qualitative descriptive study in Poland. Journal of Child Health Care. 2014 [ PubMed ] [ Google Scholar ]
• Martorella G, Boitor M, Michaud C, Gelinas C. Feasibility and acceptability of hand massage therapy for pain management of postoperative cardiac surgery patients in the intensive care unit. Heart & Lung. 2014; 43 (5):437–444. [ PubMed ] [ Google Scholar ]
• McDonough A, Callans KM, Carroll DL. Understanding the challenges during transitions of care for children with critical airway conditions. ORL Head and Neck Nursing. 2014; 32 (4):12–17. [ PubMed ] [ Google Scholar ]
• McGilton KS, Boscart VM, Brown M, Bowers B. Making tradeoffs between the reasons to leave and reasons to stay employed in long-term care homes: perspectives of licensed nursing staff. International Journal of Nursing Studies. 2014; 51 (6):917–926. [ PubMed ] [ Google Scholar ]
• Michael N, O'Callaghan C, Baird A, Hiscock N, Clayton J. Cancer caregivers advocate a patient- and family-centered approach to advance care planning. Journal of Pain and Symptom Management. 2014; 47 (6):1064–1077. [ PubMed ] [ Google Scholar ]
• Miller WR. Patient-centered outcomes in older adults with epilepsy. Seizure. 2014; 23 (8):592–597. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Milne J, Oberle K. Enhancing rigor in qualitative description: a case study. Journal of Wound Ostomy & Continence Nursing. 2005; 32 (6):413–420. [ PubMed ] [ Google Scholar ]
• Neergaard MA, Olesen F, Andersen RS, Sondergaard J. Qualitative description - the poor cousin of health research? BMC Medical Research Methodology. 2009; 9 :52. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• O'Shea MF. Staff nurses' perceptions regarding palliative care for hospitalized older adults. The American Journal of Nursing. 2014; 114 (11):26–34. [ PubMed ] [ Google Scholar ]
• Oosterveld-Vlug MG, Pasman HR, van Gennip IE, Muller MT, Willems DL, Onwuteaka-Philipsen BD. Dignity and the factors that influence it according to nursing home residents: a qualitative interview study. Journal of Advanced Nursing. 2014; 70 (1):97–106. [ PubMed ] [ Google Scholar ]
• Oruche UM, Draucker C, Alkhattab H, Knopf A, Mazurcyk J. Interventions for family members of adolescents with disruptive behavior disorders. Journal of Child and Adolescent Psychiatric Nursing. 2014; 27 (3):99–108. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Parse RR. Qualitative inquiry: The path of sciencing. Sudbury, MA: Jones and Barlett; 2001. [ Google Scholar ]
• Peacock SC, Hammond-Collins K, Forbes DA. The journey with dementia from the perspective of bereaved family caregivers: a qualitative descriptive study. BMC Nursing. 2014; 13 (1):42. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Peterson WE, Sprague AE, Reszel J, Walker M, Fell DB, Perkins SL, Johnson M. Women's perspectives of the fetal fibronectin testing process: a qualitative descriptive study. BMC Pregnancy and Childbirth. 2014; 14 :190. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Polit DF, Beck CT. Nursing research: principles and methods. 7. Philadelphia, PA: Lippincott Williams & Wilkins; 2004. [ Google Scholar ]
• Polit DF, Beck CT. International differences in nursing research, 2005–2006. Journal of Nursing Scholarship. 2009; 41 (1):44–53. [ PubMed ] [ Google Scholar ]
• Polit DF, Beck CT. Nursing research: generating and assessing evidence for nursing practice. 9. Philadelphia, PA: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2012. [ Google Scholar ]
• Polit DF, Beck CT. Essentials of Nursing Research: Appraising Evidence for Nursing Practice. 8. Philadelphia, PA: Wolters Kluwer Health; Lippincott Willians & Wilkins; 2014. Supplement for Chapter 14: Qualitative Descriptive Studies. Retrieved from http://downloads.lww.com/wolterskluwer_vitalstream_com/sample-content/9781451176797_Polit/samples/CS_Chapter_14.pdf . [ Google Scholar ]
• Pope C, Mays N. Qualitative research in health care. 3rd. Victoria, Australia: Blackwell Publishing; 2006. [ Google Scholar ]
• Pope C, Mays N. Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ. 1995; 311 (6996):42–45. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Raphael D, Waterworth S, Gott M. The role of practice nurses in providing palliative and end-of-life care to older patients with long-term conditions. International Journal of Palliative Nursing. 2014; 20 (8):373–379. [ PubMed ] [ Google Scholar ]
• Saldana J. Longitudinal qualitative research: Analyzing change through time. Walnut Creek, CA: AltaMira Press; 2003. [ Google Scholar ]
• Sandelowski M. Whatever happened to qualitative description? Research in Nursing & Health. 2000; 23 (4):334–340. [ PubMed ] [ Google Scholar ]
• Sandelowski M. What's in a name? Qualitative description revisited. Research in Nursing & Health. 2010; 33 (1):77–84. [ PubMed ] [ Google Scholar ]
• Santos HP, Jr, Sandelowski M, Gualda DM. Bad thoughts: Brazilian women's responses to mothering while experiencing postnatal depression. Midwifery. 2014; 30 (6):788–794. [ PubMed ] [ Google Scholar ]
• Sharp R, Grech C, Fielder A, Mikocka-Walus A, Cummings M, Esterman A. The patient experience of a peripherally inserted central catheter (PICC): A qualitative descriptive study. Contemporary Nurse. 2014; 48 (1):26–35. [ PubMed ] [ Google Scholar ]
• Soule I. Cultural competence in health care: an emerging theory. ANS Advances in Nursing Science. 2014; 37 (1):48–60. [ PubMed ] [ Google Scholar ]
• Stegenga K, Macpherson CF. "I'm a survivor, go study that word and you'll see my name": adolescent and cancer identity work over the first year after diagnosis. Cancer Nursing. 2014; 37 (6):418–428. [ PubMed ] [ Google Scholar ]
• Streubert HJ, Carpenter DR. Qualitative research in nursing: Advancing the humanistic imperative. 5th. Philadelphia, PA: Lippincott Williams & Wilkins; 2011. [ Google Scholar ]
• Sturesson A, Ziegert K. Prepare the patient for future challenges when facing hemodialysis: nurses' experiences. International Journal of Qualitative Studies on Health and Well-being. 2014; 9 :22952. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sullivan-Bolyai S, Bova C, Harper D. Developing and refining interventions in persons with health disparities: the use of qualitative description. Nursing Outlook. 2005; 53 (3):127–133. [ PubMed ] [ Google Scholar ]
• Sundqvist AS, Carlsson AA. Holding the patient's life in my hands: Swedish registered nurse anaesthetists' perspective of advocacy. Scandinavian Journal of Caring Sciences. 2014; 28 (2):281–288. [ PubMed ] [ Google Scholar ]
• Thomson Reuters. EndNote X7. 2014 Retrieved from http://endnote.com/product-details/x7 .
• Thorne S. Data analysis in qualitative research. Evidence Based Nursing. 2000; 3 :68–70. [ Google Scholar ]
• Thorne S, Reimer Kirkham S, O’Flynn-Magee K. The analytic challenge in interpretive description. International Journal of Qualitative Methods. 2004; 3 (1):1–11. [ Google Scholar ]
• Tseng YF, Chen CH, Wang HH. Taiwanese women's process of recovery from stillbirth: a qualitative descriptive study. Research in Nursing & Health. 2014; 37 (3):219–228. [ PubMed ] [ Google Scholar ]
• Vaismoradi M, Jordan S, Turunen H, Bondas T. Nursing students' perspectives of the cause of medication errors. Nurse Education Today. 2014; 34 (3):434–440. [ PubMed ] [ Google Scholar ]
• Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & Health Sciences. 2013; 15 (3):398–405. [ PubMed ] [ Google Scholar ]
• Valizadeh L, Zamanzadeh V, Fooladi MM, Azadi A, Negarandeh R, Monadi M. The image of nursing, as perceived by Iranian male nurses. Nursing & Health Sciences. 2014; 16 (3):307–313. [ PubMed ] [ Google Scholar ]
• Villar F, Celdran M, Faba J, Serrat R. Barriers to sexual expression in residential aged care facilities (RACFs): comparison of staff and residents' views. Journal of Advanced Nursing. 2014; 70 (11):2518–2527. [ PubMed ] [ Google Scholar ]
• Wiens S, Babenko-Mould Y, Iwasiw C. Clinical instructors' perceptions of structural and psychological empowerment in academic nursing environments. Journal of Nursing Education. 2014; 53 (5):265–270. [ PubMed ] [ Google Scholar ]
• Willis DG, Sullivan-Bolyai S, Knafl K, Zichi-Cohen M. Distinguishing Features and Similarities Between Descriptive Phenomenological and Qualitative Description Research. West J Nurs Res. 2016 [ PubMed ] [ Google Scholar ]
• Wu YP, Thompson D, Aroian KJ, McQuaid EL, Deatrick JA. Commentary: Writing and Evaluating Qualitative Research Reports. J Pediatr Psychol. 2016; 41 (5):493–505. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Zhang H, Shan W, Jiang A. The meaning of life and health experience for the Chinese elderly with chronic illness: a qualitative study from positive health philosophy. International Journal of Nursing Practice. 2014; 20 (5):530–539. [ PubMed ] [ Google Scholar ]

Pardon Our Interruption

As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

• You've disabled JavaScript in your web browser.
• You're a power user moving through this website with super-human speed.
• You've disabled cookies in your web browser.
• A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article .

To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.

• Open access
• Published: 26 August 2024

Evaluating panel discussions in ESP classes: an exploration of international medical students’ and ESP instructors’ perspectives through qualitative research

• Elham Nasiri   ORCID: orcid.org/0000-0002-0644-1646 1 &
• Laleh Khojasteh   ORCID: orcid.org/0000-0002-6393-2759 1  

BMC Medical Education volume  24 , Article number:  925 ( 2024 ) Cite this article

65 Accesses

Metrics details

This study investigates the effectiveness of panel discussions, a specific interactive teaching technique where a group of students leads a pre-planned, topic-focused discussion with audience participation, in English for Specific Purposes (ESP) courses for international medical students. This approach aims to simulate professional conference discussions, preparing students for future academic and clinical environments where such skills are crucial. While traditional group presentations foster critical thinking and communication, a gap exists in understanding how medical students perceive the complexities of preparing for and participating in panel discussions within an ESP setting. This qualitative study investigates the perceived advantages and disadvantages of these discussions from the perspectives of both panelists (medical students) and the audience (peers). Additionally, the study explores potential improvements based on insights from ESP instructors. Utilizing a two-phase design involving reflection papers and focus group discussions, data were collected from 46 medical students and three ESP instructors. Thematic analysis revealed that panel discussions offer unique benefits compared to traditional presentations, including enhanced engagement and more dynamic skill development for both panelists and the audience. Panelists reported gains in personal and professional development, including honing critical thinking, communication, and presentation skills. The audience perceived these discussions as engaging learning experiences that fostered critical analysis and information synthesis. However, challenges such as academic workload and concerns about discussion quality were also identified. The study concludes that panel discussions, when implemented effectively, can be a valuable tool for enhancing critical thinking, communication skills, and subject matter knowledge in ESP courses for medical students. These skills are transferable and can benefit students in various academic and professional settings, including future participation in medical conferences. This research provides valuable insights for ESP instructors seeking to integrate panel discussions into their curriculum, ultimately improving student learning outcomes and preparing them for future success in professional communication.

Peer Review reports

Introduction

In the field of medical education, the acquisition and application of effective communication skills are crucial for medical students in today’s global healthcare environment [ 1 ]. This necessitates not only strong English language proficiency but also the ability to present complex medical information clearly and concisely to diverse audiences.

Language courses, especially English for Specific Purposes (ESP) courses for medical students, are highly relevant in today’s globalized healthcare environment [ 2 ]. In non-English speaking countries like Iran, these courses are particularly important as they go beyond mere language instruction to include the development of critical thinking, cultural competence, and professional communication skills [ 3 ]. Proficiency in English is crucial for accessing up-to-date research, participating in international conferences, and communicating with patients and colleagues from diverse backgrounds [ 4 ]. Additionally, ESP courses help medical students understand and use medical terminologies accurately, which is essential for reading technical articles, listening to audio presentations, and giving spoken presentations [ 5 ]. In countries where English is not the primary language, ESP courses ensure that medical professionals can stay current with global advancements and collaborate effectively on an international scale [ 6 ]. Furthermore, these courses support students who may seek to practice medicine abroad, enhancing their career opportunities and professional growth [ 7 ].

Moreover, ESP courses enable medical professionals to communicate effectively with international patients, which is crucial in multicultural societies and for medical tourism, ensuring that patient care is not compromised due to language barriers [ 8 ]. Many medical textbooks, journals, and online resources are available primarily in English, and ESP courses equip medical students with the necessary language skills to access and comprehend these resources, ensuring they are well-informed about the latest medical research and practices [ 9 ].

Additionally, many medical professionals from non-English speaking countries aim to take international certification exams, such as the USMLE or PLAB, which are conducted in English, and ESP courses prepare students for these exams by familiarizing them with the medical terminology and language used in these assessments [ 10 ]. ESP courses also contribute to the professional development of medical students by improving their ability to write research papers, case reports, and other academic documents in English, which is essential for publishing in international journals and contributing to global medical knowledge [ 11 ]. In the increasingly interdisciplinary field of healthcare, collaboration with professionals from other countries is common, and ESP courses facilitate effective communication and collaboration with international colleagues, fostering innovation and the exchange of ideas [ 12 ].

With the rise of telemedicine and online medical consultations, proficiency in English is essential for non-English speaking medical professionals to provide remote healthcare services to international patients, and ESP courses prepare students for these modern medical practices [ 13 ].

Finally, ESP courses often include training on cultural competence, which is crucial for understanding and respecting the cultural backgrounds of patients and colleagues, leading to more empathetic and effective patient care and professional interactions [ 14 ]. Many ESP programs for medical students incorporate group presentations as a vital component of their curriculum, recognizing the positive impact on developing these essential skills [ 15 ].

Group projects in language courses, particularly in ESP for medical students, are highly relevant for several reasons. They provide a collaborative environment that mimics real-world professional settings, where healthcare professionals often work in multidisciplinary teams [ 16 ]. These group activities foster not only language skills but also crucial soft skills such as teamwork, leadership, and interpersonal communication, which are essential in medical practice [ 17 ].

The benefits of group projects over individual projects in language learning are significant. Hartono, Mujiyanto [ 18 ] found that group presentation tasks in ESP courses led to higher self-efficacy development compared to individual tasks. Group projects encourage peer learning, where students can learn from each other’s strengths and compensate for individual weaknesses [ 19 ]. They also provide a supportive environment that can reduce anxiety and increase willingness to communicate in the target language [ 20 ]. However, it is important to note that group projects also come with challenges, such as social loafing and unequal contribution, which need to be managed effectively [ 21 ].

Traditional lecture-based teaching methods, while valuable for knowledge acquisition, may not effectively prepare medical students for the interactive and collaborative nature of real-world healthcare settings [ 22 ]. Panel discussions (hereafter PDs), an interactive teaching technique where a group of students leads a pre-planned, topic-focused discussion with audience participation, are particularly relevant in this context. They simulate professional conference discussions and interdisciplinary team meetings, preparing students for future academic and clinical environments where such skills are crucial [ 23 ].

PDs, also known as moderated discussions or moderated panels, are a specific type of interactive format where a group of experts or stakeholders engage in a facilitated conversation on a particular topic or issue [ 22 ]. In this format, a moderator guides the discussion, encourages active participation from all panelists, and fosters a collaborative environment that promotes constructive dialogue and critical thinking [ 24 ]. The goal is to encourage audience engagement and participation, which can be achieved through various strategies such as asking open-ended questions, encouraging counterpoints and counterarguments, and providing opportunities for audience members to pose questions or share their own experiences [ 25 ]. These discussions can take place in-person or online, and can be designed to accommodate diverse audiences and settings [ 26 ].

In this study, PD is considered a speaking activity where medical students are assigned specific roles to play during the simulation, such as a physician, quality improvement specialist, policymaker, or patient advocate. By taking on these roles, students can gain a better understanding of the diverse perspectives and considerations that come into play in real-world healthcare discussions [ 23 ]. Simulating PDs within ESP courses can be a powerful tool for enhancing medical students’ learning outcomes in multiple areas. This approach improves language proficiency, academic skills, and critical thinking abilities, while also enabling students to communicate effectively with diverse stakeholders in the medical field [ 27 , 28 ].

Theoretical framework

The panel discussions in our study are grounded in the concept of authentic assessment (outlined by Villarroel, Bloxham [ 29 ]), which involves designing tasks that mirror real-life situations and problems. In the context of medical education, this approach is particularly relevant as it prepares students for the complex, multidisciplinary nature of healthcare communication. Realism can be achieved through two means: providing a realistic context that describes and delivers a frame for the problem to be solved and creating tasks that are similar to those faced in real and/or professional life [ 30 ]. In our study, the PDs provide a realistic context by simulating scenarios where medical students are required to discuss and present complex medical topics in a professional setting, mirroring the types of interactions they will encounter in their future careers.

The task of participating in PDs also involves cognitive challenge, as students are required to think critically about complex medical topics, analyze information, and communicate their findings effectively. This type of task aims to generate processes of problem-solving, application of knowledge, and decision-making that correspond to the development of cognitive and metacognitive skills [ 23 ]. For medical students, these skills are crucial in developing clinical reasoning and effective patient communication. The PDs encourage students to go beyond the textual reproduction of fragmented and low-order content and move towards understanding, establishing relationships between new ideas and previous knowledge, linking theoretical concepts with everyday experience, deriving conclusions from the analysis of data, and examining both the logic of the arguments present in the theory and its practical scope [ 24 , 25 , 27 ].

Furthermore, the evaluative judgment aspect of our study is critical in helping students develop criteria and standards about what a good performance means in medical communication. This involves students judging their own performance and regulating their own learning [ 31 ]. In the context of panel discussions, students reflect on their own work, compare it with desired standards, and seek feedback from peers and instructors. By doing so, students can develop a sense of what constitutes good performance in medical communication and what areas need improvement [ 32 ]. Boud, Lawson and Thompson [ 33 ] argue that students need to build a precise judgment about the quality of their work and calibrate these judgments in the light of evidence. This skill is particularly important for future medical professionals who will need to continually assess and improve their communication skills throughout their careers.

The theoretical framework presented above highlights the importance of authentic learning experiences in medical education. By drawing on the benefits of group work and panel discussions, university instructor-researchers aimed to provide medical students with a unique opportunity to engage with complex cases and develop their communication and collaboration skills. As noted by Suryanarayana [ 34 ], authentic learning experiences can lead to deeper learning and improved retention. Considering the advantages of group work in promoting collaborative problem-solving and language development, the instructor-researchers designed a panel discussion task that simulates real-world scenarios, where students can work together to analyze complex cases, share knowledge, and present their findings to a simulated audience.

While previous studies have highlighted the benefits of interactive learning experiences and critical thinking skills in medical education, a research gap remains in understanding how medical students perceive the relevance of PDs in ESP courses. This study aims to address this gap by investigating medical students’ perceptions of PD tasks in ESP courses and how these perceptions relate to their language proficiency, critical thinking skills, and ability to communicate effectively with diverse stakeholders in the medical field. This understanding can inform best practices in medical education, contributing to the development of more effective communication skills for future healthcare professionals worldwide [ 23 ]. The research questions guiding this study are:

What are the perceived advantages of PDs from the perspectives of panelists and the audience?

What are the perceived disadvantages of PDs from the perspectives of panelists and the audience?

How can PDs be improved for panelists and the audience based on the insights of ESP instructors?

Methodology

Aim and design.

For this study, a two-phase qualitative design was employed to gain an understanding of the advantages and disadvantages of PDs from the perspectives of both student panelists and the audience (Phase 1) and to acquire an in-depth understanding of the suggested strategies provided by experts to enhance PPs for future students (Phase 2).

Participants and context of the study

This study was conducted in two phases (Fig.  1 ) at Shiraz University of Medical Sciences (SUMS), Shiraz, Iran.

Participants of the study in two phases

In the first phase, the student participants were 46 non-native speakers of English and international students who studied medicine at SUMS. Their demographic characteristics can be seen in Table  1 .

These students were purposefully selected because they were the only SUMS international students who had taken the ESP (English for Specific Purposes) course. The number of international students attending SUMS is indeed limited. Each year, a different batch of international students joins the university. They progress through a sequence of English courses, starting with General English 1 and 2, followed by the ESP course, and concluding with academic writing. At the time of data collection, the students included in the study were the only international students enrolled in the ESP course. This mandatory 3-unit course is designed to enhance their language and communication skills specifically tailored to their profession. As a part of the Medicine major curriculum, this course aims to improve their English language proficiency in areas relevant to medicine, such as understanding medical terminology, comprehending original medicine texts, discussing clinical cases, and communicating with patients, colleagues, and other healthcare professionals.

Throughout the course, students engage in various interactive activities, such as group discussions, role-playing exercises, and case studies, to develop their practical communication skills. In this course, medical students receive four marks out of 20 for their oral presentations, while the remaining marks are allocated to their written midterm and final exams. From the beginning of the course, they are briefed about PDs, and they are shown two YouTube-downloaded videos about PDs at medical conferences, a popular format for discussing and sharing knowledge, research findings, and expert opinions on various medical topics.

For the second phase of the study, a specific group of participants was purposefully selected. This group consisted of three faculty members from SUMS English department who had extensive experience attending numerous conferences at national and international levels, particularly in the medical field, as well as working as translators and interpreters in medical congresses. Over the course of ten years, they also gained considerable experience in PDs. They were invited to discuss strategies helpful for medical students with PDs.

Panel discussion activity design and implementation

When preparing for a PD session, medical students received comprehensive guidance on understanding the roles and responsibilities of each panel member. This guidance was aimed at ensuring that each participant was well-prepared and understood their specific role in the discussion.

Moderators should play a crucial role in steering the conversation. They are responsible for ensuring that all panelists have an opportunity to contribute and that the audience is engaged effectively. Specific tasks include preparing opening remarks, introducing panelists, and crafting transition questions to facilitate smooth topic transitions. The moderators should also manage the time to ensure balanced participation and encourage active audience involvement.

Panelists are expected to be subject matter experts who bring valuable insights and opinions to the discussion. They are advised to conduct thorough research on the topic and prepare concise talking points. Panelists are encouraged to draw from their medical knowledge and relevant experiences, share evidence-based information, and engage with other panelists’ points through active listening and thoughtful responses.

The audience plays an active role in the PDs. They are encouraged to participate by asking questions, sharing relevant experiences, and contributing to the dialogue. To facilitate this, students are advised to take notes during the discussion and think of questions or comments they can contribute during the Q&A segment.

For this special course, medical students were advised to choose topics either from their ESP textbook or consider current medical trends, emerging research, and pressing issues in their field. Examples included breast cancer, COVID-19, and controversies in gene therapy. The selection process involved brainstorming sessions and consultation with the course instructor to ensure relevance and appropriateness.

To accommodate the PD sessions within the course structure, students were allowed to start their PD sessions voluntarily from the second week. However, to maintain a balance between peer-led discussions and regular course content, only one PD was held weekly. This approach enabled the ESP lecturer to deliver comprehensive content while also allowing students to engage in these interactive sessions.

A basic time structure was suggested for each PD (Fig.  2 ):

Time allocation for panel discussion stages in minutes

To ensure the smooth running of the course and maintain momentum, students were informed that they could cancel their PD session only once. In such cases, they were required to notify the lecturer and other students via the class Telegram channel to facilitate rescheduling and minimize disruptions. This provision was essential in promoting a sense of community among students and maintaining the course’s continuity.

Research tools and data collection

The study utilized various tools to gather and analyze data from participants and experts, ensuring a comprehensive understanding of the research topic.

Reflection papers

In Phase 1 of the study, 46 medical students detailed their perceptions of the advantages and disadvantages of panel discussions from dual perspectives: as panelists (presenters) and as audience members (peers).

Participants were given clear instructions and a 45-minute time frame to complete the reflection task. With approximately 80% of the international language students being native English speakers and the rest fluent in English, the researchers deemed this time allocation reasonable. The questions and instructions were straightforward, facilitating quick comprehension. It was estimated that native English speakers would need about 30 min to complete the task, while non-native speakers might require an extra 15 min for clarity and expression. This time frame aimed to allow students to respond thoughtfully without feeling rushed. Additionally, students could request more time if needed.

Focus group discussion

In phase 2 of the study, a focus group discussion was conducted with three expert participants. The purpose of the focus group was to gather insights from expert participants, specifically ESP (English for Specific Purposes) instructors, on how presentation dynamics can be improved for both panelists and the audience.

According to Colton and Covert [ 35 ], focus groups are useful for obtaining detailed input from experts. The appropriate size of a focus group is determined by the study’s scope and available resources [ 36 ]. Morgan [ 37 ] suggests that small focus groups are suitable for complex topics where specialist participants might feel frustrated if not allowed to express themselves fully.

The choice of a focus group over individual interviews was based on several factors. First, the exploratory nature of the study made focus groups ideal for interactive discussions, generating new ideas and in-depth insights [ 36 ]. Second, while focus groups usually involve larger groups, they can effectively accommodate a limited number of experts with extensive knowledge [ 37 ]. Third, the focus group format fostered a more open environment for idea exchange, allowing participants to engage dynamically [ 36 ]. Lastly, conducting a focus group was more time- and resource-efficient than scheduling three separate interviews [ 36 ].

Data analysis

The first phase of the study involved a thorough examination of the data related to the research inquiries using thematic analysis. This method was chosen for its effectiveness in uncovering latent patterns from a bottom-up perspective, facilitating a comprehensive understanding of complex educational phenomena [ 38 ]. The researchers first familiarized themselves with the data by repeatedly reviewing the reflection papers written by the medical students. Next, an initial round of coding was independently conducted to identify significant data segments and generate preliminary codes that reflected the students’ perceptions of the advantages and disadvantages of presentation dynamics PDs from both the presenter and audience viewpoints [ 38 ].

The analysis of the reflection papers began with the two researchers coding a subset of five papers independently, adhering to a structured qualitative coding protocol [ 39 ]. They convened afterward to compare their initial codes and address any discrepancies. Through discussion, they reached an agreement on the codes, which were then analyzed, organized into categories and themes, and the frequency of each code was recorded [ 38 ].

After coding the initial five papers, the researchers continued to code the remaining 41 reflection paper transcripts in batches of ten, meeting after each batch to review their coding, resolve any inconsistencies, and refine the coding framework as needed. This iterative process, characterized by independent coding, joint reviews, and consensus-building, helped the researchers establish a robust and reliable coding approach consistently applied to the complete dataset [ 40 ]. Once all 46 reflection paper transcripts were coded, the researchers conducted a final review and discussion to ensure accurate analysis. They extracted relevant excerpts corresponding to the identified themes and sub-themes from the transcripts to provide detailed explanations and support for their findings [ 38 ]. This multi-step approach of separate initial coding, collaborative review, and frequency analysis enhanced the credibility and transparency of the qualitative data analysis.

To ensure the trustworthiness of the data collected in this study, the researchers adhered to the Guba and Lincoln standards of scientific accuracy in qualitative research, which encompass credibility, confirmability, dependability, and transferability [ 41 ] (Table  2 ).

The analysis of the focus group data obtained from experts followed the same rigorous procedure applied to the student participants’ data. Thematic analysis was employed to examine the experts’ perspectives, maintaining consistency in the analytical approach across both phases of the study. The researchers familiarized themselves with the focus group transcript, conducted independent preliminary coding, and then collaboratively refined the codes. These codes were subsequently organized into categories and themes, with the frequency of each code recorded. The researchers engaged in thorough discussions to ensure agreement on the final themes and sub-themes. Relevant excerpts from the focus group transcript were extracted to provide rich, detailed explanations of each theme, thereby ensuring a comprehensive and accurate analysis of the experts’ insights.

1. What are the advantages of PDs from the perspective of panelists and the audience?

The analysis of the advantages of PDs from the perspectives of both panelists and audience members revealed several key themes and categories. Tables  2 and 3 present the frequency and percentage of responses for each code within these categories.

From the panelists’ perspective (Table  3 ), the overarching theme was “Personal and Professional Development.” The most frequently reported advantage was knowledge sharing (93.5%), followed closely by increased confidence (91.3%) and the importance of interaction in presentations (91.3%).

Notably, all categories within this theme had at least one code mentioned by over 80% of participants, indicating a broad range of perceived benefits. The category of “Effective teamwork and communication” was particularly prominent, with collaboration (89.1%) and knowledge sharing (93.5%) being among the most frequently cited advantages. This suggests that PDs are perceived as valuable tools for fostering interpersonal skills and collective learning. In the “Language mastery” category, increased confidence (91.3%) and better retention of key concepts (87.0%) were highlighted, indicating that PDs are seen as effective for both language and content learning.

The audience perspective (Table  4 ), encapsulated under the theme “Enriching Learning Experience,” showed similarly high frequencies across all categories.

The most frequently mentioned advantage was exposure to diverse speakers (93.5%), closely followed by the range of topics covered (91.3%) and increased audience interest (91.3%). The “Broadening perspectives” category was particularly rich, with all codes mentioned by over 70% of participants. This suggests that audience members perceive PDs as valuable opportunities for expanding their knowledge and viewpoints. In the “Language practice” category, the opportunity to practice language skills (89.1%) was the most frequently cited advantage, indicating that even as audience members, students perceive significant language learning benefits.

Comparing the two perspectives reveals several interesting patterns:

High overall engagement: Both panelists and audience members reported high frequencies across all categories, suggesting that PDs are perceived as beneficial regardless of the role played.

Language benefits: While panelists emphasized increased confidence (91.3%) and better retention of concepts (87.0%), audience members highlighted opportunities for language practice (89.1%). This indicates that PDs offer complementary language learning benefits for both roles.

Interactive learning: The importance of interaction was highly rated by panelists (91.3%), while increased audience interest was similarly valued by the audience (91.3%). This suggests that PDs are perceived as an engaging, interactive learning method from both perspectives.

Professional development: Panelists uniquely emphasized professional growth aspects such as experiential learning (84.8%) and real-world application (80.4%). These were not directly mirrored in the audience perspective, suggesting that active participation in PDs may offer additional professional development benefits.

Broadening horizons: Both groups highly valued the diversity aspect of PDs. Panelists appreciated diversity and open-mindedness (80.4%), while audience members valued diverse speakers (93.5%) and a range of topics (91.3%).

2. What are the disadvantages of PDs from the perspective of panelists and the audience?

The analysis of the disadvantages of panel discussions (PDs) from the perspectives of both panelists and audience members revealed several key themes and categories. Tables  4 and 5 present the frequency and percentage of responses for each code within these categories.

From the panelists’ perspective (Table  5 ), the theme “Drawbacks of PDs” was divided into two main categories: “Academic Workload Challenges” and “Coordination Challenges.” The most frequently reported disadvantage was long preparation (87.0%), followed by significant practice needed (82.6%) and the time-consuming nature of PDs (80.4%). These findings suggest that the primary concern for panelists is the additional workload that PDs impose on their already demanding academic schedules. The “Coordination Challenges” category, while less prominent than workload issues, still presented significant concerns. Diverse panel skills (78.3%) and finding suitable panelists (73.9%) were the most frequently cited issues in this category, indicating that team dynamics and composition are notable challenges for panelists.

The audience perspective (Table  6 ), encapsulated under the theme “Drawbacks of PDs,” was divided into two main categories: “Time-related Issues” and “Interaction and Engagement Issues.” In the “Time-related Issues” category, the most frequently mentioned disadvantage was the inefficient use of time (65.2%), followed by the perception of PDs as too long and boring (60.9%). Notably, 56.5% of respondents found PDs stressful due to overwhelming workload from other studies, and 52.2% considered them not very useful during exam time. The “Interaction and Engagement Issues” category revealed more diverse concerns. The most frequently mentioned disadvantage was the repetitive format (82.6%), followed by limited engagement with the audience (78.3%) and the perception of PDs as boring (73.9%). The audience also noted issues related to the panelists’ preparation and coordination, such as “Not practiced and natural” (67.4%) and “Coordination and Interaction Issues” (71.7%), suggesting that the challenges faced by panelists directly impact the audience’s experience.

Workload concerns: Both panelists and audience members highlighted time-related issues. For panelists, this manifested as long preparation times (87.0%) and difficulty balancing with other studies (76.1%). For the audience, it appeared as perceptions of inefficient use of time (65.2%) and stress due to overwhelming workload from other studies (56.5%).

Engagement issues: While panelists focused on preparation and coordination challenges, the audience emphasized the quality of the discussion and engagement. This suggests a potential mismatch between the efforts of panelists and the expectations of the audience.

Boredom and repetition: The audience frequently mentioned boredom (73.9%) and repetitive format (82.6%) as issues, which weren’t directly mirrored in the panelists’ responses. This indicates that while panelists may be focused on content preparation, the audience is more concerned with the delivery and variety of the presentation format.

Coordination challenges: Both groups noted coordination issues, but from different perspectives. Panelists struggled with team dynamics and finding suitable co-presenters, while the audience observed these challenges manifesting as unnatural or unpracticed presentations.

Academic pressure: Both groups acknowledged the strain PDs put on their academic lives, with panelists viewing it as a burden (65.2%) and the audience finding it less useful during exam times (52.2%).

3. How can PDs be improved for panelists and the audience from the experts’ point of view?

The presentation of data for this research question differs from the previous two due to the unique nature of the information gathered. Unlike the quantifiable student responses in earlier questions, this data stems from expert opinions and a reflection discussion session, focusing on qualitative recommendations for improvement rather than frequency of responses (Braun & Clarke, 2006). The complexity and interconnectedness of expert suggestions, coupled with the integration of supporting literature, necessitate a more narrative approach (Creswell & Poth, 2018). This format allows for a richer exploration of the context behind each recommendation and its potential implications (Patton, 2015). Furthermore, the exploratory nature of this question, aimed at generating ideas for improvement rather than measuring prevalence of opinions, is better served by a detailed, descriptive presentation (Merriam & Tisdell, 2016). This approach enables a more nuanced understanding of how PDs can be enhanced, aligning closely with the “how” nature of the research question and providing valuable insights for potential implementation (Yin, 2018).

The experts provided several suggestions to address the challenges faced by students in panel discussions (PDs) and improve the experience for both panelists and the audience. Their recommendations focused on six key areas: time management and workload, preparation and skill development, engagement and interactivity, technological integration, collaboration and communication, and institutional support.

To address the issue of time management and heavy workload, one expert suggested teaching students to “ break down the task to tackle the time-consuming nature of panel discussions and balance it with other studies .” This approach aims to help students manage the extensive preparation time required for PDs without compromising their other academic responsibilities. Another expert emphasized “ enhancing medical students’ abilities to prioritize tasks , allocate resources efficiently , and optimize their workflow to achieve their goals effectively .” These skills were seen as crucial not only for PD preparation but also for overall academic success and future professional practice.

Recognizing the challenges of long preparation times and the perception of PDs being burdensome, an expert proposed “ the implementation of interactive training sessions for panelists .” These sessions were suggested to enhance coordination skills and improve the ability of group presenters to engage with the audience effectively. The expert emphasized that such training could help students view PDs as valuable learning experiences rather than additional burdens, potentially increasing their motivation and engagement in the process.

To combat issues of limited engagement and perceived boredom, experts recommended increasing engagement opportunities for the audience through interactive elements like audience participation and group discussions. They suggested that this could transform PDs from passive listening experiences to active learning opportunities. One expert suggested “ optimizing time management and restructuring the format of panel discussions ” to address inefficiency during sessions. This restructuring could involve shorter presentation segments interspersed with interactive elements to maintain audience attention and engagement.

An innovative solution proposed by one expert was “ using ChatGPT to prepare for PDs by streamlining scenario presentation preparation and role allocation. ” The experts collectively discussed the potential of AI to assist medical students in reducing their workload and saving time in preparing scenario presentations and allocating roles in panel discussions. They noted that AI could help generate initial content drafts, suggest role distributions based on individual strengths, and even provide practice questions for panelists, significantly reducing preparation time while maintaining quality.

Two experts emphasized the importance of enhancing collaboration and communication among panelists to address issues related to diverse panel skills and coordination challenges. They suggested establishing clear communication channels and guidelines to improve coordination and ensure a cohesive presentation. This could involve creating structured team roles, setting clear expectations for each panelist, and implementing regular check-ins during the preparation process to ensure all team members are aligned and progressing.

All experts were in agreement that improving PDs would not be possible “ if nothing is done by the university administration to reduce the ESP class size for international students .” They believed that large class sizes in ESP or EFL classes could negatively influence group oral presentations, hindering language development and leading to uneven participation. The experts suggested that smaller class sizes would allow for more individualized attention, increased speaking opportunities for each student, and more effective feedback mechanisms, all of which are crucial for developing strong presentation skills in a second language.

Research question 1: what are the advantages of PDs from the perspective of panelists and the audience?

The results of this study reveal significant advantages of PDs for both panelists and audience members in the context of medical education. These findings align with and expand upon previous research in the field of educational presentations and language learning.

Personal and professional development for panelists

The high frequency of reported benefits in the “Personal and Professional Development” theme for panelists aligns with several previous studies. The emphasis on language mastery, particularly increased confidence (91.3%) and better retention of key concepts (87.0%), supports the findings of Hartono, Mujiyanto [ 42 ], Gedamu and Gezahegn [ 15 ], Li [ 43 ], who all highlighted the importance of language practice in English oral presentations. However, our results show a more comprehensive range of benefits, including professional growth aspects like experiential learning (84.8%) and real-world application (80.4%), which were not as prominently featured in these earlier studies.

Interestingly, our findings partially contrast with Chou [ 44 ] study, which found that while group oral presentations had the greatest influence on improving students’ speaking ability, individual presentations led to more frequent use of metacognitive, retrieval, and rehearsal strategies. Our results suggest that PDs, despite being group activities, still provide significant benefits in these areas, possibly due to the collaborative nature of preparation and the individual responsibility each panelist bears. The high frequency of knowledge sharing (93.5%) and collaboration (89.1%) in our study supports Harris, Jones and Huffman [ 45 ] emphasis on the importance of group dynamics and varied perspectives in educational settings. However, our study provides more quantitative evidence for these benefits in the specific context of PDs.

Enriching learning experience for the audience

The audience perspective in our study reveals a rich learning experience, with high frequencies across all categories. This aligns with Agustina [ 46 ] findings in business English classes, where presentations led to improvements in all four language skills. However, our study extends these findings by demonstrating that even passive participation as an audience member can lead to significant perceived benefits in language practice (89.1%) and broadening perspectives (93.5% for diverse speakers). The high value placed on diverse speakers (93.5%) and range of topics (91.3%) by the audience supports the notion of PDs as a tool for expanding knowledge and viewpoints. This aligns with the concept of situated learning experiences leading to deeper understanding in EFL classes, as suggested by Li [ 43 ] and others [ 18 , 31 ]. However, our study provides more specific evidence for how this occurs in the context of PDs.

Interactive learning and engagement

Both panelists and audience members in our study highly valued the interactive aspects of PDs, with the importance of interaction rated at 91.3% by panelists and increased audience interest at 91.3% by the audience. This strong emphasis on interactivity aligns with Azizi and Farid Khafaga [ 19 ] study on the benefits of dynamic assessment and dialogic learning contexts. However, our study provides more detailed insights into how this interactivity is perceived and valued by both presenters and audience members in PDs.

Professional growth and real-world application

The emphasis on professional growth through PDs, particularly for panelists, supports Li’s [ 43 ] assertion about the power of oral presentations as situated learning experiences. Our findings provide more specific evidence for how PDs contribute to professional development, with high frequencies reported for experiential learning (84.8%) and real-world application (80.4%). This suggests that PDs may be particularly effective in bridging the gap between academic learning and professional practice in medical education.

Research question 2: what are the disadvantages of pds from the perspective of panelists and the audience?

Academic workload challenges for panelists.

The high frequency of reported challenges in the “Academic Workload Challenges” category for panelists aligns with several previous studies in medical education [ 47 , 48 , 49 ]. The emphasis on long preparation (87.0%), significant practice needed (82.6%), and the time-consuming nature of PDs (80.4%) supports the findings of Johnson et al. [ 24 ], who noted that while learners appreciate debate-style journal clubs in health professional education, they require additional time commitment. This is further corroborated by Nowak, Speed and Vuk [ 50 ], who found that intensive learning activities in medical education, while beneficial, can be time-consuming for students.

Perceived value of pds relative to time investment

While a significant portion of the audience (65.2%) perceived PDs as an inefficient use of time, the high frequency of engagement-related concerns (82.6% for repetitive format, 78.3% for limited engagement) suggests that the perceived lack of value may be more closely tied to the quality of the experience rather than just the time investment. This aligns with Dyhrberg O’Neill [ 27 ] findings on debate-based oral exams, where students perceived value despite the time-intensive nature of the activity. However, our results indicate a more pronounced concern about the return on time investment in PDs. This discrepancy might be addressed through innovative approaches to PD design and implementation, such as those proposed by Almazyad et al. [ 22 ], who suggested using AI tools to enhance expert panel discussions and potentially improve efficiency.

Coordination challenges for panelists

The challenges related to coordination in medical education, such as diverse panel skills (78.3%) and finding suitable panelists (73.9%), align with previous research on teamwork in higher education [ 21 ]. Our findings support the concept of the free-rider effect discussed by Hall and Buzwell [ 21 ], who explored reasons for non-contribution in group projects beyond social loafing. This is further elaborated by Mehmood, Memon and Ali [ 51 ], who proposed that individuals may not contribute their fair share due to various factors including poor communication skills or language barriers, which is particularly relevant in medical education where clear communication is crucial [ 52 ]. Comparing our results to other collaborative learning contexts in medical education, Rodríguez-Sedano, Conde and Fernández-Llamas [ 53 ] measured teamwork competence development in a multidisciplinary project-based learning environment. They found that while teamwork skills improved over time, initial coordination challenges were significant. This aligns with our findings on the difficulties of coordinating diverse panel skills and opinions in medical education settings.

Our results also resonate with Chou’s [ 44 ] study comparing group and individual oral presentations, which found that group presenters often had a limited understanding of the overall content. This is supported by Wilson, Ho and Brookes [ 54 ], who examined student perceptions of teamwork in undergraduate science degrees, highlighting the challenges and benefits of collaborative work, which are equally applicable in medical education [ 52 ].

Quality of discussions and perception for the audience

The audience perspective in our study reveals significant concerns about the quality and engagement of PDs in medical education. The high frequency of issues such as repetitive format (82.6%) and limited engagement with the audience (78.3%) aligns with Parmar and Bickmore [ 55 ] findings on the importance of addressing individual audience members and gathering feedback. This is further supported by Nurakhir et al. [ 25 ], who explored students’ views on classroom debates as a strategy to enhance critical thinking and oral communication skills in nursing education, which shares similarities with medical education. Comparing our results to other interactive learning methods in medical education, Jones et al. [ 26 ] reviewed the use of journal clubs and book clubs in pharmacy education. They found that while these methods enhanced engagement, they also faced challenges in maintaining student interest over time, similar to the boredom issues reported in our study of PDs in medical education. The perception of PDs as boring (73.9%) and not very useful during exam time (52.2%) supports previous research on the stress and pressure experienced by medical students [ 48 , 49 ]. Grieve et al. [ 20 ] specifically examined student fears of oral presentations and public speaking in higher education, which provides context for the anxiety and disengagement observed in our study of medical education. Interestingly, Bhuvaneshwari et al. [ 23 ] found positive impacts of panel discussions in educating medical students on specific modules. This contrasts with our findings and suggests that the effectiveness of PDs in medical education may vary depending on the specific context and implementation.

Comparative analysis and future directions

Our study provides a unique comparative analysis of the challenges faced by both panelists and audience members in medical education. The alignment of concerns around workload and time management between the two groups suggests that these are overarching issues in the implementation of PDs in medical curricula. This is consistent with the findings of Pasandín et al. [ 56 ], who examined cooperative oral presentations in higher education and their impact on both technical and soft skills, which are crucial in medical education [ 52 ]. The mismatch between panelist efforts and audience expectations revealed in our study is a novel finding that warrants further investigation in medical education. This disparity could be related to the self-efficacy beliefs of presenters, as explored by Gedamu and Gezahegn [ 15 ] in their study of TEFL trainees’ attitudes towards academic oral presentations, which may have parallels in medical education. Looking forward, innovative approaches could address some of the challenges identified in medical education. Almazyad et al. [ 22 ] proposed using AI tools like ChatGPT to enhance expert panel discussions in pediatric palliative care, which could potentially address some of the preparation and engagement issues identified in our study of medical education. Additionally, Ragupathi and Lee [ 57 ] discussed the role of rubrics in higher education, which could provide clearer expectations and feedback for both panelists and audience members in PDs within medical education.

Research question 3: how can PDs be improved for panelists and the audience from the experts’ point of view?

The expert suggestions for improving PDs address several key challenges identified in previous research on academic presentations and student workload management. These recommendations align with current trends in educational technology and pedagogical approaches, while also considering the unique needs of medical students.

The emphasis on time management and workload reduction strategies echoes findings from previous studies on medical student stress and academic performance. Nowak, Speed and Vuk [ 50 ] found that medical students often struggle with the fast-paced nature of their courses, which can lead to reduced motivation and superficial learning approaches. The experts’ suggestions for task breakdown and prioritization align with Rabbi and Islam [ 58 ] recommendations for reducing workload stress through effective assignment prioritization. Additionally, Popa et al. [ 59 ] highlight the importance of acceptance and planning in stress management for medical students, supporting the experts’ focus on these areas.

The proposed implementation of interactive training sessions for panelists addresses the need for enhanced presentation skills in professional contexts, a concern highlighted by several researchers [ 17 , 60 ]. This aligns with Grieve et al. [ 20 ] findings on student fears of oral presentations and public speaking in higher education, emphasizing the need for targeted training. The focus on interactive elements and audience engagement also reflects current trends in active learning pedagogies, as demonstrated by Pasandín et al. [ 56 ] in their study on cooperative oral presentations in engineering education.

The innovative suggestion to use AI tools like ChatGPT for PD preparation represents a novel approach to leveraging technology in education. This aligns with recent research on the potential of AI in scientific research, such as the study by Almazyad et al. [ 22 ], which highlighted the benefits of AI in supporting various educational tasks. However, it is important to consider potential ethical implications and ensure that AI use complements rather than replaces critical thinking and creativity.

The experts’ emphasis on enhancing collaboration and communication among panelists addresses issues identified in previous research on teamwork in higher education. Rodríguez-Sedano, Conde and Fernández-Llamas [ 53 ] noted the importance of measuring teamwork competence development in project-based learning environments. The suggested strategies for improving coordination align with best practices in collaborative learning, as demonstrated by Romero-Yesa et al. [ 61 ] in their qualitative assessment of challenge-based learning and teamwork in electronics programs.

The unanimous agreement on the need to reduce ESP class sizes for international students reflects ongoing concerns about the impact of large classes on language learning and student engagement. This aligns with research by Li [ 3 ] on issues in developing EFL learners’ oral English communication skills. Bosco et al. [ 62 ] further highlight the challenges of teaching and learning ESP in mixed classes, supporting the experts’ recommendation for smaller class sizes. Qiao, Xu and bin Ahmad [ 63 ] also emphasize the implementation challenges for ESP formative assessment in large classes, further justifying the need for reduced class sizes.

These expert recommendations provide a comprehensive approach to improving PDs, addressing not only the immediate challenges of preparation and delivery but also broader issues of student engagement, workload management, and institutional support. By implementing these suggestions, universities could potentially transform PDs from perceived burdens into valuable learning experiences that enhance both academic and professional skills. This aligns with Kho and Ting [ 64 ] systematic review on overcoming oral presentation anxiety among tertiary ESL/EFL students, which emphasizes the importance of addressing both challenges and strategies in improving presentation skills.

This study has shed light on the complex challenges associated with PDs in medical education, revealing a nuanced interplay between the experiences of panelists and audience members. The findings underscore the need for a holistic approach to implementing PDs that addresses both the academic workload concerns and the quality of engagement.

Our findings both support and extend previous research on the challenges of oral presentations and group work in medical education settings. The high frequencies of perceived challenges across multiple categories for both panelists and audience members suggest that while PDs may offer benefits, they also present significant obstacles that need to be addressed in medical education. These results highlight the need for careful consideration in the implementation of PDs in medical education, with particular attention to workload management, coordination strategies, and audience engagement techniques. Future research could focus on developing and testing interventions to mitigate these challenges while preserving the potential benefits of PDs in medical education.

Moving forward, medical educators should consider innovative approaches to mitigate these challenges. This may include:

Integrating time management and stress coping strategies into the PD preparation process [ 59 ].

Exploring the use of AI tools to streamline preparation and enhance engagement [ 22 ].

Developing clear rubrics and expectations for both panelists and audience members [ 57 ].

Incorporating interactive elements to maintain audience interest and participation [ 25 ].

Limitations and future research

One limitation of this study is that it focused on a specific population of medical students, which may limit the generalizability of the findings to other student populations. Additionally, the study relied on self-report data from panelists and audience members, which may introduce bias and affect the validity of the results. Future research could explore the effectiveness of PDs in different educational contexts and student populations to provide a more comprehensive understanding of the benefits and challenges of panel discussions.

Future research should focus on evaluating the effectiveness of these interventions and exploring how PDs can be tailored to the unique demands of medical education. By addressing the identified challenges, PDs have the potential to become a more valuable and engaging component of medical curricula, fostering both academic and professional development. Ultimately, the goal should be to transform PDs from perceived burdens into opportunities for meaningful learning and skill development, aligning with the evolving needs of medical education in the 21st century.

Future research could also examine the long-term impact of PDs on panelists’ language skills, teamwork, and communication abilities. Additionally, exploring the effectiveness of different training methods and tools, such as AI technology, in improving coordination skills and reducing workload stress for panelists could provide valuable insights for educators and administrators. Further research could also investigate the role of class size and audience engagement in enhancing the overall effectiveness of PDs in higher education settings. By addressing these gaps in the literature, future research can contribute to the ongoing development and improvement of PDs as a valuable learning tool for students in higher education.

However, it is important to note that implementing these changes may require significant institutional resources and a shift in pedagogical approaches. Future research could focus on piloting these recommendations and evaluating their effectiveness in improving student outcomes and experiences with PDs.

Data availability

We confirm that the data supporting the findings are available within this article. Raw data supporting this study’s findings are available from the corresponding author, upon request.

Abbreviations

Artificial Intelligence

English as a Foreign Language

English for Specific Purposes

Panel Discussion

Shiraz University of Medical Sciences

Harden RM, Laidlaw JM. Essential skills for a medical teacher: an introduction to teaching and learning in medicine. Elsevier Health Sciences; 2020.

Ibrahim Mohamed O, Al Jadaan DO. English for Specific purposes (Esp) Needs Analysis for Health Sciences students: a cross-sectional study at a University in the UAE. English for Specific purposes (Esp) Needs Analysis for Health Sciences Students: A Cross-Sectional Study at a University in the UAE.

Li Y, Heron M. English for general academic purposes or English for specific purposes? Language learning needs of medical students at a Chinese university. Theory Pract Lang Stud. 2021;11(6):621–31.

Article   Google Scholar  

Chan SMH, Mamat NH, Nadarajah VD. Mind your language: the importance of English language skills in an International Medical Programme (IMP). BMC Med Educ. 2022;22(1):405.

Cortez Faustino BS, Ticas de Córdova CK, de la Hernández DI. Teaching English for specific purposes: contents and methodologies that could be implemented in the English for Medical purposes (EMP) course for the doctor of Medicine Major at the University of El Salvador. Universidad de El Salvador; 2022.

BENYAMINA E-Z BOUKAHLAH. Enhancing Specialty Language learning through content-based instruction: students of Paramedical Institute of Tiaret as a case study. Université IBN KHALDOUN-Tiaret; 2023.

Prikazchikov M. Medical English course for russian-speaking dentists: a needs analysis study. Iowa State University; 2024.

Kim C, Lee SY, Park S-H. Is Korea Ready to be a key player in the Medical Tourism Industry? An English Education Perspective. Iran J Public Health. 2020;49(2):267–73.

Google Scholar  

Syakur A, Zainuddin H, Hasan MA. Needs analysis English for specific purposes (esp) for vocational pharmacy students. Budapest International Research and Critics in Linguistics and Education (BirLE). Journal. 2020;3(2):724–33.

Chan S, Taylor L. Comparing writing proficiency assessments used in professional medical registration: a methodology to inform policy and practice. Assess Writ. 2020;46:100493.

Hyland K, Jiang FK. Delivering relevance: the emergence of ESP as a discipline. Engl Specif Purp. 2021;64:13–25.

Maftuna B. The role of English in ESP. Am J Adv Sci Res. 2024;1(2):1–5.

LEON LI, HUMANIZING THE FOREIGN LANGUAGE. COURSE: NEW TEACHING METHODS FOR MEDICAL STUDENTS. Language, Culture and Change. 2022:243.

Dahm MR, Yates L. Rapport, empathy and professional identity: Some challenges for international medical graduates speaking English as a second or foreign language. Multilingual Healthcare: A Global View on Communicative Challenges. 2020:209 – 34.

Gedamu AD, Gezahegn TH. TEFL trainees’ attitude to and self-efficacy beliefs of academic oral presentation. Cogent Educ. 2023;10(1):2163087.

Saliu B, Hajrullai H. Best practices in the English for specific purpose classes at the language center. Procedia-Social Behav Sci. 2016;232:745–9.

Clokie TL, Fourie E. Graduate employability and communication competence: are undergraduates taught relevant skills? Bus Prof Communication Q. 2016;79(4):442–63.

Hartono H, Mujiyanto J, Fitriati SW, Sakhiyya Z, Lotfie MM, Maharani MM. English Presentation Self-Efficacy Development of Indonesian ESP students: the effects of Individual versus Group Presentation tasks. Int J Lang Educ. 2023;7(3):361–76.

Azizi Z, Farid Khafaga A. Scaffolding via Group-dynamic Assessment to positively affect motivation, learning anxiety, and willingness to Communicate: a Case Study of High School Students. J Psycholinguist Res. 2023;52(3):831–51.

Grieve R, Woodley J, Hunt SE, McKay A. Student fears of oral presentations and public speaking in higher education: a qualitative survey. J Furth High Educ. 2021;45(9):1281–93.

Hall D, Buzwell S. The problem of free-riding in group projects: looking beyond social loafing as reason for non-contribution. Act Learn High Educ. 2013;14(1):37–49.

Almazyad M, Aljofan F, Abouammoh NA, Muaygil R, Malki KH, Aljamaan F, et al. Enhancing Expert Panel discussions in Pediatric Palliative Care: innovative scenario development and summarization with ChatGPT-4. Cureus. 2023;15(4):e38249.

Bhuvaneshwari S, Rashmi R, Deepika K, Anirudh VM, Vijayamathy A, Rekha S, Kathiravan R. Impact of panel discussion in educating AETCOM First Module among Undergraduate Medical Students. Latin Am J Pharmacy: Life Sci J. 2023;42(6):407–12.

Johnson BR, Logan LD, Darley A, Stone RH, Smith SE, Osae SP, et al. A scoping review for Debate-Style Journal Clubs in Health Professional Education. Am J Pharm Educ. 2023;87(6):100064.

Nurakhir A, Palupi FN, Langeveld C, Nurmalia D. Students’ views of classroom debates as a strategy to enhance critical thinking and oral communication skills. 2020.

Jones EP, Nelson NR, Thorpe CT, Rodgers PT, Carlson RB. Use of journal clubs and book clubs in pharmacy education: a scoping review. Currents Pharm Teach Learn. 2022;14(1):110–9.

Dyhrberg O’Neill L. Assessment of student debates in support of active learning? Students’ perceptions of a debate-based oral final exam. Act Learn High Educ. 2024.

Dyment JE, O’Connell TS. Assessing the quality of reflection in student journals: a review of the research. Teach High Educ. 2011;16(1):81–97.

Villarroel V, Bloxham S, Bruna D, Bruna C, Herrera-Seda C. Authentic assessment: creating a blueprint for course design. Assess Evaluation High Educ. 2018;43(5):840–54.

Schultz M, Young K, Gunning K, Harvey T. Defining and measuring authentic assessment: a case study in the context of tertiary science. Assess Evaluation High Educ. 2022;47(1):77–94.

Sundrarajun C, Kiely R. The oral presentation as a context for learning and assessment. Innov Lang Learn Teach. 2010;4(2):101–17.

Wyatt-Smith C, Adie L. The development of students’ evaluative expertise: enabling conditions for integrating criteria into pedagogic practice. J Curriculum Stud. 2021;53(4):399–419.

Boud D, Lawson R, Thompson DG. The calibration of student judgement through self-assessment: disruptive effects of assessment patterns. High Educ Res Dev. 2015;34(1):45–59.

A. S. Enhancing Meaningful Learning experiences through Comprehension and Retention by students. Twentyfirst Century Publications Patiala. 2023;49.

Colton D, Covert RW. Designing and constructing instruments for social research and evaluation. Wiley; 2007.

Krueger RA, Casey MA. Focus group interviewing. Handbook of practical program evaluation. 2015:506 – 34.

Morgan DL. Handbook of interview research: Context and method. Oaks, CA, USA: Sage Publications Thousand; 2002.

Braun V, Clarke V. Conceptual and design thinking for thematic analysis. Qualitative Psychol. 2022;9(1):3.

Elliott V. Thinking about the coding process in qualitative data analysis. Qualitative Rep. 2018;23(11).

Syed M, Nelson SC. Guidelines for establishing reliability when coding narrative data. Emerg Adulthood. 2015;3(6):375–87.

Lincoln Y. Naturalistic inquiry: Sage; 1985.

Hartono H, Mujiyanto J, Fitriati SW, Sakhiyya Z, Lotfie MM, Maharani MM. English presentation self-efficacy development of Indonesian ESP students: the effects of Individual versus Group Presentation tasks. Int J Lang Educ. 2023;7(3).

Li X. Teaching English oral presentations as a situated task in an EFL classroom: a quasi-experimental study of the effect of video-assisted self-reflection. Revista Signos. 2018;51(98):359–81.

Chou M-h. The influence of learner strategies on oral presentations: a comparison between group and individual performance. Engl Specif Purp. 2011;30(4):272–85.

Harris A, Jones M, Huffman J. Teachers leading educational reform. The power of. 2017.

Agustina L. Stimulating students to speak up through presentation in business English class. J Appl Stud Lang. 2019;3(1):21–8.

Babal JC, Abraham O, Webber S, Watterson T, Moua P, Chen J. Student pharmacist perspectives on factors that influence wellbeing during pharmacy school. Am J Pharm Educ. 2020;84(9):ajpe7831.

Moir F, Yielder J, Sanson J, Chen Y. Depression in medical students: current insights. Adv Med Educ Pract. 2018;323:33.

Pavlinac Dodig I, Lusic Kalcina L, Demirovic S, Pecotic R, Valic M, Dogas Z. Sleep and lifestyle habits of medical and non-medical students during the COVID-19 lockdown. Behav Sci. 2023;13(5):407.

Nowak G, Speed O, Vuk J. Microlearning activities improve student comprehension of difficult concepts and performance in a biochemistry course. Currents Pharm Teach Learn. 2023;15(1):69–78.

Mehmood K, Memon S, Ali F. Language barriers to Effective Communication in speaking English: a phenomenological study of Pakistan International cricketers. Pakistan Lang Humanit Rev. 2024;8(1):107–14.

Buelow JR, Downs D, Jorgensen K, Karges JR, Nelson D. Building interdisciplinary teamwork among allied health students through live clinical case simulations. J Allied Health. 2008;37(2):e109–23.

Rodríguez-Sedano FJ, Conde M, Fernández-Llamas C, editors. Measuring teamwork competence development in a multidisciplinary project based learning environment. Learning and Collaboration Technologies Design, Development and Technological Innovation: 5th International Conference, LCT 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15–20, 2018, Proceedings, Part I 5; 2018: Springer.

Wilson L, Ho S, Brookes RH. Student perceptions of teamwork within assessment tasks in undergraduate science degrees. Assess Evaluation High Educ. 2018;43(5):786–99.

Parmar D, Bickmore T. Making it personal: addressing individual audience members in oral presentations using augmented reality. Proc ACM Interact Mob Wearable Ubiquitous Technol. 2020;4(2):1–22.

Pasandín AMR, Pérez IP, Iglesias PO, Díaz JJG. Cooperative oral presentations in higher education to enhance technical and soft skills in engineering students. Int J Continuing Eng Educ Life Long Learn. 2023;33(6):592–607.

Ragupathi K, Lee A. Beyond fairness and consistency in grading: The role of rubrics in higher education. Diversity and inclusion in global higher education: Lessons from across Asia. 2020:73–95.

Rabbi MF, Islam MS. The effect of academic stress and Mental anxiety among the students of Khulna University. Edukasiana: Jurnal Inovasi Pendidikan. 2024;3(3):280–99.

Popa CO, Schenk A, Rus A, Szasz S, Suciu N, Szabo DA, Cojocaru C. The role of acceptance and planning in stress management for medical students. Acta Marisiensis-Seria Med. 2020;66(3):101–5.

Christianson M, Payne S. Helping students develop skills for better presentations: Using the 20x20 format for presentation training. 語学研究. 2012;26:1–15.

Romero-Yesa S, Fonseca D, Aláez M, Amo-Filva D. Qualitative assessment of a challenge-based learning and teamwork applied in electronics program. Heliyon. 2023;9(12).

Bosco TJ, Gabriel B, Florence M, Gilbert N. Towards effective teaching and learning ESP in mixed classes: students’ interest, challenges and remedies. Int J Engl Literature Social Sci. 2020;5(2):506–16.

Qiao L, Xu Y, bin Ahmad N, An Analysis Of Implementation Challenges For English, For Specific Purposes (Esp) Formative Assessment Via Blended Learning Mode At Chinese Vocational Polytechnics. Journal Of Digital Education, Communication, And Arts (DECA). 2023;6(02):64–76.

Kho MG-W, Ting S-H. Overcoming oral presentation anxiety: a systematic review of Tertiary ESL/EFL Students’ challenges and strategies. Qeios. 2023.

Download references

We confirm that no funding was received for this work.

Author information

Authors and affiliations.

Department of English Language, School of Paramedical Sciences, Shiraz University of Medical Sciences, Shiraz, Iran

Elham Nasiri & Laleh Khojasteh

You can also search for this author in PubMed   Google Scholar

Contributions

L.KH was involved in writing the proposal, reviewing the text, analyzing the data, and writing the manuscript. E. N was involvedin designing the research and collecting and analyzing the data. Both authors have reviewed and approved the final version of the manuscript.

Corresponding author

Correspondence to Laleh Khojasteh .

Ethics declarations

Ethics approval and consent to participate.

Our study, entitled “Evaluating Panel Discussions in ESP Classes: An Exploration of International Medical Students’ and ESP Instructors’ Perspectives through Qualitative Research,” was reviewed by the Institutional Review Board (IRB) of the School of Paramedical Sciences, Shiraz University of Medical Sciences (SUMS). The IRB reviewed the study on August 14th, 2024, and determined that formal ethics approval or a reference number was not required. This decision was based on the fact that the research posed minimal risk to participants and focused solely on their educational experiences without involving any intervention or the collection of sensitive personal data.

Consent for publication

Not Applicable.

Competing interests

We confirm that there are no known conflicts of interest associated with this publication and that this work did not receive any financial support.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Nasiri, E., Khojasteh, L. Evaluating panel discussions in ESP classes: an exploration of international medical students’ and ESP instructors’ perspectives through qualitative research. BMC Med Educ 24 , 925 (2024). https://doi.org/10.1186/s12909-024-05911-3

Download citation

Received : 08 May 2024

Accepted : 14 August 2024

Published : 26 August 2024

DOI : https://doi.org/10.1186/s12909-024-05911-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Group oral presentations
• Medical students
• Panel discussions
• ESP courses

BMC Medical Education

ISSN: 1472-6920

• General enquiries: [email protected]

• Open access
• Published: 28 August 2024

Pre-implementation planning for a sepsis intervention in a large learning health system: a qualitative study

• Tara A. Eaton 1 ,
• Marc Kowalkowski 1 , 2 ,
• Ryan Burns 3 ,
• Hazel Tapp 4 ,
• Katherine O’Hare 5 &
• Stephanie P. Taylor 6  

BMC Health Services Research volume  24 , Article number:  996 ( 2024 ) Cite this article

Metrics details

Sepsis survivors experience high morbidity and mortality. Though recommended best practices have been established to address the transition and early post hospital needs and promote recovery for sepsis survivors, few patients receive recommended post-sepsis care. Our team developed the Sepsis Transition and Recovery (STAR) program, a multicomponent transition intervention that leverages virtually-connected nurses to coordinate the application of evidence-based recommendations for post-sepsis care with additional clinical support from hospitalist and primary care physicians. In this paper, we present findings from a qualitative pre-implementation study, guided by the Consolidated Framework for Implementation Research (CFIR), of factors to inform successful STAR implementation at a large learning health system prior to effectiveness testing as part of a Type I Hybrid trial.

We conducted semi-structured qualitative interviews ( n  = 16) with 8 administrative leaders and 8 clinicians. Interviews were transcribed and analyzed in ATLAS.ti using a combination deductive/inductive strategy based on CFIR domains and constructs and the Constant Comparison Method.

Six facilitators and five implementation barriers were identified spanning all five CFIR domains (Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individuals and Process). Facilitators of STAR included alignment with health system goals, fostering stakeholder engagement, sharing STAR outcomes data, good communication between STAR navigators and patient care teams/PCPs, clinician promotion of STAR with patients, and good rapport and effective communication between STAR navigators and patients, caregivers, and family members. Barriers of STAR included competing demands for staff time and resources, insufficient communication and education of STAR’s value and effectiveness, underlying informational and technology gaps among patients, lack of patient access to community resources, and patient distrust of the program and/or health care.

Conclusions

CFIR proved to be a robust framework for examining facilitators and barriers for pre-implementation planning of post-sepsis care programs within diverse hospital and community settings in a large LHS. Conducting a structured pre-implementation evaluation helps researchers design with implementation in mind prior to effectiveness studies and should be considered a key component of Type I hybrid trials when feasible.

Trial registration

Clinicaltrials.gov, NCT04495946 . Registered August 3, 2020.

Peer Review reports

Contributions to literature

This qualitative pre-implementation study of a telehealth nurse navigator-led sepsis transition and recovery (STAR) program demonstrates the Consolidated Framework for Implementation Research (CFIR) is useful to explore contextual conditions of healthcare settings as part of rigorous pre-implementation planning efforts.

This analysis identified actionable facilitators and barriers spanning all five CFIR domains (e.g., inner setting, outer setting) to inform and enhance initial implementation strategies of STAR.

These findings help to close recognized gaps in the literature on post-sepsis survivorship, including how to plan implementation of evidenced-based practices to address transition and early post hospital needs of sepsis survivors and promote recovery.

Sepsis, a common and life-threatening dysregulated response to infection, remains a major cause of morbidity, mortality, and healthcare costs [ 1 , 2 , 3 ]. Although hospital survival has improved in recent years, the increasing number of sepsis survivors are vulnerable to additional health problems [ 4 , 5 , 6 ]. Fewer than one-half of sepsis survivors return to their pre-sepsis health status and many experience new or worsening physical, cognitive, and psychological impairments, along with high rates of rehospitalization and excess mortality for years after sepsis hospitalization [ 7 , 8 , 9 ]. Given increasing recognition of the substantial long-term sequelae and social determinants of health-related needs after sepsis [ 10 ], recommended best practices have been established to address the transition and early post hospital needs and promote recovery for sepsis survivors [ 11 , 12 , 13 ]. However, like the majority of other evidence-based practices (EBPs) that have yet to be successfully adopted into routine practice, few patients receive recommended post-sepsis care [ 14 , 15 ].

To address the transition and early post hospital needs for sepsis survivors, our team developed the Sepsis Transition and Recovery (STAR) program, a multicomponent transition intervention that leverages centrally-located, virtually-connected nurses to coordinate the application of evidence-based recommendations for post-sepsis care with additional clinical support from hospitalist and primary care physicians [ 16 ]. The STAR program, based on the chronic care model [ 17 ], empowers patients and clinicians, via targeted education and coordinated care approaches, and was found to improve mortality and readmission outcomes among sepsis survivors [ 18 ]. There are complex barriers to translation of research findings into real-world post-sepsis care which we sought to identify and mitigate prior to effectiveness testing as part of a Type I Hybrid trial [ 19 ].

Before initiating a large-scale, pragmatic effectiveness evaluation of the STAR program (NCT04495946), we conducted a qualitative pre-implementation study with the aim to identify actionable facilitators and barriers to inform and enhance initial implementation strategies of the program across diverse hospital and community settings in a large Learning Health System (LHS). Qualitative methods are considered an integral component of implementation research and are well-known for being rigorous and efficient in the study of the hows and whys of implementation [ 20 ]. Conducting a robust pre-implementation evaluation was an intentional design choice for the overall project given the critical role of this step in the implementation process [ 21 ]. Through our qualitative investigation, we explored variations in stakeholder perspectives of the program by interviewing both administrators and clinicians.

We guided our study with the Consolidated Framework for Implementation Research (CFIR), due to its breadth, widespread use [ 22 , 23 ], and expert-recommended mapping from CFIR-identified barriers to defined implementation strategies [ 24 ]. As a framework, the CFIR offers a systematic approach well-known for planning, evaluating, and supporting behavioral change for a diverse array of studies [ 25 ], using a consistent language of 39 constructs organized across five domains—Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individuals and Process [ 22 ]. It can be used to build implementation knowledge to describe determinants of implementation [ 23 ], as well as tailor pre-implementation strategies to promote intervention success [ 26 , 27 ].

For this pre-implementation study, we conducted a qualitative investigation to identify facilitators and barriers to implementing the STAR program in hospital transition care, and to elaborate and compare key stakeholder perspectives. Instrument development, data collection, analysis, and interpretation of study results were guided by the CFIR. A PhD-level trained qualitative health services researcher (TE) on the study team with experience conducting qualitative research for program evaluations and intervention development led the process of interview instrument design, data collection, and analysis. She was not known to participants of the research prior to undertaking the study. Our study team followed the Standards for Reporting Qualitative Research in the reporting of this work [ 28 ].

Study design

The pre-implementation study was conducted from March through July of 2020 in preparation for the planned implementation of the STAR program intervention in July 2020 at a large LHS. Headquartered in Charlotte, North Carolina, Atrium Health provides not-for-profit healthcare supporting over 14 million patient encounters annually across 40 hospitals and over 1,000 care locations in North Carolina, Georgia, and Alabama. We identified all stakeholders involved with post-sepsis care in this health system according to a framework for stakeholder mapping in health research [ 29 ]. With sepsis survivors and caregivers at the center of our focus for STAR, we identified stakeholder categories relevant to them to determine our recruitment approach for the pre-implementation interviews. By employing an iterative process of delineation between key individuals and groups involved in post-sepsis care at the LHS, we identified key stakeholders.

These stakeholders comprised two main groups: administrative leaders and clinicians. Administrative leaders were chief medical and nursing officers. We selected administrators due to their understanding of outer and inner setting factors and influence on organizational policy. Clinicians were hospitalists and ambulatory care providers representing diverse practice settings. We selected clinicians as representative intervention users with knowledge of intervention characteristics, outer setting, inner setting, characteristics of individuals, and process factors. We purposively sampled potential participants to reflect these organizational roles and responsibilities at the planned intervention sites. We aimed to recruit individuals to sufficiently capture a range of beliefs about post-sepsis care in these practice settings, while limiting redundancy in our data collection.

The final sample included 8 administrators (Chief Medical Officers, Nursing Executives, and a Departmental Chair; representing 7 study hospitals and leadership over post-hospital continuing care and primary care services) and 8 clinicians (with specialty areas in one or more of the following: Hospital Medicine, Internal Medicine, Infectious Disease, Family Medicine and Critical Care; representing individuals with care privileges at 6 study hospitals and primary care responsibilities in the communities served by these hospitals). See Table 1 : Participant Characteristics.

Data collection

We conducted semi-structured qualitative interviews with 16 stakeholders from diverse hospitals and care settings to explore organizational support, culture, workflow processes, needs, and recommendations for STAR’s implementation. Separate and original interview guides were developed for administrator and clinician groups (See Additional file 1: Administrator Interview Guide and Additional file 2: Clinician Interview Guide) in this study, however, both guides included questions about stakeholder roles and work environments, the fit of the STAR program for their facilities which was facilitated using a printed intervention workflow diagram (See Fig. 1 : Patient Trajectory through the STAR Program), and questions about the implementation of STAR. Interview guides intentionally included questions representative of all 5 of the CFIR domains (Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individuals and Process) and were initially scripted by adapting questions from the CFIR Interview Guide Tool available at the CFIR website, www.cfguide.org [ 30 ]. Some of the sample questions from the guides are included below:

Patient trajectory through the STAR program

Do you think effectiveness data about the sepsis transition program would be needed to get team buy-in in your facility? (Intervention Characteristics)

How well, would you say, are new ideas (e.g., work processes, new interventions, QI projects, research) embraced and used to make improvements in your facility? (Inner Setting)

What, if any, barriers do you think patients will face to participate in the intervention? (Outer Setting)

What is your role within the organization? (Characteristics of Individuals)

Who would you recommend are the key individuals to speak with to make sure new interventions are successful in your practice or department? (Process)

We pilot tested (field tested) the interview guides in three rounds prior to their administration and iteratively refined the guides based on participant feedback and research team members’ perceptions of the usefulness of the data collection instruments for eliciting information we intended to capture for each stakeholder group (See Fig. 2 : Diagram of Interview Guide Development at Pre-Implementation). Field testing is an established technique in qualitative research for developing interview guides as it provides researchers with the opportunity to practice asking the interview questions and identify weaknesses in the wording and order of questions when spoken aloud [ 31 ]. We then used the refined data collection instruments for the interviews reported here.

Diagram of interview guide development at pre-implementation

Prior to each interview, participants received standardized background information about the study topic and verbal informed consent was obtained. As an adaptation due to research restrictions during the COVID-19 pandemic, interviews were conducted telephonically. Interviews were on average 30 min in duration, which was expected given the number of questions asked of participants (13 questions for the administrators and 15 questions for the clinicians) and what was seen during the pilot testing of the interview guides prior to data collection. Participants were offered a $25 gift card for their participation. Ethical approval for this study was granted by the Advarra IRB Committee.

Data analysis

Interview recordings were transcribed and entered into ATLAS.ti X8 as text documents for thematic coding and analysis. One team member with extensive experience in qualitative research methods (TE) led the analysis of the data set using a combination deductive/inductive strategy based on CFIR domains and constructs and the Constant Comparison Method. The Constant Comparison Method is an inductive approach for developing code structure through the iterative comparison of newly coded text with previously coded text of the same theme until final thematic refinement is achieved [ 32 ]. We referred to the cfirguide.org website’s CFIR Codebook Template [ 33 ], containing domain and construct definitions and guidance for coding qualitative data with the framework and inclusion and exclusion criteria for most constructs, in our application of the framework to our codebook development and analysis. This process included creating a codebook (a complete list of codes and definitions for each code), coding the data set among team members, comparing identified codes, and merging codes when it was necessary based on analytical discussion. Each code was labeled using the following convention: 1) if it was an implementation facilitator or barrier code, 2) a simplified title indicating what the code was, and 3) and a tag of the CFIR domains and constructs that corresponded to the code. E.g., ImplFacilitator_Family support for PT: OUTSET-PT Needs & Res. Throughout the process of analyzing the qualitative interview data, our study team met bi-weekly to discuss the results and engaged with the larger stakeholder group monthly to discuss ideas for overcoming identified barriers.

To promote the reliability of the analysis and prevent interpretive bias, two study team members (TE and RB) completed inter-rater reliability (IRR) coding for 50% of the administrator interviews (n = 4). Three team members (TE, KO, and HT) completed IRR for 50% of the clinician interviews ( n  = 4). IRR was conducted by having additional coders (RB, KO, and HT) apart from the principal analyst (TE) apply the codebook to the data set to determine whether they agreed with the original coding of selected interview transcripts. Instances of disagreement were discussed thoroughly and, at times, resulted in the application of additional codes for selected quotations. All identified conflicts in coding were fully resolved, resulting in a final agreement of 100% between coders.

Using a combination deductive/inductive coding strategy, we found 77 codes related to STAR implementation facilitators ( n  = 38) and barriers ( n  = 39) and labeled those codes with applicable CFIR domains and constructs as appropriate. The STAR implementation facilitators and barriers codes were then aggregated into 11 themes consisting of 6 facilitators (See Table 2 ) and 5 implementation barriers (See Table 3 ). STAR implementation facilitators and barriers, together, spanned all five CFIR domains (Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individuals and Process). Administrators and clinicians reported no other sepsis-specific transition programs in their facilities at the time of data collection and indicated the STAR program would be important to address sepsis survivor needs.

Facilitators influencing the implementation of STAR

Our analysis identified six themes pertaining to implementation facilitators. See Table 2 : CFIR-Guided Facilitators of STAR Implementation.

Alignment between STAR and health system goals

Participants reported that STAR’s alignment with other telehealth programs at the LHS, such as virtual hospital care, amidst surge of telehealth care during the COVID-19 pandemic would promote implementation of STAR as indicated in the administrator’s response below:

“I also think it [STAR] would be well received based on the information regarding virtual hospital and what we have been able to achieve with that. And, again with just looking for the bright spots in COVID, there have been a lot of transitions that have taken place in the last couple of months that I think you would have a much easier time implementing this in the new world of healthcare.” (A7)

Beyond virtual care, participants also described other existing infrastructure within the LHS that would align with the STAR program objectives, including sepsis work groups and sepsis champions from physicians, nurses, pharmacy, and case management. These inner setting facilitators combined demonstrate how STAR’s alignment with the implementation climate (compatibility) and structural characteristics of the LHS would influence its adoption.

Fostering engagement with stakeholders

Participants stated that fostering engagement to promote buy-in with stakeholders, including administrators, care teams, patients and caregivers, would facilitate the implementation of STAR. They recommended stakeholders be educated about what STAR is, its benefits, and for organizational stakeholders, how best to integrate STAR into their facility. See the clinician’s response below:

“I think just education [about STAR]. Just tons of education to everyone in the hospital that touches a patient. The nurses. The critical care physicians. The Hospitalists…But I think just educating the patient [about STAR] at the time of admission, just start that process. You know, this is our sepsis program, and let them know that this is going to happen at the time of discharge. And then also provide education to the providers.” (C1)

Participants also emphasized the importance of leaders heading communication about STAR with care teams and STAR navigators establishing a good rapport with clinicians who have patients enrolled in the program. See the clinician’s response below:

“Well, definitely share the information [about STAR] with their [health system leaders] teams. We have a normal leadership structure that provides the mechanism for things like this to be communicated in top down. And for sure, expecting the leaders to disseminate it from Level 2 to Level 3, Level 3 to Level 4 and on down. You know, that would be a minimum expectation…I think they should welcome you all [the STAR study team] at the meetings and give you time on the agenda to share your initiatives, at a minimum.” (C3)

These responses illustrate the relevance of the CFIR outer setting, process, and characteristics of individuals domains for the implementation of STAR, where prioritizing patient needs, attracting and involving appropriate individuals, and individual attitudes about the intervention would be facilitators of its adoption.

Share positive STAR outcomes data

Participants reported sharing positive results or impacts from the program would be helpful. They recommended using STAR performance metrics as motivation for continued buy-in and that leaders share effectiveness data. See the clinician responses below:

“I think readmission data [would be good to provide], like at 90 days, because if you are trying to get people to buy in for 90 days, cause that’s a long time, that’s about three months, I think you need to prove that it is worthwhile. If you’re trying to cut back on that 90-day readmission, because that’s what Medicare looks at, I think that would maybe entice some people to participate.” (C7)

“But, if you want to implement it as a standard process then we are going to have to see some sort of data on it before we say “yep, let’s do it”. Because there are many things that are competing for the resources that we have. So we have to on the basis on which our decisions on where the money goes, where those resources get diverted to is based on how efficiently they affect patient care, rates of readmission, and patient mortality. So we need the data to make an informed decision.” (C2)

Responses pertaining to this theme point to the significance of the CFIR intervention characteristics, inner setting and process domains in STAR’s implementation. Participants’ remarks regarding STAR’s evidence, strength and quality, shared receptivity to STAR within the LHS, and the recommendation to provide quantitative and qualitative feedback for reflecting and evaluating STAR’s quality would be facilitators of its implementation.

Good communication between STAR navigators and patient care teams/PCPs

Participants stated that good communication and recommendation-sharing between the STAR navigator and the patient’s care team and PCP will make STAR’s implementation successful. See the clinician’s response below:

“So, I think, effectively communicating with one another [the STAR navigator and clinician] what is beneficial and helping us ultimately provide for the patient from our end would be helpful. It will be a learning process, but you know, I think once we both communicate what we need from the other to be able to do our jobs, then I think that would be fine if that makes sense.” (C5)

These intervention characteristics and inner setting facilitators demonstrate the importance of intervention design, including how well STAR is bundled, presented and assembled to stakeholders, and navigator-led communication in its implementation.

Clinician promotion of STAR with patients

Our study participants emphasized the importance of clinician promotion of STAR with enrolled patients for implementation success. Specifically, our participants recommended that the LHS show patients their primary care providers and STAR navigators are in alignment to engender patient trust in the program. See the administrator’s response below:

“It always helps if they [patients] feel like it’s their own physicians or their own team that is a part of this. I think it would be important for it not to look like it was some external program that their clinicians were not involved in. So, I think, you know, trust always is important if you feel like people that you trust are endorsing something or believing it’s going to be useful.” (A8)

Similarly, one clinician said:

“I think trust, you know, would be a factor. A lot of times if patients view resources as being disconnected from their Primary Care, they may not be very accepting of them. So, if they view them as being part of “my team”, I think patients are much more likely to participate.” (C3)

Participant responses within this theme underscore the multi-domain influence of outer setting, inner setting and the process of implementation in the success of STAR, where the LHS’s prioritization of patient needs, LHS members’ and structures’ characteristics and behaviors, and the engagement of individuals with STAR would be facilitators of its implementation.

Good rapport and effective communication between STAR navigators and patients, caregivers, and family members

Participants reported that good rapport and effective communication between STAR navigators and enrolled patients and their caregivers/families would be important for implementing STAR. They emphasized the need for STAR navigators to foster a good connection with patients and their caregivers or family members. They also spoke to the integral role caregivers and family members play in patients’ post-sepsis recovery as additional points of contact who are familiar with the program if the patient does not recall what STAR is or if the patient is too ill to speak for themselves. See the clinicians’ responses below:

“I think patients get called a lot about a lot of things and they don’t always know who the person on the phone is. So, I think having that established and really something that the patient is okay with is important. And engaging, if possible, family or support members. I think that reduces barriers if they have support people available.” (C6)

“I think obviously reaching out to the family and support staff and things like that may be helpful. Some of our patients, in general, even at their baseline and at their best day aren’t going to be able to provide you the information that you need, or may not be able to provide an adequate history, or have an appropriate follow-up, and things like that, in place to be able to give you the information you need to help them as well as you would like.” (C5)

Responses within this facilitator theme highlight the importance of intervention characteristics, such as the perceived quality of STAR, and outer setting domains and constructs (patient needs and resources) in STAR navigator communication with patients and their caregivers and family members. Results show how effective navigator communication when presenting STAR to patients and their caregivers/family members, consideration of patient needs and barriers to participation, and the involvement of caregivers or family members would be facilitators of STAR’s implementation.

Barriers influencing the implementation of STAR

Our analysis identified five themes pertaining to implementation barriers. See Table 3 : CFIR-Guided Barriers to STAR Implementation.

Competing demands for staff time and resources

Participants reported that competing demands for staff time and resources, including the busy state of the LHS’ facilities at the time, COVID priorities, other concurrent program implementations, and a lack of time among clinicians to engage with STAR could be barriers to its implementation. See the administrator’s response below:

“So, I think barriers would be too many implementations going on at the same time. It would fail. The other is, right now in COVID time, it’s unlikely to muster enough support or enough interest to do it. I think we need to look at what else is going on, so that there is not information overload for the front-end teammates. And the other thing we look at is, most of these programs become paper intensive or computer intensive. That means, you are just putting things there, and then, if you ask people to do too much, yes, they do too much, but they don’t really do the thing…So just be mindful of that, what you expect them to spend time on.” (A3)

Similarly, one clinician commented:

“Now, from a willingness standpoint, not that people would necessarily disagree with the overall goals and the process of your program, it’s just that if you’re in my field, and in some of my partners, if we are being pulled in ten different directions at one time, you have to prioritize what you can do in a day. So, not willingness from the standpoint of people not wanting to participate, but sometimes people not being able to weight or value that as high as something else that needs to be done.” (C5)

Participants responses pertaining to this barrier theme illustrate the role that the LHS’s inner setting, specifically its implementation climate of decreased organizational capacity to absorb change and a lack of resources dedicated for STAR, would play in hindering the implementation of the program.

Insufficient communication and education of program value and effectiveness

Participants reported that insufficient communication and education of STAR’s value and effectiveness to other clinicians could be barriers to its implementation. See the administrator’s response below:

“To me, it’s always a matter of communication. If there was, if communication didn’t work, people didn’t see it had value, they didn’t want to put any effort into it, you know, those would really be obviously the big things.” (A8)

“So, if it’s not marketed like correctly or appropriately. If we really as attending or residents don’t see the benefit. You know, is this just another checky box, or is this really going to impact our patients in the long term? Will this make a difference in their survival? Or getting them back to a base line or improvements on a base line? I think that’s probably what’s going to help make it successful or not.” (C8)

Responses related to this barrier theme show that the LHS’s inner setting and characteristics of individuals (clinicians) are important implementation domains in the adoption of STAR. Participants identified poor quality communication, and a lack of clinician knowledge and positive beliefs about STAR’s value, would be barriers to the implementation of the program.

Underlying informational and technology gaps among patients

Participants reported several patient-facing factors related to information and technology gaps among patients that could be barriers to implementing STAR. This included a patient’s health literacy or understanding of STAR, a patient’s digital literacy, and a patient’s lack of access to technology when communicating with the STAR telehealth navigator. See the clinician responses below:

“Well, I think a lot of our patients don’t have secure housing. I think our patients’ baseline social determinants of health, like consistent phone numbers, housing, health literacy around that, I think that’s a barrier that a patient would experience [to participate in the intervention].” (C6)

“I think the only barrier is that they [patients] may not understand what is going on. But that’s okay [as if not a big deal], as long as they are receptive to someone talking to them. And like I said, I want to be respectful of our patients, but some of them just do not have the medical literacy or the insight to understand….So, I think a barrier might be that the patient may not understand why you are calling and why you are asking those questions.” (C1)

“Definitely patients have to be capable of doing it uh participating with the Telehealth. At least from the perspective of a lot of my patients and during the Coronavirus pandemic, it has been difficult to get some buy in with Telehealth linkages to care. We have a very rural population and there is some adherence issues with trying to initiate, you know, telephonic or video visits that we have kind of noticed over the last several months. So, patient participation I think in some settings would be challenging.” (C4)

Participant responses within this barrier theme highlight the importance of the outer setting (external to the LHS) in the challenge of implementing STAR, where literacy and technology gaps among patients could be barriers to program enrollees’ participating in the telehealth-based intervention.

Lack of access to community resources for patients

Finally, participants reported that a patient’s lack of access to community resources, including limited primary care, paramedicine, home physical therapy, speech therapy and mental health resources in certain communities (e.g., rural communities), could pose a barrier to the implementation of the STAR program. See the clinicians’ responses below:

“I think that the idea is a good idea [pauses], but it’s just where it would work best based upon the resources of the area. I think that is going to be the major challenge.” (C7)

“Just getting plugged into community resources that can assist with their psycho-social needs as well as their comorbidities” [would be a barrier to patient participation]. (A1)

Participant responses within this theme demonstrate the relevance of intervention characteristics and the outer setting when implementing EBPs for post-sepsis care for patients who lack access to community resources. The extent to which STAR cannot adapt and meet patients’ local needs, especially those of patients who live in areas where there are insufficient resources, will be a barrier to its implementation.

Patient distrust of the program and/or healthcare

Both administrators and clinicians interviewed stated that patient distrust of the program and/or healthcare could be a potential barrier to STAR’s implementation. These reasons included patients being slow to trust a new provider, discomfort when talking with a navigator, feeling skeptical of providers who seem unaffiliated with their primary care, and general distrust of the healthcare system, particularly for patients in rural communities or impoverished areas. See the administrator and clinician responses below:

“You know, people are always a little wary of people they do not know, especially in small and rural communities.” (A1)

“Yeah, I think most of the barriers that are already well known that go with socio economic status or poverty. Trust in the healthcare system. I think those are all going to be barriers.” (C4)

Responses within this theme point to the significance of outer setting factors and the extent to which a patient’s need to trust their provider is accurately known and prioritized by the STAR navigator. Data suggests patient distrust of the STAR program or other providers would be a barrier to implementing EBPs for post-sepsis care.

A foundation of implementation science is that intervention delivery should be tailored to local context to maximize uptake and impact [ 34 , 35 ]. Formative, or pre-implementation, evaluations facilitate initial assessment of the local context and the potential determinants for implementation success within that context. Multiple theoretical frameworks have been applied to pre-implementation evaluations; the Consolidated Framework for Implementation Science (CFIR) is one of the most widely used due to its ability to comprehensively identify implementation facilitators and barriers [ 36 ]. In this study, we utilized qualitative pre-implementation interviews to identify actionable facilitators and barriers to inform and enhance initial implementation strategies of the STAR program across diverse hospital and community settings in a large LHS. From this work, our study offers several contributions to the literature on post-sepsis care.

First, our study successfully leveraged the CFIR to inform and enhance initial implementation strategies of the STAR program across diverse hospital and community settings in a large LHS. This is in line with other studies that similarly applied the CFIR during pre-implementation and found implementation determinants like ours, such as stakeholder involvement being necessary to promote buy-in and the relevance of intervention fit within the organization’s inner setting [ 26 , 37 ]. While some have applied CFIR in the pre-implementation planning of a sepsis management intervention at a single site [ 38 ], to our knowledge, our team is the first to apply the CFIR at pre-implementation to inform the design and dissemination of a sepsis transition and recovery intervention for patients within a large LHS. We decided to guide our interview instrument development and subsequent analysis using the CFIR because we were interested, fundamentally, in the organizational change that will be needed to successfully implement the STAR program. By incorporating the CFIR domains and constructs into our interview instruments and intervention planning, our study was able to identify implementation partners and collect stakeholder input on the potential facilitators and barriers to the STAR program at a large LHS. One benefit of using the CFIR for pre-implementation work is the potential for direct translation to implementation strategies selection using the Expert Recommendations for Implementing Change (ERIC) mapping.

Second, study findings revealed the importance of stakeholder buy-in like other CFIR-guided pre-implementation studies [ 26 , 39 ] across diverse groups, including administrators, care teams, patients, and caregivers. Implementation facilitators related to buy-in that were identified included active engagement with stakeholders, education about STAR, the sharing of positive outcomes data from STAR with clinicians, and promotion of the program’s value throughout implementation. Participants also emphasized the criticality of demonstrating alignment between clinicians and the STAR program. This included the recommendation for clinician support and promotion of STAR with patients to engender patient trust in the program. Conversely, our study found implementation barriers pertaining to lack of stakeholder buy-in as well. These included that a lack of engagement and education about the post-sepsis care program’s value and effectiveness, possible patient distrust of STAR and/or of health care, and patients’ lack of access to community resources could be potential barriers to its implementation. Together these findings point to the necessity of stakeholder buy-in for overcoming inner and outer setting barriers to implementation. They also suggest successful championing of STAR should extend beyond navigator efforts alone and include system and care team participation as well.

Third, our study found the STAR program’s fit with the LHS’s inner setting to be informative for our planning. Participants reported STAR’s compatibility with the structural characteristics and implementation climate of the LHS to likely be important considerations for implementation. At the time of this study, virtual hospital care and other telehealth programs were highly active within the LHS, in part related to the need for such programs during the surge of the COVID-19 pandemic. Additionally, STAR’s alignment with other sepsis-focused work groups and sepsis champions across the LHS was identified as another possible facilitator for implementation success. We found implementation barriers pertaining to the implementation climate of the LHS’s inner setting as well. Despite acknowledging that the program would likely align with current health system goals, participants cautioned STAR would have to compete with demands for staff time and resources. Decreased organizational capacity for a new program was another potential implementation barrier identified. Participants recommended engaging clinicians about the value and effectiveness of the program to promote support and assuage concerns. These facilitators and barriers suggest health system priorities and routine healthcare practice in the inner setting should be identified and considered carefully when making post-sepsis care program implementation decisions. They also underscore how the inner setting is not simply a background of implementation but can rather serve as an important context in implementation success.

Finally, our study findings highlighted the importance of good communication between the STAR navigator and other stakeholders, including clinicians, patients, caregivers, and family members, for successful implementation. Participants recommended clear and reciprocal communication between STAR navigators and clinicians. Similarly, they advised that navigators attempt to establish good rapport with patients, caregivers, and family members by using effective communication. Several potential implementation barriers related to communication were also reported. Participants discussed underlying patient-facing information and technology gaps that could be potential barriers to communicating with STAR navigators related to digital literacy, health literacy, or a lack of access to technology to participate in STAR. These suggest further study may be recommended to identify other patient-facing environmental conditions, such as social determinants of health, affecting sepsis recovery, as proposed in other’s work [ 10 ]. These points underscore the necessity of both effective communication and communication technology to support telehealth-based sepsis transition and recovery intervention implementation.

Study limitations

A limitation of the present research is that it is based on interviews with a small sample of employees at one, albeit large, health system. Although we carefully sampled stakeholders based on their awareness, organizational authority, and involvement in activities related to implementation of a post-sepsis care intervention at study facilities, these perspectives may not necessarily reflect the experience of all facilities within the same LHS or outside of the LHS. A second limitation is that patients were not included as participants at pre-implementation, despite later finding several facilitators and barriers related to patient needs. Third, we deliberately used the CFIR, and included all domains, to inform our approach to data collection and analysis due to its comprehensive assessment of implementation determinants and well-described associations with implementation strategies. However, using CFIR alone may have limited collection of other relevant contextual factors not represented by CFIR or specifically incorporated in our data collection. Our analysis strategy that combined inductive and deductive methods did allow for capture of themes outside of CFIR, if new information emerged from participant responses. Finally, our analysis strategy focused specifically on identifying key individual determinants; thus, additional empirical analyses examining the causal pathways or combinations of contextual factors may be helpful to advance evidence and guide decision making regarding effective implementation strategies tailored to complex determinants.

Our findings demonstrate effective use of the CFIR as a robust framework to examine facilitators and barriers for pre-implementation planning of post-sepsis care programs within diverse hospital and community settings in a large LHS. The comprehensive structure of the framework enabled researchers to identify key implementation determinants across external-, internal-, and program-level domains, plan for organizational change associated with implementation, and engage with relevant stakeholders. Conducting a structured pre-implementation evaluation helps researchers design with implementation in mind prior to effectiveness studies and should be considered a key component of Type I hybrid trials when feasible.

Availability of data and materials

The datasets generated and analyzed during the study are not available due to participant privacy and ethics restrictions, but the codebook and data collection tools may be available from the corresponding author on reasonable request.

Abbreviations

Sepsis transition and recovery

Learning health system

Consolidated framework for implementation research

Evidence-based practices

Buchman T, Simpson SQ, Sciarretta KL, et al. Sepsis among medicare beneficiaries: 1. The burdens of sepsis, 2012–2018*. Crit Care Med. 2020;48(3):276–88. https://doi.org/10.1097/CCM.0000000000004224 . Published Online First: Epub Date.

Article   PubMed   PubMed Central   Google Scholar  

Rudd KE, Johnson SC, Agesa KM, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study. Lancet. 2020;395(10219):200–11. https://doi.org/10.1016/S0140-6736(19)32989-7 . Published Online First: Epub Date.

W. MK, Marc R. Most Frequent Principal Diagnoses for Inpatient Stays in U.S. Hospitals, 2018. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs 2021. 2021.  https://www.ncbi.nlm.nih.gov/books/NBK573113/ .

Levy MM, Evans LE, Rhodes A. The surviving sepsis campaign bundle: 2018 update. Crit Care Med. 2018;46(6):997–1000. https://doi.org/10.1097/ccm.0000000000003119 . Published Online First: Epub Date.

Article   PubMed   Google Scholar  

Rhee C, Dantes R, Epstein L, et al. Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009–2014. JAMA. 2017;318(13):1241–9. https://doi.org/10.1001/jama.2017.13836 . Published Online First: Epub Date.

Mostel Z, Perl A, Marck M, et al. Post-sepsis syndrome - an evolving entity that afflicts survivors of sepsis. Molecular medicine (Cambridge, Mass.) 2019;26(1):6. https://doi.org/10.1186/s10020-019-0132-z . Published Online First: Epub Date.

Farrah K, McIntyre L, Doig CJ, et al. Sepsis-associated mortality, resource use, and healthcare costs: a propensity-matched cohort study. Crit Care Med. 2021;49(2):215–27. https://doi.org/10.1097/ccm.0000000000004777 . Published Online First: Epub Date.

Prescott HC, Angus DC. Enhancing recovery from sepsis: a review. JAMA. 2018;319(1):62–75. https://doi.org/10.1001/jama.2017.17687 . Published Online First: Epub Date.

Shankar-Hari M, Saha R, Wilson J, et al. Rate and risk factors for rehospitalisation in sepsis survivors: systematic review and meta-analysis. Intensive Care Med. 2020;46(4):619–36. https://doi.org/10.1007/s00134-019-05908-3 . Published Online First: Epub Date.

Hilton RS, Hauschildt K, Shah M, Kowalkowski M, Taylor S. The assessment of social determinants of health in postsepsis mortality and readmission: a scoping review. Crit Care Explor. 2022;4(8):e0722. https://doi.org/10.1097/cce.0000000000000722 . Published Online First: Epub Date.

Prescott HC, Iwashyna TJ, Blackwood B, et al. Understanding and enhancing sepsis survivorship. Priorities for research and practice. Am J Respir Crit Care Med. 2019;200(8):972–81. https://doi.org/10.1164/rccm.201812-2383CP . Published Online First: Epub Date.

Huang CY, Daniels R, Lembo A, et al. Life after sepsis: an international survey of survivors to understand the post-sepsis syndrome. Int J Qual Health Care. 2019;31(3):191–8. https://doi.org/10.1093/intqhc/mzy137 . Published Online First: Epub Date.

Evans L, Rhodes A, Alhazzani W, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med. 2021;47(11):1181–247. https://doi.org/10.1007/s00134-021-06506-y . published Online First: Epub Date.

Taylor SP, Chou SH, Sierra MF, et al. Association between adherence to recommended care and outcomes for adult survivors of sepsis. Ann Am Thorac Soc. 2020;17(1):89–97. https://doi.org/10.1513/AnnalsATS.201907-514OC . published Online First: Epub Date.

Watson MA, Anderson C, Karlic KJ, et al. Receipt of recovery-oriented care practices during hospitalization for sepsis. Crit Care Explor. 2022;4(9):e0766. https://doi.org/10.1097/cce.0000000000000766 . Published Online First: Epub Date.

Taylor SP, Murphy S, Rios A, et al. Effect of a multicomponent sepsis transition and recovery program on mortality and readmissions after sepsis: the improving morbidity during post-acute care transitions for sepsis randomized clinical trial. Crit Care Med. 2022;50(3):469–79. https://doi.org/10.1097/ccm.0000000000005300 . Published Online First: Epub Date.

Wagner EH. Chronic disease management: what will it take to improve care for chronic illness? Effective clinical practice : ECP. 1998;1(1):2–4.

CAS   PubMed   Google Scholar  

Leppin AL, Gionfriddo MR, Kessler M, et al. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Intern Med. 2014;174(7):1095–107. https://doi.org/10.1001/jamainternmed.2014.1608 . Published Online First: Epub Date.

Kowalkowski M, Eaton T, McWilliams A, et al. Protocol for a two-arm pragmatic stepped-wedge hybrid effectiveness-implementation trial evaluating Engagement and Collaborative Management to Proactively Advance Sepsis Survivorship (ENCOMPASS). BMC Health Serv Res. 2021;21(1):544. https://doi.org/10.1186/s12913-021-06521-1 . Published Online First: Epub Date.

Hamilton AB, Finley EP. Qualitative methods in implementation research: an introduction. Psychiatry Res. 2019;280:112516. https://doi.org/10.1016/j.psychres.2019.112516 . Published Online First: Epub Date.

Alley ZM, Chapman JE, Schaper H, Saldana L. The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs. Implement Sci. 2023;18(1):30. https://doi.org/10.1186/s13012-023-01285-0 . Published Online First: Epub Date.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation science : IS. 2009;4:50. https://doi.org/10.1186/1748-5908-4-50

Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72. https://doi.org/10.1186/s13012-016-0437-z . Published Online First: Epub Date.

Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. https://doi.org/10.1186/s13012-015-0209-1 . Published Online First: Epub Date.

Birken SA, Powell BJ, Presseau J, et al. Combined use of the Consolidated Framework for Implementation Research (CFIR) and the Theoretical Domains Framework (TDF): a systematic review. Implement Sci. 2017;12(1):2. https://doi.org/10.1186/s13012-016-0534-z . Published Online First: Epub Date.

King DK, Shoup JA, Raebel MA, et al. Planning for implementation success using RE-AIM and CFIR Frameworks: a qualitative study. Front Public Health. 2020;8:59. https://doi.org/10.3389/fpubh.2020.00059 . Published Online First: Epub Date.

Dolansky MA, Pohnert A, Ball S, McCormack M, Hughes R, Pino L. Pre-Implementation of the age-friendly health systems evidence-based 4ms framework in a multi-state convenient care practice. Worldviews on evidence-based nursing. 2021;18(2):118–28. https://doi.org/10.1111/wvn.12498 . Published Online First: Epub Date.

O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51. https://doi.org/10.1097/acm.0000000000000388 . Published Online First: Epub Date.

Schiller C, Winters M, Hanson HM, Ashe MC. A framework for stakeholder identification in concept mapping and health research: a novel process and its application to older adult mobility and the built environment. BMC Public Health. 2013;13:428. https://doi.org/10.1186/1471-2458-13-428 . Published Online First: Epub Date.

Research CRT-CfCM. CFIR Interview guide tool. Secondary CFIR interview guide tool 2019. https://cfirguide.org/guide/app/#/ .

McGrath C, Palmgren PJ, Liljedahl M. Twelve tips for conducting qualitative research interviews. Med Teach. 2019;41(9):1002–6. https://doi.org/10.1080/0142159x.2018.1497149 . Published Online First: Epub Date.

Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health services research. 2007;42(4):1758–72. https://doi.org/10.1111/j.1475-6773.2006.00684.x . Published Online First: Epub Date.

Research CRT-CfCM. CFIR Codebook Template. Secondary CFIR Codebook Template 2019. https://cfirguide.org/tools/tools-and-templates/ .

Baker R, Camosso-Stefinovic J, Gillies C, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;3:Cd005470. https://doi.org/10.1002/14651858.CD005470.pub2 . Published Online First: Epub Date.

Article   Google Scholar  

Mittman BS. Implementation science in health care. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. New York: Oxford; 2012. p. 400–18.

Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Sci. 2022;17(1):75. https://doi.org/10.1186/s13012-022-01245-0 . Published Online First: Epub Date.

Piper KN, Brown LL, Tamler I, Kalokhe AS, Sales JM. Application of the consolidated framework for implementation research to facilitate delivery of trauma-informed HIV care. Ethn Dis. 2021;31(1):109–18. https://doi.org/10.18865/ed.31.1.109 . Published Online First: Epub Date.

Ahmed SI, Khowaja BMH, Barolia R, et al. Adapting the FAST-M maternal sepsis intervention for implementation in Pakistan: a qualitative exploratory study. BMJ Open. 2022;12(9):e059273. https://doi.org/10.1136/bmjopen-2021-73 . eCollection 2022.

Fokuo JK, McCuistian CL, Masson CL, et al. Pre-implementation assessment of tobacco cessation interventions in substance use disorder residential programs in California. Subst Use Misuse. 2022;57(9):1345–55. https://doi.org/10.1080/10826084.2022.2079139 . Published Online First: Epub Date.

Download references

Acknowledgements

The authors would like to thank all of the study participants for sharing their time and insights.

Research reported in this publication was supported by the National Institute Of Nursing Research of the National Institutes of Health under Award Number R01NR018434. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and affiliations.

Center for Health System Sciences, Atrium Health, 1300 Scott Ave, Charlotte, NC, 28203, USA

Tara A. Eaton & Marc Kowalkowski

Department of Internal Medicine, Medical Center Boulevard, Wake Forest University School of Medicine, Winston-Salem, NC, 27157, USA

Marc Kowalkowski

Department of Community Health, Atrium Health, 4135 South Stream Blvd, Charlotte, NC, 28217, USA

Department of Family Medicine, Atrium Health, 2001 Vail Ave., Suite 400B, Charlotte, NC, 28207, USA

Center for Outcomes Research and Evaluation, Yale New Haven Health, 195 Church Street, New Haven, CT, 06510, USA

Katherine O’Hare

Department of Internal Medicine, Taubman Center, University of Michigan, 1500 East Medical Center Drive, 3110SPC 5368, Ann Arbor, MI, 48109-5368, USA

Stephanie P. Taylor

You can also search for this author in PubMed   Google Scholar

Contributions

TE, ST, MK, and HT contributed to the design of the study. TE, ST, MK, RB and KO contributed to the acquisition of study data. TE, RB, KO, and HT analyzed the data. TE, ST, and MK contributed to the interpretation of the data. TE, ST, MK, RB, and HT drafted the manuscript. All authors critically revised the intellectual content of the manuscript. All authors approved the final manuscript.

Corresponding author

Correspondence to Tara A. Eaton .

Ethics declarations

Ethics approval and consent to participate.

This study has been approved to be conducted on behalf of all trial sites with a waiver of signed informed consent by the Advarra Institutional Review Board (#Pro00036873) on June 26, 2019. Participants were all given a study information sheet that was reviewed with them prior to their participation. All study methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Eaton, T.A., Kowalkowski, M., Burns, R. et al. Pre-implementation planning for a sepsis intervention in a large learning health system: a qualitative study. BMC Health Serv Res 24 , 996 (2024). https://doi.org/10.1186/s12913-024-11344-x

Download citation

Received : 18 August 2023

Accepted : 23 July 2024

Published : 28 August 2024

DOI : https://doi.org/10.1186/s12913-024-11344-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Sepsis (33)
• Pre-Implementation (16)
• Qualitative (10)

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Research on K-12 maker education in the early 2020s – a systematic literature review

• Open access
• Published: 27 August 2024

Cite this article

You have full access to this open access article

• Sini Davies   ORCID: orcid.org/0000-0003-3689-7967 1 &
• Pirita Seitamaa-Hakkarainen   ORCID: orcid.org/0000-0001-7493-7435 1  

30 Accesses

Explore all metrics

This systematic literature review focuses on the research published on K-12 maker education in the early 2020s, providing a current picture of the field. Maker education is a hands-on approach to learning that encourages students to engage in collaborative and innovative activities, using a combination of traditional design and fabrication tools and digital technologies to explore real-life phenomena and create tangible artifacts. The review examines the included studies from three perspectives: characteristics, research interests and findings, previous research gaps filled, and further research gaps identified. The review concludes by discussing the overall picture of the research on maker education in the early 2020s and suggesting directions for further studies. Overall, this review provides a valuable resource for researchers, educators, and policymakers to understand the current state of K-12 maker education research.

Similar content being viewed by others

Learning through Making and Maker Education

Maker Education: Opportunities and Threats for Engineering and Technology Education

Maker Movement in Education: History and Prospects

Explore related subjects.

• Artificial Intelligence
• Digital Education and Educational Technology

Avoid common mistakes on your manuscript.

Introduction

Maker culture developed through the pioneering efforts of Papert ( 1980 ) and his followers, such as Blikstein ( 2013 ), Kafai and Peppler ( 2011 ), and Resnick ( 2017 ). It has gained popularity worldwide as an educational approach to encourage student engagement in learning science, technology, engineering, arts, and mathematics (STEAM) (Martin, 2015 ; Papavlasopoulou et al., 2017 ; Vossoughi & Bevan, 2014 ). Maker education involves engaging students to collaborate and innovate together by turning their ideas into tangible creations through the use of conceptual ideas (whether spoken or written), visual representations such as drawings and sketches, and material objects like prototypes and models (Kangas et al., 2013 ; Koh et al., 2015 ). Another core aspect of maker education is combining traditional design and fabrication tools and methods with digital technologies, such as 3D CAD and 3D printing, electronics, robotics, and programming, which enables students to create multifaceted artifacts and hybrid solutions to their design problems that include both digital and virtual features (e.g., Blikstein, 2013 ; Davies et al., 2023 ; Riikonen, Seitamaa-Hakkarainen, et al., 2020 ). The educational value of such multi-dimensional, concrete making has become widely recognized (e.g., Blikstein, 2013 ; Kafai, 1996 ; Kafai et al., 2014 ; Martin, 2015 ).

Maker education has been studied intensively, as indicated by several previous literature reviews (Iivari et al., 2016 ; Lin et al., 2020 ; Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ; Yulis San Juan & Murai, 2022 ). These reviews have revealed how the field has been evolving and provided a valuable overall picture of the research on maker education before the 2020s, including only a few studies published in 2020 or 2021. However, the early years of the 2020s have been an extraordinary period in time in many ways. The world was hit by the COVID-19 pandemic, followed by the global economic crises, increasing geopolitical tensions, and wars that have had a major impact on societies, education, our everyday lives, and inevitably on academic research as well. Furthermore, 2023 was a landmark year in the development of artificial intelligence (AI). In late 2022, OpenAI announced the release of ChatGPT 3.5, a major update to their large language model that is able to generate human-like text. Since then, sophisticated AI systems have rushed into our lives at an accelerating speed and are now becoming integrated with other technologies and applications, shaping how we live, work, our cultures, and our environments irreversibly (see, e.g., World Economic Forum, 2023 ). Thus, it can be argued that towards the end of 2023, the world had transitioned into the era of AI. It is essential that researchers, educators, and policymakers have a fresh overall understanding and a current picture of research on K-12 maker education to develop new, research-based approaches to technology and design education in the present rapidly evolving technological landscape of AI. This is especially important in order to avoid falling back towards shallow epistemic and educational practices of repetition and reproduction. The present systematic review was conducted to provide a ‘big picture’ of the research on K-12 maker education published in the extraordinary times of the early 2020s and to act as a landmark between the research on the field before and after the transition to the AI era. The review was driven by one main research question: How has the research on maker education developed in the early 2020s? To answer this question, three specific research questions were set:

What were the characteristics of the studies in terms of geographical regions, quantity of publications, research settings, and research methods?.

What were the research interests and findings of the reviewed studies?.

How did the reviewed studies fulfill the research gaps identified in previous literature reviews, and what further research gaps they identified?.

The following will outline the theoretical background of the systematic literature review by examining previous literature reviews on maker culture and maker education. This will be followed by an explanation of the methodologies used and findings. Finally, the review will conclude by discussing the overall picture of the research on maker education in the early 2020s and suggesting directions for further studies.

Previous literature reviews on maker culture and maker education

Several literature reviews have been conducted on maker education over the past ten years. The first one by Vossoughi and Bevan ( 2014 ) concentrated on the impact of tinkering and making on children’s learning, design principles and pedagogical approaches in maker programs, and specific tensions and possibilities within the maker movement for equity-oriented teaching and learning. They approached the maker movement in the context of out-of-school time STEM from three perspectives: (1) entrepreneurship and community creativity, (2) STEM pipeline and workforce development, and (3) inquiry-based education. At the time of their review, the research on maker education was just emerging, and therefore, their review included only a few studies. The review findings highlighted how STEM practices were developed through tinkering and striving for equity and intellectual safety (Vossoughi & Bevan, 2014 ). Furthermore, they also revealed how making activities support new ways of learning and collaboration in STEM. Their findings also pointed out some tensions and gaps in the literature, especially regarding a focus that is too narrow on STEM, tools, and techniques, as well as a lack of maker projects conducted within early childhood education or families.

In subsequent literature reviews (Iivari et al., 2016 ; Lin et al., 2020 ; Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Yulis San Juan & Murai, 2022 ), the interests of the reviews were expanded. Iivari and colleagues ( 2016 ) reviewed the potential of digital fabrication and making for empowering children and helping them see themselves as future digital innovators. They analyzed the studies based on five conditions: conditions for convergence, entry, social support, competence, and reflection, which were initially developed to help with project planning (Chawla & Heft, 2002 ). Their findings revealed that most of the studies included in their review emphasized the conditions for convergence, entry, and competence. However, only a few studies addressed the conditions for social support and reflection (Iivari et al., 2016 ). The reviewed studies emphasized children’s own interests and their voluntary participation in the projects. Furthermore, the studies highlighted projects leading to both material and learning-related outcomes and the development of children’s competencies in decision-making, design, engineering, technology, and innovation through projects.

Papavlasopoulou and colleagues ( 2017 ) took a broader scope on their systematic literature review, characterizing the overall development and stage of research on maker education through analyzing research settings, interests, and methods, synthesizing findings, and identifying research gaps. They were specifically interested in the technology used, subject areas that implement making activities, and evaluation methods of making instruction across all levels of education and in both formal and informal settings. Their data comprised 43 peer-reviewed empirical studies on maker-centered teaching and learning with children in their sample, providing participants with any making experience. In Papavlasopoulou and colleagues’ ( 2017 ) review, the included studies were published between 2011 and November 2015 as journal articles, conference papers, or book chapters. Most of the studies were conducted with fewer than 50 participants ( n  = 34), the most prominent age group being children from the beginning of primary school up to 14 years old ( n  = 22). The analyzed studies usually utilized more than one data collection method, mainly focusing on qualitative ( n  = 22) or mixed method ( n  = 11) approaches. Most included studies focused on programming skills and computational thinking ( n  = 32) or STEM subjects ( n  = 6). The studies reported a wide range of positive effects of maker education on learning, the development of participants’ self-efficacy, perceptions, and engagement (Papavlasopoulou et al., 2017 ). There were hardly any studies reporting adverse effects.

Schad and Jones ( 2020 ) focused their literature review on empirical studies of the maker movement’s impacts on formal K12 educational environments, published between 2000 and 2018. Their Boolean search (maker movement AND education) to three major academic research databases resulted in 599 studies, of which 20 were included in the review. Fourteen of these studies focused on K12 students, and six on K12 teachers. All but three of the studies were published between 2014 and 2018. Similarly to the studies reported in the previous literature reviews (Iivari et al., 2016 ; Papavlasopoulou et al., 2017 ; Vossoughi & Bevan, 2014 ), the vast majority of the studies were qualitative studies that reported positive opportunities for maker-centered approaches in STEM learning and promotion of excitement and motivation. On the other hand, the studies on K12 in- and preservice teacher education mainly focused on the importance of offering opportunities for teachers to engage in making activities. Both, studies focused on students or teachers, promoting equity and offering equally motivating learning experiences regardless of participants’ gender or background was emphasized.

Lin and colleagues’ ( 2020 ) review focused on the assessment of maker-centered learning activities. After applying inclusion and exclusion criteria, their review consisted of 60 peer-reviewed empirical studies on making activities that included making tangible artifacts and assessments to measure learning outcomes. The studies were published between 2006 and 2019. Lin and colleagues ( 2020 ) also focused on all age groups and activities in both formal and informal settings. Most studies included applied STEM as their main subject domain and utilized a technology-based platform, such as LilyPad Arduino microcontroller, Scratch, or laser cutting. The results of the review revealed that in most studies, learning outcomes were usually measured through the assessment of artifacts, tests, surveys, interviews, and observations. The learning outcomes measured were most often cognitive skills on STEM-related content knowledge or students’ feelings and attitudes towards STEM or computing.

The two latest systematic reviews, published in 2022, also focused on specific research interests in maker education (Rouse & Rouse, 2022 ; Yulis San Juan & Murai, 2022 ). Rouse and Rouse ( 2022 ) reviewed studies that specifically investigated learning in preK-12 maker education in formal school-based settings. Their analysis included 22 papers from seven countries, all but two published between 2017 and 2019. Only two of the studies focused on early childhood education, and three involved participants from the elementary level. Like previous reviews, most studies were conducted with qualitative methods ( n  = 17). On the other hand, in contrast to the earlier reviews (Lin et al., 2020 ; Papavlasopoulou et al., 2017 ; Schad & Jones, 2020 ), the studies included in the review did not concentrate on content-related outcomes on STEM or computing. Instead, a wide range of learning outcomes was investigated, such as 21st-century skills, agency, and materialized knowledge. On the other hand, they found that equity and inclusivity were not ubiquitously considered when researchers design makerspace interventions. Yulis San Juan and Murai’s ( 2022 ) literature review focused on frustration in maker-centered learning activities. Their analysis consisted of 28 studies published between 2013 and 2021. Their findings of the studies identified six factors that are most often recognized as the causes of frustration in makerspace activities: ‘unfamiliar pedagogical approach, time constraints, collaboration, outcome expectations, lack of skills and knowledge, and tool affordances and availability’ (Yulis San Juan & Murai, 2022 , p. 4).

From these previous literature reviews, five significant research gaps emerged that required further investigation and attention:

Teacher training, pedagogies, and orchestration of learning activities in maker education (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ).

Wide variety of learning outcomes that potentially emerge from making activities, as well as the development of assessment methods and especially systematic ways to measure student learning (Lin et al., 2020 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ).

Equity and inclusivity in maker education (Rouse & Rouse, 2022 ; Vossoughi & Bevan, 2014 ).

Practices, tools, and technologies used in makerspaces and digital fabrication (Iivari et al., 2016 ; Papavlasopoulou et al., 2017 ).

Implementation and effects of maker education in formal, school-based settings and specific age groups, especially early childhood education (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ).

Methodology

This review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, adapting it to educational settings where studies are conducted with qualitative, quantitative, and mixed methods (Page et al., 2021 ; Tong et al., 2012 ). Review protocols were defined for data collection, inclusion, exclusion, and quality criteria and the data analysis. In the following, the method used for each stage of the review process will be defined in detail.

Data collection

To gather high-quality and comprehensive data, a search for peer-reviewed articles was conducted in three international online bibliographic databases: Scopus, Education Resources Information Center (ERIC), and Academic Search Complete (EBSCO). Scopus and EBSCO are extensive multi-disciplinary databases for research literature, covering research published in over 200 disciplines, including education, from over 6000 publishers. ERIC concentrates exclusively on educational-related literature, covering publications from over 1900 full-text journals. These three databases were considered to offer a broad scope to capture comprehensive new literature on K-12 maker education. The search aimed to capture peer-reviewed literature on maker education and related processes conducted in both formal and informal K-12 educational settings. The search was limited to articles published in English between 2020 and 2023. Major search terms and their variations were identified to conduct the search, and a Boolean search string was formed from them. The search was implemented in October 2023 with the following search string that was used to search on titles, abstracts, and keywords:

(“maker education” OR “maker pedagogy” OR “maker-centered learning” OR “maker centered learning” OR “maker-centred learning” OR “maker centred learning” OR “maker learning” OR “maker space*” OR makerspace* OR “maker culture” OR “design learning” OR “maker practices” OR “collaborative invention*” OR co-invention*) AND (“knowledge-creation” OR “knowledge creation” OR “knowledgecreation” OR maker* OR epistemic OR “technology education” OR “design-based learning” OR “design based learning” OR “designbased learning” OR “design learning” OR “design thinking” OR “codesign” OR “co-design” OR “co design” OR craft* OR tinker* OR “collaborative learning” OR inquiry* OR “STEAM” OR “project-based learning” OR “project based learning” OR “projectbased learning” OR “learning project*” OR “knowledge building” OR “making” OR creati* OR innovat* OR process*) AND (school* OR pedago* OR “secondary education” OR “pre-primary education” OR “primary education” OR “special education” OR “early childhood education” OR “elementary education” OR primary-age* OR elementary-age* OR “k-12” OR “youth” OR teen* OR adolescen* OR child* OR “tween”) .

Inclusion and exclusion criteria

The search provided 700 articles in total, 335 from Scopus, 345 from EBSCO, and 20 from ERIC that were aggregated to Rayyan (Ouzzani et al., 2016 ), a web and mobile app for systematic reviews, for further processing and analysis. After eliminating duplicates, 513 studies remained. At the next stage, the titles and abstracts of these studies were screened independently by two researchers to identify papers within the scope of this review. Any conference papers, posters, work-in-progress studies, non-peer-reviewed papers, review articles, and papers focusing on teacher education or teachers’ professional development were excluded from the review. To be included, the study had to meet all the following four inclusion criteria. It had to:

show empirical evidence.

describe any making experience or testing process conducted by the participants.

include participants from the K-12 age group in their sample.

have an educational purpose.

For example, studies that relied purely on statistical data collected outside a maker educational setting or studies that described a maker space design process but did not include any research data from an actual making experience conducted by participants from the K-12 age group were excluded. Studies conducted both in formal and informal settings were included in the review. Also, papers were included regardless of whether they were conducted using qualitative, quantitative, or mixed methods. After the independent screening process, the results were combined, and any conflicting assessments were discussed and settled. Finally, 149 studies were included to be retrieved for further evaluation of eligibility, of which five studies were not available for retrieval. Thus, the screening resulted in 144 included studies with full text retrieved to apply quality criteria and further analysis.

Quality criteria

The quality of each of the remaining 144 studies was assessed against the Critical Appraisal Skills Programme’s ( 2023 ) qualitative study checklist, which was slightly adjusted for the context of this review. The checklist consisted of ten questions that each address one quality criterion:

Was there a clear statement of the aims of the research?.

Are the methodologies used appropriate?.

Was the research design appropriate to address the research aims?.

Was the recruitment strategy appropriate to the aims of the research?.

Was the data collected in a way that addressed the research issue?.

Has the relationship between the researcher and participants been adequately considered?.

Have ethical issues been taken into consideration?.

Was the data analysis sufficiently rigorous?.

Is there a clear statement of findings?.

How valuable is the research?.

The first author assessed the quality by reading each study’s full text. To be included in the final analysis, the study had to meet both the inclusion-exclusion and the quality criteria. In this phase, the final assessment for eligibility, 50 studies were excluded due to not meeting the initial inclusion and exclusion criteria, and 32 studies for not filling the criteria for quality. A total of 62 studies were included in the final analysis of this literature review. The PRISMA flow chart (Haddaway et al., 2022 ; see also Page et al., 2021 ) of the study selection process is presented in Fig.  1 .

PRISMA study selection flow chart (Haddaway et al., 2022 )

Qualitative content analysis of the reviewed studies

The analysis of the studies included in the review was conducted through careful reading of the full texts of the articles by the first author. To answer the first research question: What were the characteristics of the studies in terms of geographical regions, quantity of publications, research settings, and methods; a deductive coding framework was applied that consisted of characterizing factors of the study, its research setting as well as data collection and analysis methods applied. The predetermined categories of the study characteristics and the codes associated with each category are presented in Table  1 . The educational level of the participants was determined by following The International Standard Classification of Education (ISCED) (UNESCO Institute for Statistics, 2012 ). Educational level was chosen instead of an age group as a coding category because, during the first abstract and title screening of the articles, it became evident that the studies describe their participants more often by their educational level than age. The educational levels were converted from national educational systems following the ISCED diagrams (UNESCO Institute for Statistics, 2021 ).

In addition to the deductive coding, the following analysis categories were gathered from the articles through inductive analysis: journal, duration of the project, number of participants, types of research data collected, and specific data analysis methods. Furthermore, the following characteristics of the studies were marked in the data when applicable: if the research was conducted as a case study, usage of control groups, specific focus on minority groups, gifted students, special needs students, or inclusion. Inductive coding and thematic analysis were applied to answer the second research question: what were the research interests and findings of the reviewed studies? The categorization of research interests was then combined with some aspects of the first part of the analysis to reveal further interesting characteristics about the latest developments in the research in maker education.

In the following, the findings of this systematic literature review will be presented for each research question separately.

Characteristics of research in K-12 maker education in the 2020s

Of the studies included in the review, presented in Table  2 and 20 studies were published in 2020, 17 in 2021, 12 in 2022, and 13 in 2023. The slight decline in publications does not necessarily indicate a decline in interest towards maker education but is more likely due to the COVID-19 pandemic that heavily limited hands-on activities and in situ data collection. Compared to the latest wide-scope review on maker education (Papavlasopoulou et al., 2017 ), the number of high-quality studies published yearly appears to be at similar levels to those in the previous reviews. The studies included in the present review were published in 34 different peer-reviewed academic journals, of which 13 published two or more articles.

Regarding the geographic distribution of studies conducted on maker education, the field seems to be becoming more internationally spread. In 2020, the studies mainly published research conducted in either the USA ( n  = 6) or Finland ( n  = 12), whereas in the subsequent years, the studies were distributed more evenly around the world. However, North America and Scandinavia remained the epicenters of research on maker education, conducting over half of the studies published each year.

Most of the reviewed studies used qualitative methods ( n  = 42). Mixed methods were utilized in 13 studies, and quantitative methods in seven. Forty-four studies were described as case studies by their authors, and, on the other hand, a control group was used in four quantitative and two mixed methods studies. The analysis indicated an interesting research shift towards making activities part of formal educational settings instead of informal, extracurricular activities. Of the studies included in this review, 82% ( n  = 51) were conducted exclusively in formal educational settings. This contrasts significantly with the previous literature review by Papavlasopoulou and colleagues ( 2017 ), where most studies were conducted in informal settings. Furthermore, Schad and Jones ( 2020 ) identified only 20 studies between 2000 and 2018 conducted in formal educational settings in K12-education, and Rouse and Rouse ( 2022 ) identified 22 studies in similar settings from 2014 to early 2020. In these reviews, nearly all studies done in formal educational settings were published in the last years of the 2010 decade. Thus, this finding suggests that the change in learning settings started to emerge in the latter half of the 2010s, and in the 2020s, maker education in formal settings has become the prominent focus of research. The need for further research in formal settings was one of the main research gaps identified in previous literature reviews (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ).

In addition to the shift from informal to formal educational settings, the projects studied in the reviewed articles were conducted nearly as often in school and classroom environments ( n  = 26) as in designated makerspaces ( n  = 28). Only seven of the studied projects took place in other locations, such as youth clubs, libraries, or summer camps. One project was conducted entirely in an online learning environment. Most of the studied projects involved children exclusively from primary ( n  = 27) or lower secondary ( n  = 26) education levels. Only three studies were done with students in upper secondary education. Like the previous literature reviews, only a few studies concentrated on children in early childhood education (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ). Three articles reported projects conducted exclusively on early childhood education age groups, and three studies had participants from early childhood education together with children from primary ( n  = 2) or lower secondary education ( n  = 1).

The number of child participants in the studies varied between 1 and 576, and 14 studies also included teachers or other adults in their sample. The number of participating children in relation to the methods used is presented in Fig.  2 . Most of the qualitative studies had less than 100 children in their sample. However, there were three qualitative studies with 100 to 199 child participants (Friend & Mills, 2021 ; Leskinen et al., 2021 ; Riikonen, Kangas, et al., 2020 ) and one study with 576 participating children (Forbes et al., 2021 ). Studies utilizing mixed methods were either conducted with a very large number of child participants or with less than 100 participants, ranging from 4 to 99. Studies using quantitative methods, on the other hand, in most cases had 50–199 participants ( n  = 6). One quantitative study was conducted with 35 child participants (Yin et al., 2020 ). Many studies included participants from non-dominant backgrounds or with special educational needs. However, only two studies focused specifically on youth from non-dominant backgrounds (Brownell, 2020 ; Hsu et al., 2022 ), and three studies focused exclusively on inclusion and students with special needs (Giusti & Bombieri, 2020 ; Martin et al., 2020 ; Sormunen et al., 2020 ). In addition, one study specifically chose gifted students in their sample (Andersen et al., 2022 ).

Child participants in the reviewed studies in relation to the methods used

Slightly over half of the studied projects had only collaborative tasks ( n  = 36), 11 projects involved both collaborative and individual tasks, and in 11 projects, the participants worked on their own individual tasks. Four studies did not specify whether the project was built around collaborative or individual tasks. In most cases, the projects involved both traditional tangible tools and materials as well as digital devices and fabrication technologies ( n  = 54). In five projects, the students worked entirely with digital design and making methods, and in 3 cases, only with traditional tangible materials. Similarly, the outcomes of the project tasks were mainly focused on designing and building artifacts that included both digital and material elements ( n  = 31), or the project included multiple activities and building of several artifacts that were either digital, material, or had both elements ( n  = 17). Eleven projects included digital exploration without an aim to build a design artifact as a preparatory activity, whereas one project was based solely on digital exploration as the making activity. Material artifacts without digital elements were made in seven of the studied projects, and six concentrated solely on digital artifact making.

The duration of the projects varied between two hours (Tisza & Markopoulos, 2021 ) and five years (Keune et al., 2022 ). The number of studies in each categorized project duration range, in relation to the methods used, is presented in Fig.  3 . Over half of the projects lasted between 1 month and one year ( n  = 35), nine were longer, lasting between 1 and 5 years, and 14 were short projects lasting less than one month. Three qualitative studies and one quantitative study did not give any indication of the duration of the project. Most of the projects of qualitative studies took at least one month ( n  = 32), whereas projects in mixed method studies usually were shorter than three months ( n  = 10). On the other hand, quantitative studies usually investigated projects that were either shorter than three months ( n  = 4) or longer than one year ( n  = 2).

Duration of the studied projects in relation to the methods used

A multitude of different types of data was used in the reviewed studies. The data collection methods utilized by at least three reviewed studies are presented in Table  3 . Qualitative studies usually utilized several (2 to 6) different data gathering methods ( n  = 31), and all mixed method studies used more than one type of data (2 to 6). The most common data collection methods in qualitative studies were video data, interviews, and ethnographic observations combined with other data, such as design artifacts, photographs, and student portfolios. In addition to the data types specified in Table  3 , some studies used more unusual data collection methods such as lesson plans (Herro et al., 2021b ), the think-aloud protocol (Friend & Mills, 2021 ; Impedovo & Cederqvist, 2023 ), and social networks (Tenhovirta et al., 2022 ). Eleven qualitative studies used only one type of data, mainly video recordings ( n  = 9). Mixed method studies, on the other hand, relied often on interviews, pre-post measurements, surveys, and video data. In addition to the data types in Table  3 , mixed-method studies utilized biometric measurements (Hsu et al., 2022 ; Lee, 2021 ), lesson plans (Falloon et al., 2022 ), and teacher assessments (Doss & Bloom, 2023 ). In contrast to the qualitative and mixed method studies, all quantitative studies, apart from one (Yin et al., 2020 ), used only one form of research data, either pre-post measurements or surveys.

The findings of the data collection methods are similar to the previous literature review of Papavlasopoulou and colleagues ( 2017 ) regarding the wide variety of data types used in qualitative and mixed-method studies. However, when compared to their findings on specific types of research data used, video recordings have become the most popular way of collecting data in recent years, replacing interviews and ethnographic observations.

Research interests and findings of the reviewed studies

Seven categories of research interests emerged from the inductive coding of the reviewed studies. The categories are presented in Table  4 in relation to the research methods and educational levels of the participating children. Five qualitative studies, four mixed methods studies, and two quantitative studies had research interests from more than one category. Processes, activity, and practices, as well as sociomateriality in maker education, were studied exclusively with qualitative methods, and, on the other hand, nearly all studies on student motivation, interests, attitudes, engagement, and mindset were conducted with mixed or quantitative methods. In the two biggest categories, most of the studies utilized qualitative methods. Studies conducted with mixed or quantitative methods mainly concentrated on two categories: student learning and learning opportunities and student motivation, interests, attitudes, engagement, and mindset. In the following section, the research interests and findings for each category will be presented in detail.

Nearly half of the reviewed studies ( n  = 30) had a research interest in either student learning through making activities in general or learning opportunities provided by such activities. Five qualitative case studies (Giusti & Bombieri, 2020 ; Hachey et al., 2022 ; Hagerman et al., 2022 ; Hartikainen et al., 2023 ; Morado et al., 2021 ) and two mixed method studies (Martin et al., 2020 ; Vuopala et al., 2020 ) investigated the overall educational value of maker education. One of these studies was conducted in early childhood education (Hachey et al., 2022 ), and two in the context of inclusion in primary and lower secondary education (Giusti & Bombieri, 2020 ; Martin et al., 2020 ). They all reported positive findings on the development of children’s identity formation and skills beyond subject-specific competencies, such as creativity, innovation, cultural literacy, and learning skills. The studies conducted in the context of inclusion especially emphasized the potential of maker education in pushing students with special needs to achieve goals exceeding their supposed cognitive abilities (Giusti & Bombieri, 2020 ; Martin et al., 2020 ). Three studies (Forbes et al., 2021 ; Kumpulainen et al., 2020 ; Xiang et al., 2023 ) investigated student learning through the Maker Literacies Framework (Marsh et al., 2018 ). They also reported positive findings on student learning and skill development in early childhood and primary education, especially on the operational dimension of the framework, as well as on the cultural and critical dimensions. These positive results were further confirmed by the reviewed studies that investigated more specific learning opportunities provided by maker education on developing young people’s creativity, innovation skills, design thinking and entrepreneurship (Liu & Li, 2023 ; Timotheou & Ioannou, 2021 ; Weng et al., 2022a , b ), as well as their 21st-century skills (Iwata et al., 2020 ; Tan et al., 2021 ), and critical data literacies and critical thinking (Stornaiuolo, 2020 ; Weng et al., 2022a ).

Studies that investigated subject-specific learning most often focused on STEM subjects or programming and computational thinking. Based on the findings of these studies, maker-centered learning activities are effective but underused (Mørch et al., 2023 ). Furthermore, in early childhood education, such activities may support children taking on the role of a STEM practitioner (Hachey et al., 2022 ) and, on the other hand, provide them access to learning about STEM subjects beyond their grade level, even in upper secondary education (Tofel-Grehl et al., 2021 ; Winters et al., 2022 ). However, two studies (Falloon et al., 2022 ; Forbes et al., 2021 ) highlighted that it cannot be assumed that students naturally learn science and mathematics conceptual knowledge through making. To achieve learning in STEM subjects, especially science and mathematics, teachers need to specifically identify, design, and focus the making tasks on these areas. One study also looked at the effects of the COVID-19 pandemic on STEM disciplines and found the restrictions on the use of common makerspaces and the changes in the technologies used to have been detrimental to student’s learning in these areas (Dúo-Terrón et al., 2022 ).

Only positive findings emerged from the reviewed studies on how digital making activities promote the development of programming and computational thinking skills and practices (Iwata et al., 2020 ; Liu & Li, 2023 ; Yin et al., 2020 ) and understanding of programming methods used in AI and machine learning (Ng et al., 2023 ). Experiences of fun provided by the making activities were also found to enhance further student learning about programming (Tisza & Markopoulos, 2021 ). One study also reported positive results on student learning of academic writing skills (Stewart et al., 2023 ). There were also three studies (Brownell, 2020 ; Greenberg et al., 2020 ; Wargo, 2021 ) that investigated the potential of maker education to promote equity and learning about social justice and injustice, as well as one study that examined learning opportunities on sustainability (Impedovo & Cederqvist, 2023 ). All these studies found making activities and makerspaces to be fertile ground for learning as well as identity and community building around these topics.

The studies with research interests in the second largest category, facilitation and teaching practices ( n  = 13), investigated a multitude of different aspects of this area. The studies on assessment methods highlighted the educational value of process-based portfolios (Fields et al., 2021 ; Riikonen, Kangas et al., 2020 ) and connected portfolios that are digital portfolios aligned with a connected learning framework (Keune et al., 2022 ). On the other hand, Walan and Brink ( 2023 ) concentrated on developing and analyzing the outcomes of a self-assessment tool for maker-centered learning activities designed to promote 21st-century skills. Several research interests emerged from the review related to scaffolding and implementation of maker education in schools. Riikonen, Kangas, and colleagues ( 2020 ) investigated the pedagogical infrastructures of knowledge-creating, maker-centered learning. It emphasized longstanding iterative, socio-material projects, where real-time support and embedded scaffolding are provided to the participants by a multi-disciplinary teacher team and ideally also by peer tutors. Multi-disciplinary collaboration was also emphasized by Pitkänen and colleagues ( 2020 ) in their study on the role of facilitators as educators in Fab Labs. Cross-age peer tutoring was investigated by five studies and found to be highly effective in promoting learning in maker education (Kumpulainen et al., 2020 ; Riikonen, Kangas, et al., 2020 ; Tenhovirta et al., 2022 ; Weng et al., 2022a ; Winters et al., 2022 ). Kajamaa and colleagues ( 2020 ) further highlighted the importance of team teaching and emphasized moving from authoritative interaction with students to collaboration. Sormunen and colleagues’ ( 2020 ) findings on teacher support in an inclusive setting demonstrated how teacher-directed scaffolding and facilitation of student cooperation and reflective discussions are essential in promoting inclusion-related participation, collaboration skills, and student competence building. One study (Andersen et al., 2022 ) took a different approach and investigated the possibilities of automatic scaffolding of making activities through AI. They concluded that automated scaffolding has excellent potential in maker education and went as far as to suggest that a transition should be made to it. One study also recognized the potential of combining making activities with drama education (Walan, 2021 ).

Versatile aspects of different processes, activities, and practices in maker-centered learning projects were studied by 11 qualitative studies included in this review. Two interlinked studies (Davies et al., 2023 ; Riikonen, Seitamaa-Hakkarainen et al., 2020 ) investigated practices and processes related to collaborative invention, making, and knowledge-creation in lower secondary education. Their findings highlighted the multifaceted and iterative nature of such processes as well as the potential of maker education to offer students authentic opportunities for knowledge creation. Sinervo and colleagues ( 2021 ) also investigated the nature of the co-invention processes from the point of view of how children themselves describe and reflect their own processes. Their findings showed how children could recognize different external constraints involved in their design and the importance of iterative ideation processes and testing the ideas through prototyping. Innovation and invention practices were also studied by two other studies in both formal and informal settings with children from the primary level of education (Leskinen et al., 2023 ; Skåland et al., 2020 ). Skåland and colleagues’ ( 2020 ) findings suggest that narrative framing, that is, storytelling with the children, is an especially fruitful approach in a library setting and helps children understand their process of inventing. Similar findings were made in the study on the role of play in early childhood maker education (Fleer, 2022 ), where play enhanced design cognition and related processes and helped young children make sense of design. On the other hand, Leskinen and colleagues ( 2023 ) showed how innovations are jointly practiced in the interaction between students and teachers. They also emphasized the importance of using manifold information sources and material elements in creative innovation processes.

One study (Kajamaa & Kumpulainen, 2020 ) investigated collaborative knowledge practices and how those are mediated in school makerspaces. They identified four types of knowledge practices involved in maker-centered learning activities: orienting, interpreting, concretizing, and expanding knowledge, and how discourse, materials, embodied actions, and the physical space mediate these practices. Their findings also showed that due to the complexity of these practices, students might find maker-centered learning activities difficult. The sophisticated epistemic practices involved in collaborative invention processes were also demonstrated by the findings of Mehto, Riikonen, Hakkarainen, and colleagues ( 2020a ). Other investigators examined how art-based (Lindberg et al., 2020 ), touch-related (Friend & Mills, 2021 ), and information (Li, 2021 ) practices affect and can be incorporated into making. All three studies reported positive findings on the effects of these practices on student learning and, on the other hand, on the further development of the practices themselves.

Research interests related to student motivation, interests, attitudes, engagement, and mindset were studied by eight reviewed articles, all conducted with either mixed (n = 6) or quantitative methods (n = 2). The studies that investigated student motivation and engagement in making activities (Lee, 2021 ; Martin et al., 2020 ; Ng et al., 2023 ; Nikou, 2023 ) highlighted the importance of social interactions and collaboration as highly influential factors in these areas. On the other hand, positive attitudes towards collaboration also developed through these activities (Nguyen et al., 2023 ). Making activities conducted in the context of equity-oriented pedagogy were found to have great potential in sustaining non-dominant youths’, especially girls’, positive attitudes toward science (Hsu et al., 2022 ). On the other hand, a similar potential was not found in the development of interest in STEM subjects with autistic students (Martin et al., 2020 ). Two studies investigated student mindsets in maker-centered learning activities (Doss & Bloom, 2023 ; Vongkulluksn et al., 2021 ). Doss and Bloom ( 2023 ) identified seven different student mindset profiles present in making activities. Over half (56.67%) of the students in their study were found to share the same mindset profile, characterized as: ‘Flexible, Goal-Oriented, Persistent, Optimistic, Humorous, Realistic about Final Product’ (Doss & Bloom, 2023 , p. 4). In turn, Vongkulluksn and colleagues ( 2021 ) investigated the growth mindset trends for students who participated in a makerspace program for two years in an elementary school. Their findings revealed positive results of how makerspace environments can potentially improve students’ growth mindset.

Six studies included in this review analyzed collaboration within making activities. Students were found to be supportive and respectful towards each other as well as recognize and draw on each other’s expertise (Giusti & Bombieri, 2020 ; Herro et al., 2021a , b ). The making activities and outcomes were found to act as mediators in promoting mutual recognition between students with varying cognitive capabilities and special needs in inclusive settings (Herro et al., 2021a ). Furthermore, a community of interest that emerges through collaborative making activities was also found to be effective in supporting interest development and sustainability (Tan et al., 2021 ). Students were observed to divide work and share roles during their team projects, usually based on students’ interests, expertise, and skills (Herro et al., 2021a , b ). The findings of Stewart et al.‘s ( 2023 ) study suggested that when roles are preassigned to the team members by teachers, it decreases student stress in maker activities. However, if dominating leadership roles emerged in a team, that was found to lead to less advanced forms of collaboration than shared leadership within the team (Leskinen et al., 2021 ).

Sociomaterial aspects of making activities were in the interest of three reviewed studies (Kumpulainen & Kajamaa, 2020 ; Mehto et al., 2020a ; Mehto et al., 2020b ). Materials were shown to have an active role in knowledge-creation and ideation in open-ended maker-centered learning (Mehto et al., 2020a ), which allows for thinking together with the materials (Mehto et al., 2020b ). The task-related physical materials act as a focal point for team collaboration and invite participation (Mehto et al., 2020b ). Furthermore, a study by Kumpulainen and Kajamaa ( 2020 ) emphasized the sociomaterial dynamics of agency, where agency flows in any combination between students, teachers, and materials. However, the singularity or multiplicity of the materials potentially affects the opportunities for access and control of the process (Mehto et al., 2020b ).

In addition to empirical research interests, five studies focused on developing research methods for measuring and analyzing different aspects of maker education. Biometric measurements were investigated as a potential data source to detect engagement in making activities (Lee, 2021 ). Yin and colleagues ( 2020 ) focused on developing instruments for the quantitative measurement of computational thinking skills. On the other hand, Timotheou and Ioannou ( 2021 ) designed and tested an analytic framework and coding scheme to analyze learning and innovation skills from qualitative interviews and video data. Artificial intelligence as a potential, partially automated tool for analyzing CSCL artifacts was also investigated by one study (Andersen et al., 2022 ). Finally, Riikonen, Seitamaa-Hakkarainen, and colleagues ( 2020 ) developed visual video data analysis methods for investigating collaborative design and making activities.

Slightly over half of the reviewed studies ( n  = 33) made clear suggestions for future research. Expectedly, these studies suggested further investigation of their own research interests. However, across the studies, five themes of recommendations for future research interests and designs emerged from the data:

1. Studies conducted with diverse range of participants , pedagogical designs , and contexts (Hartikainen et al., 2023 ; Kumpulainen & Kajamaa, 2020 ; Leskinen et al., 2023 ; Lindberg et al., 2020 ; Liu & Li, 2023 Martin et al., 2020 ; Mehto et al., 2020b ; Nguyen et al., 2023 ; Sormunen et al., 2020 ; Tan et al., 2021 ; Weng et al., 2022a , b ; Yin et al., 2020 ).

2. Longitudinal studies to confirm the existing research findings, further develop pedagogical approaches to making, and to better understand the effects of maker education on students later in their lives (Davies et al., 2023 ; Fields et al., 2021 ; Kumpulainen et al., 2020 ; Kumpulainen & Kajamaa, 2020 ; Stornaiuolo, 2020 ; Tisza & Markopoulos, 2021 ; Walan & Brink, 2023 ; Weng et al., 2022a ).

3. Development of new methods and applying existing methods in different conditions (Doss & Bloom, 2023 ; Kumpulainen et al., 2020 ; Leskinen et al., 2021 ; Mehto et al., 2020b ; Mørch et al., 2023 ; Tan et al., 2021 ; Timotheou & Ioannou, 2021 ; Tisza & Markopoulos, 2021 ).

4. Identifying optimal conditions and practices for learning, skill, and identity development through making (Davies et al., 2023 ; Fields et al., 2021 ; Hartikainen et al., 2023 ; Tofel-Grehl et al., 2021 ).

5. Collaboration from the perspectives of how it affects processes and outcomes of making activities and, on the other hand, how such activities affect collaboration (Pitkänen et al., 2020 ; Tisza & Markopoulos, 2021 ; Weng et al., 2022a ).

Discussion and conclusions

This systematic literature review was conducted to describe the development of research on maker education in the early 2020s. Sixty-two studies from the initial 700 studies identified from the three major educational research databases were included in the review. The qualitative analysis of the reviewed studies revealed some interesting developments in the field. Overall, the research on maker education appears to be active. Maker education seems to be attracting interest from researchers around the globe. However, two epicenters of research, North America and Scandinavia, namely Finland, appear to have an active role in maker research.

Most studies relied on rich qualitative data, often collected using several methods. Video recordings have become a popular way to collect data in maker education research. Although qualitative methods remained the dominant methodological approach in the field (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ), mixed and quantitative methods were used in nearly a third of the reviewed studies. These studies mainly measured learning outcomes or participants’ motivation, interests, attitudes, engagement, and mindsets. There was a great variety in the duration of the maker projects and the number of participants. The projects lasted from less than a day up to five years, and the number of participants varied similarly from one to nearly six hundred. Methodological development was also within the research interests of several studies in this review. Developments were made both in qualitative and quantitative methodologies. Such methodological development was one of the research gaps identified in the previous literature reviews (e.g., Schad & Jones, 2020 ).

The analysis of the reviewed studies revealed an interesting shift in research on maker education from informal settings to formal education. Our review revealed that most studies were conducted exclusively in formal education and often as part of the curricular activity. The need for this development was called for in the previous literature reviews (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ). However, only a handful of studies were conducted in early childhood education. Winters and colleagues’ ( 2022 ) study adopted a very interesting setting where children from early childhood education worked together and were mentored by students from lower secondary education. This type of research setting could have great potential for future research in maker education.

Another research gap identified in the previous literature reviews was the need to study and measure a wide variety of potential learning opportunities and outcomes of maker education (Lin et al., 2020 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ). The analysis revealed that new research in the field is actively filling this gap. Skills that go beyond subject-specific content and the development of participants’ identities through making activities were especially actively studied from various perspectives. The findings of these studies were distinctively positive, corresponding with the conclusions of the previous literature reviews (e.g., Papavlasopoulou et al., 2017 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ). This potential of maker education should be recognized by educators and policymakers, especially when the advancements in AI technologies will forefront the need for the humane skills of working creatively with knowledge and different ways of knowing, empathic engagement, and collaboration (e.g., Liu et al., 2024 ; Markauskaite et al., 2022 ; Qadir, 2023 ; World Economic Forum, 2023 ). Some of these studies also addressed the issue of promoting equity through maker education, which was called for in the previous literature review (Rouse & Rouse, 2022 ; Vossoughi & Bevan, 2014 ). However, considering the small number of these studies, more research will still be needed.

The two other popular research interest categories that emerged from the analysis were facilitation and teaching practices as well as processes, activities, and practices involved in making – both identified as research gaps in the previous literature reviews (Iivari et al., 2016 ; Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ). The teaching practices and scaffolding of making activities were investigated from different aspects, such as assessment methods, implementation of maker education in schools, and cross-age peer tutoring. The results of these studies highlighted the positive effects of multi-disciplinary collaboration and peer tutoring. Such pedagogical approaches should be more widely promoted as integral parts of the pedagogical infrastructure in schools. However, this calls for measures from policymakers and school authorities to enable such collaborative ways of teaching that extend beyond the traditional structures of school organizations. Furthermore, although research on this area has been active and multi-faceted, the facilitation of maker education in inclusive settings especially calls for further investigation. In terms of processes, practices, and activities involved in making, the reviewed studies investigated a variety of aspects that revealed the sophisticated epistemic practices involved and the importance of concrete making, prototyping, and iterative ideation in maker-centered learning activities. These studies further highlighted the potential of maker education to offer students authentic opportunities for knowledge creation. Studies also examined collaboration and sociomateriality involved in maker education. Especially sociomateriality is a relatively new, emerging area of research in maker education.

The reviewed studies identified five research gaps that require further investigation: (1) conducting studies with a diverse range of participants, pedagogical designs, and contexts; (2) carrying out longitudinal studies; (3) developing new methods and applying existing methods in different settings; (4) identifying the most effective conditions and practices for learning, skill development and identity formation in maker education, and (5) understanding how collaboration affects the processes and outcomes of making activities and vice versa. In addition to the research gaps identified by reviewed studies, the analysis revealed additional gaps. Studies conducted in early childhood education and inclusive settings remain especially under-represented, although maker pedagogies have been found to have great potential in these areas. Similarly, many researchers have recognized the potential of maker education to promote equality between children from different backgrounds and genders. Still, only a handful of studies investigated these issues. Thus, more research is needed, especially on best practices and pedagogical approaches in this area. Furthermore, the processes involved in and affecting maker-centered learning call for further investigations.

The field has matured based on the analysis of the reviewed studies. It is moving from striving to understand what can be achieved to investigating the underlying conditions behind learning through making, how desired outcomes can be best achieved, as well as how the processes involved in making unfold, what the effects are in the long run, and how to understand best and measure different phenomena related to making. Furthermore, researchers are looking into more and more opportunities to expand the learning opportunities of maker education by combining them with other creative pedagogies and applying them to projects that seek to introduce subject-specific content beyond STEM to students.

This systematic literature review has several limitations. The typical limitations of most review studies, the potential loss of search results due to limited search terms and databases used, apply to this review. For example, more culturally diverse search results might have been reached with the addition of other databases and further search terms. However, the search string was carefully designed and tested to include as many common terms used in maker education research as possible, including possible variations. Furthermore, the three databases used in the search, Scopus, ERIC, and EBSCO, are regarded as the most comprehensive databases of educational research available. Thus, although some studies might not have been identified because of these limitations, it can be assumed that this review gives a comprehensive enough snapshot of research on maker education in the early years of the 2020s.

Andersen, R., Mørch, A. I., & Litherland, K. T. (2022). Collaborative learning with block-based programming: Investigating human-centered artificial intelligence in education. Behaviour & Information Technology , 41 (9), 1830–1847. https://doi.org/10.1080/0144929X.2022.2083981

Article   Google Scholar  

Blikstein, P. (2013). Digital fabrication and ‘making’ in education: The democratization of invention. In C. Büching & J. Walter-Herrmann (Eds.), FabLabs: Of machines, makers and inventors (pp. 203–222). Transcript Publishers. https://doi.org/10.1515/transcript.9783839423820.203

Brownell, C. J. (2020). Keep walls down instead of up: Interrogating writing/making as a vehicle for black girls’ literacies. Education Sciences , 10 (6), 159. https://doi.org/10.3390/educsci10060159

Chawla, L., & Heft, H. (2002). Children’s competence and the ecology of communities: A functional approach to the evaluation of participation. Journal of Environmental Psychology , 22 (1–2), 201–216. https://doi.org/10.1006/jevp.2002.0244

Critical Appraisal Skills Programme (2023). CASP Qualitative Studies Checklist . https://casp-uk.net/checklists/casp-qualitative-studies-checklist-fillable.pdf

Davies, S., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2023). Idea generation and knowledge creation through maker practices in an artifact-mediated collaborative invention project. Learning, Culture and Social Interaction, 39 , 100692. https://doi.org/10.1016/j.lcsi.2023.100692

Doss, K., & Bloom, L. (2023). Mindset and the desire for feedback during creative tasks. Journal of Creativity , 33 (1), 100047. https://doi.org/10.1016/j.yjoc.2023.100047

Dúo-Terrón, P., Hinojo-Lucena, F. J., Moreno-Guerrero, A. J., & López-Belmonte, J. (2022). Impact of the pandemic on STEAM disciplines in the sixth grade of primary education. European Journal of Investigation in Health Psychology and Education , 12 (8), 989–1005. https://doi.org/10.3390/ejihpe12080071

Falloon, G., Forbes, A., Stevenson, M., Bower, M., & Hatzigianni, M. (2022). STEM in the making? Investigating STEM learning in junior school makerspaces. Research in Science Education , 52 (2), 511–537. https://doi.org/10.1007/s11165-020-09949-3

Fields, D. A., Lui, D., Kafai, Y. B., Jayathirtha, G., Walker, J., & Shaw, M. (2021). Communicating about computational thinking: Understanding affordances of portfolios for assessing high school students’ computational thinking and participation practices. Computer Science Education , 31 (2), 224–258. https://doi.org/10.1080/08993408.2020.1866933

Fleer, M. (2022). The genesis of design: Learning about design, learning through design to learning design in play. International Journal of Technology and Design Education , 32 (3), 1441–1468. https://doi.org/10.1007/s10798-021-09670-w

Forbes, A., Falloon, G., Stevenson, M., Hatzigianni, M., & Bower, M. (2021). An analysis of the nature of young students’ STEM learning in 3D technology-enhanced makerspaces. Early Education and Development , 32 (1), 172–187. https://doi.org/10.1080/10409289.2020.1781325

Friend, L., & Mills, K. A. (2021). Towards a typology of touch in multisensory makerspaces. Learning Media and Technology , 46 (4), 465–482. https://doi.org/10.1080/17439884.2021.1928695

Giusti, T., & Bombieri, L. (2020). Learning inclusion through makerspace: A curriculum approach in Italy to share powerful ideas in a meaningful context. The International Journal of Information and Learning Technology , 37 (3), 73–86. https://doi.org/10.1108/IJILT-10-2019-0095

Greenberg, D., Calabrese Barton, A., Tan, E., & Archer, L. (2020). Redefining entrepreneurialism in the maker movement: A critical youth approach. Journal of the Learning Sciences , 29 (4–5), 471–510. https://doi.org/10.1080/10508406.2020.1749633

Hachey, A. C., An, S. A., & Golding, D. E. (2022). Nurturing kindergarteners’ early STEM academic identity through makerspace pedagogy. Early Childhood Education Journal , 50 (3), 469–479. https://doi.org/10.1007/s10643-021-01154-9

Haddaway, N. R., Page, M. J., Pritchard, C. C., & McGuinness, L. A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and open synthesis. Campbell Systematic Reviews , 18 (2). https://doi.org/10.1002/cl2.1230

Hagerman, M. S., Cotnam-Kappel, M., Turner, J. A., & Hughes, J. M. (2022). Literacies in the making: Exploring elementary students’ digital-physical meaning-making practices while crafting musical instruments from recycled materials. Technology Pedagogy and Education , 31 (1), 63–84. https://doi.org/10.1080/1475939X.2021.1997794

Hartikainen, H., Ventä-Olkkonen, L., Kinnula, M., & Iivari, N. (2023). We were proud of our idea: How teens and teachers gained value in an entrepreneurship and making project. International Journal of Child-Computer Interaction , 35 , 100552. https://doi.org/10.1016/j.ijcci.2022.100552

Herro, D., Quigley, C., & Abimbade, O. (2021a). Assessing elementary students’ collaborative problem-solving in makerspace activities. Information and Learning Sciences , 122 (11/12), 774–794. https://doi.org/10.1108/ILS-08-2020-0176

Herro, D., Quigley, C., Plank, H., & Abimbade, O. (2021b). Understanding students’ social interactions during making activities designed to promote computational thinking. The Journal of Educational Research , 114 (2), 183–195. https://doi.org/10.1080/00220671.2021.1884824

Hsu, P. S., Lee, E. M., & Smith, T. J. (2022). Exploring the influence of equity-oriented pedagogy on non-dominant youths’ attitudes toward science through making. RMLE Online , 45 (8), 1–16. https://doi.org/10.1080/19404476.2022.2116668

Iivari, N., Molin-Juustila, T., & Kinnula, M. (2016). The future digital innovators: Empowering the young generation with digital fabrication and making completed research paper. Proceedings of the 37th International Conference on Information Systems, ICIS 2016. 2 .

Impedovo, M., & Cederqvist, A. M. (2023). Socio-(im)material-making activities in minecraft: Retracing digital literacy applied to ESD. Research in Science & Technological Education , 1–21. https://doi.org/10.1080/02635143.2023.2245355

International Organization for Standardization (2020). ISO 3166-2:2020 - Codes for the representation of names of countries and their subdivisions — Part 2: Country subdivision code . https://www.iso.org/standard/72483.html

Iwata, M., Pitkänen, K., Laru, J., & Mäkitalo, K. (2020). Exploring potentials and challenges to develop twenty-first century skills and computational thinking in K-12 maker education. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00087

Kafai, Y. B. (1996). Learning through artifacts: Communities of practice in classrooms. AI and Society , 10 (1), 89–100. https://doi.org/10.1007/BF02716758

Kafai, Y. B., & Peppler, K. A. (2011). Youth, technology, and DIY: Developing participatory competencies in creative media production. Review of Research in Education , 35 (1), 89–119. https://doi.org/10.3102/0091732X10383211

Kafai, Y., Fields, D. A., & Searle, K. (2014). Electronic textiles as disruptive designs: Supporting and challenging maker activities in schools. Harvard Educational Review , 84 (4), 532–556. https://doi.org/10.17763/haer.84.4.46m7372370214783

Kajamaa, A., & Kumpulainen, K. (2020). Students’ multimodal knowledge practices in a makerspace learning environment. International Journal of Computer-Supported Collaborative Learning , 15 (4), 411–444. https://doi.org/10.1007/s11412-020-09337-z

Kajamaa, A., Kumpulainen, K., & Olkinuora, H. (2020). Teacher interventions in students’ collaborative work in a technology-rich educational makerspace. British Journal of Educational Technology , 51 (2), 371–386. https://doi.org/10.1111/bjet.12837

Kangas, K., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2013). Figuring the world of designing: Expert participation in elementary classroom. International Journal of Technology and Design Education, 23 (2), 425–442. https://doi.org/10.1007/s10798-011-9187-z

Keune, A., Peppler, K., & Dahn, M. (2022). Connected portfolios: Open assessment practices for maker communities. Information and Learning Sciences , 123 (7/8), 462–481. https://doi.org/10.1108/ILS-03-2022-0029

Koh, J. H. L., Chai, C. S., Wong, B., & Hong, H. Y. (2015). Design thinking for education: Conceptions and applications in teaching and learning . Springer. https://doi.org/10.1007/978-981-287-444-3

Kumpulainen, K., & Kajamaa, A. (2020). Sociomaterial movements of students’ engagement in a school’s makerspace. British Journal of Educational Technology , 51 (4), 1292–1307. https://doi.org/10.1111/bjet.12932

Kumpulainen, K., Kajamaa, A., Leskinen, J., Byman, J., & Renlund, J. (2020). Mapping digital competence: Students’ maker literacies in a school’s makerspace. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00069

Lee, V. R. (2021). Youth engagement during making: Using electrodermal activity data and first-person video to generate evidence-based conjectures. Information and Learning Sciences , 122 (3/4), 270–291. https://doi.org/10.1108/ILS-08-2020-0178

Leskinen, J., Kajamaa, A., & Kumpulainen, K. (2023). Learning to innovate: Students and teachers constructing collective innovation practices in a primary school’s makerspace. Frontiers in Education , 7 . https://doi.org/10.3389/feduc.2022.936724

Leskinen, J., Kumpulainen, K., Kajamaa, A., & Rajala, A. (2021). The emergence of leadership in students’ group interaction in a school-based makerspace. European Journal of Psychology of Education , 36 (4), 1033–1053. https://doi.org/10.1007/s10212-020-00509-x

Lindberg, L., Fields, D. A., & Kafai, Y. B. (2020). STEAM maker education: Conceal/reveal of personal, artistic and computational dimensions in high school student projects. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00051

Lin, Q., Yin, Y., Tang, X., Hadad, R., & Zhai, X. (2020). Assessing learning in technology-rich maker activities: A systematic review of empirical research. Computers and Education , 157 . https://doi.org/10.1016/j.compedu.2020.103944

Liu, S., & Li, C. (2023). Promoting design thinking and creativity by making: A quasi-experiment in the information technology course. Thinking Skills and Creativity , 49 , 101335. https://doi.org/10.1016/j.tsc.2023.101335

Liu, W., Fu, Z., Zhu, Y., Li, Y., Sun, Y., Hong, X., Li, Y., & Liu, M. (2024). Co-making the future: Crafting tomorrow with insights and perspectives from the China-U.S. young maker competition. International Journal of Technology and Design Education . https://doi.org/10.1007/s10798-024-09887-5

Li, X. (2021). Young people’s information practices in library makerspaces. Journal of the Association for Information Science and Technology , 72 (6), 744–758. https://doi.org/10.1002/asi.24442

Markauskaite, L., Marrone, R., Poquet, O., Knight, S., Martinez-Maldonado, R., Howard, S., Tondeur, J., De Laat, M., Buckingham Shum, S., Gašević, D., & Siemens, G. (2022). Rethinking the entwinement between artificial intelligence and human learning: What capabilities do learners need for a world with AI? Computers and Education: Artificial Intelligence , 3 . https://doi.org/10.1016/j.caeai.2022.100056

Marsh, J., Arnseth, H., & Kumpulainen, K. (2018). Maker literacies and maker citizenship in the MakEY (makerspaces in the early years) project. Multimodal Technologies and Interaction , 2 (3), 50. https://doi.org/10.3390/mti2030050

Martin, L. (2015). The promise of the maker movement for education. Journal of Pre-College Engineering Education Research , 5 (1), 30–39. https://doi.org/10.7771/2157-9288.1099

Martin, W. B., Yu, J., Wei, X., Vidiksis, R., Patten, K. K., & Riccio, A. (2020). Promoting science, technology, and engineering self-efficacy and knowledge for all with an autism inclusion maker program. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00075

Mehto, V., Riikonen, S., Hakkarainen, K., Kangas, K., & Seitamaa‐Hakkarainen, P. (2020a). Epistemic roles of materiality within a collaborative invention project at a secondary school. British Journal of Educational Technology, 51 (4), 1246–1261. https://doi.org/10.1111/bjet.12942

Mehto, V., Riikonen, S., Kangas, K., & Seitamaa-Hakkarainen, P. (2020b). Sociomateriality of collaboration within a small team in secondary school maker-centered learning project. International Journal of Child-Computer Interaction , 26. https://doi.org/10.1016/j.ijcci.2020.100209

Morado, M. F., Melo, A. E., & Jarman, A. (2021). Learning by making: A framework to revisit practices in a constructionist learning environment. British Journal of Educational Technology , 52 (3), 1093–1115. https://doi.org/10.1111/bjet.13083

Mørch, A. I., Flø, E. E., Litherland, K. T., & Andersen, R. (2023). Makerspace activities in a school setting: Top-down and bottom-up approaches for teachers to leverage pupils’ making in science education. Learning Culture and Social Interaction , 39 , 100697. https://doi.org/10.1016/j.lcsi.2023.100697

Ng, D. T. K., Su, J., & Chu, S. K. W. (2023). Fostering secondary school students’ AI literacy through making AI-driven recycling bins. Education and Information Technologies , 1–32. https://doi.org/10.1007/s10639-023-12183-9

Nguyen, H. B. N., Hong, J. C., Chen, M. L., Ye, J. N., & Tsai, C. R. (2023). Relationship between students’ hands-on making self-efficacy, perceived value, cooperative attitude and competition preparedness in joining an iSTEAM contest. Research in Science & Technological Education , 41 (1), 251–270. https://doi.org/10.1080/02635143.2021.1895100

Nikou, S. A. (2023). Student motivation and engagement in maker activities under the lens of the activity theory: A case study in a primary school. Journal of Computers in Education . https://doi.org/10.1007/s40692-023-00258-y

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan – a web and mobile app for systematic reviews. Systematic Reviews , 5 (1), 210. https://doi.org/10.1186/s13643-016-0384-4

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Systematic Reviews , 10 (1), 89. https://doi.org/10.1186/s13643-021-01626-4

Papavlasopoulou, S., Giannakos, M. N., & Jaccheri, L. (2017). Empirical studies on the Maker Movement, a promising approach to learning: A literature review. Entertainment Computing , 18 , 57–78. https://doi.org/10.1016/j.entcom.2016.09.002

Papert, S. (1980). Mindstroms: Children, computers, and powerful ideas . Basic Books.

Pitkänen, K., Iwata, M., & Laru, J. (2020). Exploring technology-oriented fab lab facilitators’ role as educators in K-12 education: Focus on scaffolding novice students’ learning in digital fabrication activities. International Journal of Child-Computer Interaction , 26 , 100207. https://doi.org/10.1016/j.ijcci.2020.100207

Qadir, J. (2023). Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. IEEE Global Engineering Education Conference, EDUCON , 2023-May . https://doi.org/10.1109/EDUCON54358.2023.10125121

Resnick, M. (2017). Lifelong kindergarten: Cultivating creativity through projects, passions, peers, and play . MIT Press.

Riikonen, S., Kangas, K., Kokko, S., Korhonen, T., Hakkarainen, K., & Seitamaa-Hakkarainen, P. (2020). The development of pedagogical infrastructures in three cycles of maker-centered learning projects. Design and Technology Education: An International Journal, 25 (2), 29–49.

Riikonen, S., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2020). Bringing maker practices to school: Tracing discursive and materially mediated aspects of student teams’ collaborative making processes. International Journal of Computer-Supported Collaborative Learning, 15 (3), 319–349. https://doi.org/10.1007/s11412-020-09330-6

Rouse, R., & Rouse, A. G. (2022). Taking the maker movement to school: A systematic review of preK-12 school-based makerspace research. Educational Research Review , 35 . Elsevier Ltd. https://doi.org/10.1016/j.edurev.2021.100413

Schad, M., & Jones, W. M. (2020). The maker movement and education: A systematic review of the literature. Journal of Research on Technology in Education , 52 (1), 65–78. https://doi.org/10.1080/15391523.2019.1688739

Sinervo, S., Sormunen, K., Kangas, K., Hakkarainen, K., Lavonen, J., Juuti, K., Korhonen, T., & Seitamaa-Hakkarainen, P. (2021). Elementary school pupils’ co-inventions: Products and pupils’ reflections on processes. International Journal of Technology and Design Education, 31 (4), 653–676. https://doi.org/10.1007/s10798-020-09577-y

Skåland, G., Arnseth, H. C., & Pierroux, P. (2020). Doing inventing in the library. Analyzing the narrative framing of making in a public library context. Education Sciences , 10 (6), 158. https://doi.org/10.3390/educsci10060158

Sormunen, K., Juuti, K., & Lavonen, J. (2020). Maker-centered project-based learning in inclusive classes: Supporting students’ active participation with teacher-directed reflective discussions. International Journal of Science and Mathematics Education , 18 (4), 691–712. https://doi.org/10.1007/s10763-019-09998-9

Stewart, A., Yuan, J., Kale, U., Valentine, K., & McCartney, M. (2023). Maker activities and academic writing in a middle school science classroom. International Journal of Instruction , 16 (2), 125–144. https://doi.org/10.29333/iji.2023.1628a

Stornaiuolo, A. (2020). Authoring data stories in a media makerspace: Adolescents developing critical data literacies. Journal of the Learning Sciences , 29 (1), 81–103. https://doi.org/10.1080/10508406.2019.1689365

Tan, A. L., Jamaludin, A., & Hung, D. (2021). In pursuit of learning in an informal space: A case study in the Singapore context. International Journal of Technology and Design Education , 31 (2), 281–303. https://doi.org/10.1007/s10798-019-09553-1

Tenhovirta, S., Korhonen, T., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2022). Cross-age peer tutoring in a technology-enhanced STEAM project at a lower secondary school. International Journal of Technology and Design Education, 32 (3), 1701–1723. https://doi.org/10.1007/s10798-021-09674-6

Timotheou, S., & Ioannou, A. (2021). Learning and innovation skills in making contexts: A comprehensive analytical framework and coding scheme. Educational Technology Research and Development , 69 (6), 3179–3207. https://doi.org/10.1007/s11423-021-10067-8

Tisza, G., & Markopoulos, P. (2021). Understanding the role of fun in learning to code. International Journal of Child-Computer Interaction , 28 , 100270. https://doi.org/10.1016/j.ijcci.2021.100270

Tofel-Grehl, C., Ball, D., & Searle, K. (2021). Making progress: Engaging maker education in science classrooms to develop a novel instructional metaphor for teaching electric potential. The Journal of Educational Research , 114 (2), 119–129. https://doi.org/10.1080/00220671.2020.1838410

Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012). Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Medical Research Methodology , 12 (1), 181. https://doi.org/10.1186/1471-2288-12-181

UNESCO Institute for Statistics (2012). International standard classification of education: ISCED 2011 . https://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-isced-2011-en.pdf

UNESCO Institute for Statistics (2021). Using ISCED Diagrams to Compare Education Systems . https://neqmap.bangkok.unesco.org/wp-content/uploads/2021/06/UIS-ISCED-DiagramsCompare-web.pdf

Vongkulluksn, V. W., Matewos, A. M., & Sinatra, G. M. (2021). Growth mindset development in design-based makerspace: A longitudinal study. The Journal of Educational Research , 114 (2), 139–154. https://doi.org/10.1080/00220671.2021.1872473

Vossoughi, S., & Bevan, B. (2014). Making and tinkering: A review of the literature. National Research Council Committee on Out of School Time STEM , 67 , 1–55.

Google Scholar  

Vuopala, E., Guzmán Medrano, D., Aljabaly, M., Hietavirta, D., Malacara, L., & Pan, C. (2020). Implementing a maker culture in elementary school – students’ perspectives. Technology Pedagogy and Education , 29 (5), 649–664. https://doi.org/10.1080/1475939X.2020.1796776

Walan, S. (2021). The dream performance – a case study of young girls’ development of interest in STEM and 21st century skills, when activities in a makerspace were combined with drama. Research in Science & Technological Education , 39 (1), 23–43. https://doi.org/10.1080/02635143.2019.1647157

Walan, S., & Brink, H. (2023). Students’ and teachers’ responses to use of a digital self-assessment tool to understand and identify development of twenty-first century skills when working with makerspace activities. International Journal of Technology and Design Education . https://doi.org/10.1007/s10798-023-09845-7

Wargo, J. M. (2021). Sound civics, heard histories: A critical case of young children mobilizing digital media to write (right) injustice. Theory & Research in Social Education , 49 (3), 360–389. https://doi.org/10.1080/00933104.2021.1874582

Weng, X., Chiu, T. K. F., & Jong, M. S. Y. (2022a). Applying relatedness to explain learning outcomes of STEM maker activities. Frontiers in Psychology , 12 . https://doi.org/10.3389/fpsyg.2021.800569

Weng, X., Chiu, T. K. F., & Tsang, C. C. (2022b). Promoting student creativity and entrepreneurship through real-world problem-based maker education. Thinking Skills and Creativity , 45 , 101046. https://doi.org/10.1016/j.tsc.2022.101046

Winters, K. L., Gallagher, T. L., & Potts, D. (2022). Creativity, collaboration, and cross-age mentorships using STEM-infused texts. Elementary STEM Journal , 27 (2), 7–14.

World Economic Forum (2023). The future of jobs report 2023 . https://www.weforum.org/reports/the-future-of-jobs-report-2023/

Xiang, S., Yang, W., & Yeter, I. H. (2023). Making a makerspace for children: A mixed-methods study in Chinese kindergartens. International Journal of Child-Computer Interaction , 36 , 100583. https://doi.org/10.1016/j.ijcci.2023.100583

Yin, Y., Hadad, R., Tang, X., & Lin, Q. (2020). Improving and assessing computational thinking in maker activities: The integration with physics and engineering learning. Journal of Science Education and Technology , 29 (2), 189–214. https://doi.org/10.1007/s10956-019-09794-8

Yulis San Juan, A., & Murai, Y. (2022). Turning frustration into learning opportunities during maker activities: A review of literature. International Journal of Child-Computer Interaction , 33 , 100519. https://doi.org/10.1016/j.ijcci.2022.100519

Download references

Open Access funding provided by University of Helsinki (including Helsinki University Central Hospital). This work has been funded by the Strategic Research Council (SRC) established within the Research Council of Finland, grants #312527, #352859, and # 352971.

Open Access funding provided by University of Helsinki (including Helsinki University Central Hospital).

Author information

Authors and affiliations.

Faculty of Educational Sciences, University of Helsinki, Helsinki, Finland

Sini Davies & Pirita Seitamaa-Hakkarainen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sini Davies .

Ethics declarations

Conflict of interest.

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Davies, S., Seitamaa-Hakkarainen, P. Research on K-12 maker education in the early 2020s – a systematic literature review. Int J Technol Des Educ (2024). https://doi.org/10.1007/s10798-024-09921-6

Download citation

Accepted : 02 July 2024

Published : 27 August 2024

DOI : https://doi.org/10.1007/s10798-024-09921-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Maker education
• K-12 education
• Systematic literature review
• Maker-centered learning
• Maker culture
• Design and making
• Find a journal
• Publish with us
• Track your research