features of a good research work

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Matthew DeCarlo

Chapter Outline

• Empirical vs. ethical questions (4 minute read)
• Characteristics of a good research question (4 minute read)
• Quantitative research questions (7 minute read)
• Qualitative research questions (3 minute read)
• Evaluating and updating your research questions (4 minute read)

Content warning: examples in this chapter include references to sexual violence, sexism, substance use disorders, homelessness, domestic violence, the child welfare system, cissexism and heterosexism, and truancy and school discipline.

9.1 Empirical vs. ethical questions

Learning objectives.

Learners will be able to…

• Define empirical questions and provide an example
• Define ethical questions and provide an example

Writing a good research question is an art and a science. It is a science because you have to make sure it is clear, concise, and well-developed. It is an art because often your language needs “wordsmithing” to perfect and clarify the meaning. This is an exciting part of the research process; however, it can also be one of the most stressful.

Creating a good research question begins by identifying a topic you are interested in studying. At this point, you already have a working question. You’ve been applying it to the exercises in each chapter, and after reading more about your topic in the scholarly literature, you’ve probably gone back and revised your working question a few times. We’re going to continue that process in more detail in this chapter. Keep in mind that writing research questions is an iterative process, with revisions happening week after week until you are ready to start your project.

Empirical vs. ethical questions

When it comes to research questions, social science is best equipped to answer empirical questions —those that can be answered by real experience in the real world—as opposed to  ethical questions —questions about which people have moral opinions and that may not be answerable in reference to the real world. While social workers have explicit ethical obligations (e.g., service, social justice), research projects ask empirical questions to help actualize and support the work of upholding those ethical principles.

In order to help you better understand the difference between ethical and empirical questions, let’s consider a topic about which people have moral opinions. How about SpongeBob SquarePants? [1] In early 2005, members of the conservative Christian group Focus on the Family (2005) [2] denounced this seemingly innocuous cartoon character as “morally offensive” because they perceived his character to be one that promotes a “pro-gay agenda.” Focus on the Family supported their claim that SpongeBob is immoral by citing his appearance in a children’s video designed to promote tolerance of all family forms (BBC News, 2005). [3] They also cited SpongeBob’s regular hand-holding with his male sidekick Patrick as further evidence of his immorality.

So, can we now conclude that SpongeBob SquarePants is immoral? Not so fast. While your mother or a newspaper or television reporter may provide an answer, a social science researcher cannot. Questions of morality are ethical, not empirical. Of course, this doesn’t mean that social science researchers cannot study opinions about or social meanings surrounding SpongeBob SquarePants (Carter, 2010). [4] We study humans after all, and as you will discover in the following chapters of this textbook, we are trained to utilize a variety of scientific data-collection techniques to understand patterns of human beliefs and behaviors. Using these techniques, we could find out how many people in the United States find SpongeBob morally reprehensible, but we could never learn, empirically, whether SpongeBob is in fact morally reprehensible.

Let’s consider an example from a recent MSW research class I taught. A student group wanted to research the penalties for sexual assault. Their original research question was: “How can prison sentences for sexual assault be so much lower than the penalty for drug possession?” Outside of the research context, that is a darn good question! It speaks to how the War on Drugs and the patriarchy have distorted the criminal justice system towards policing of drug crimes over gender-based violence.

Unfortunately, it is an ethical question, not an empirical one. To answer that question, you would have to draw on philosophy and morality, answering what it is about human nature and society that allows such unjust outcomes. However, you could not answer that question by gathering data about people in the real world. If I asked people that question, they would likely give me their opinions about drugs, gender-based violence, and the criminal justice system. But I wouldn’t get the real answer about why our society tolerates such an imbalance in punishment.

As the students worked on the project through the semester, they continued to focus on the topic of sexual assault in the criminal justice system. Their research question became more empirical because they read more empirical articles about their topic. One option that they considered was to evaluate intervention programs for perpetrators of sexual assault to see if they reduced the likelihood of committing sexual assault again. Another option they considered was seeing if counties or states with higher than average jail sentences for sexual assault perpetrators had lower rates of re-offense for sexual assault. These projects addressed the ethical question of punishing perpetrators of sexual violence but did so in a way that gathered and analyzed empirical real-world data. Our job as social work researchers is to gather social facts about social work issues, not to judge or determine morality.

Key Takeaways

• Empirical questions are distinct from ethical questions.
• There are usually a number of ethical questions and a number of empirical questions that could be asked about any single topic.
• While social workers may research topics about which people have moral opinions, a researcher’s job is to gather and analyze empirical data.
• Take a look at your working question. Make sure you have an empirical question, not an ethical one. To perform this check, describe how you could find an answer to your question by conducting a study, like a survey or focus group, with real people.

9.2 Characteristics of a good research question

• Identify and explain the key features of a good research question
• Explain why it is important for social workers to be focused and clear with the language they use in their research questions

Now that you’ve made sure your working question is empirical, you need to revise that working question into a formal research question. So, what makes a good research question? First, it is generally written in the form of a question. To say that your research question is “the opioid epidemic” or “animal assisted therapy” or “oppression” would not be correct. You need to frame your topic as a question, not a statement. A good research question is also one that is well-focused. A well-focused question helps you tune out irrelevant information and not try to answer everything about the world all at once. You could be the most eloquent writer in your class, or even in the world, but if the research question about which you are writing is unclear, your work will ultimately lack direction.

In addition to being written in the form of a question and being well-focused, a good research question is one that cannot be answered with a simple yes or no. For example, if your interest is in gender norms, you could ask, “Does gender affect a person’s performance of household tasks?” but you will have nothing left to say once you discover your yes or no answer. Instead, why not ask, about the relationship between gender and household tasks. Alternatively, maybe we are interested in how or to what extent gender affects a person’s contributions to housework in a marriage? By tweaking your question in this small way, you suddenly have a much more fascinating question and more to say as you attempt to answer it.

A good research question should also have more than one plausible answer. In the example above, the student who studied the relationship between gender and household tasks had a specific interest in the impact of gender, but she also knew that preferences might be impacted by other factors. For example, she knew from her own experience that her more traditional and socially conservative friends were more likely to see household tasks as part of the female domain, and were less likely to expect their male partners to contribute to those tasks. Thinking through the possible relationships between gender, culture, and household tasks led that student to realize that there were many plausible answers to her questions about how  gender affects a person’s contribution to household tasks. Because gender doesn’t exist in a vacuum, she wisely felt that she needed to consider other characteristics that work together with gender to shape people’s behaviors, likes, and dislikes. By doing this, the student considered the third feature of a good research question–she thought about relationships between several concepts. While she began with an interest in a single concept—household tasks—by asking herself what other concepts (such as gender or political orientation) might be related to her original interest, she was able to form a question that considered the relationships  among  those concepts.

This student had one final component to consider. Social work research questions must contain a target population. Her study would be very different if she were to conduct it on older adults or immigrants who just arrived in a new country. The target population is the group of people whose needs your study addresses. Maybe the student noticed issues with household tasks as part of her social work practice with first-generation immigrants, and so she made it her target population. Maybe she wants to address the needs of another community. Whatever the case, the target population should be chosen while keeping in mind social work’s responsibility to work on behalf of marginalized and oppressed groups.

In sum, a good research question generally has the following features:

• It is written in the form of a question
• It is clearly written
• It cannot be answered with “yes” or “no”
• It has more than one plausible answer
• It considers relationships among multiple variables
• It is specific and clear about the concepts it addresses
• It includes a target population
• A poorly focused research question can lead to the demise of an otherwise well-executed study.
• Research questions should be clearly worded, consider relationships between multiple variables, have more than one plausible answer, and address the needs of a target population.

Okay, it’s time to write out your first draft of a research question.

• Once you’ve done so, take a look at the checklist in this chapter and see if your research question meets the criteria to be a good one.

Brainstorm whether your research question might be better suited to quantitative or qualitative methods.

• Describe why your question fits better with quantitative or qualitative methods.
• Provide an alternative research question that fits with the other type of research method.

9.3 Quantitative research questions

• Describe how research questions for exploratory, descriptive, and explanatory quantitative questions differ and how to phrase them
• Identify the differences between and provide examples of strong and weak explanatory research questions

Quantitative descriptive questions

The type of research you are conducting will impact the research question that you ask. Probably the easiest questions to think of are quantitative descriptive questions. For example, “What is the average student debt load of MSW students?” is a descriptive question—and an important one. We aren’t trying to build a causal relationship here. We’re simply trying to describe how much debt MSW students carry. Quantitative descriptive questions like this one are helpful in social work practice as part of community scans, in which human service agencies survey the various needs of the community they serve. If the scan reveals that the community requires more services related to housing, child care, or day treatment for people with disabilities, a nonprofit office can use the community scan to create new programs that meet a defined community need.

Quantitative descriptive questions will often ask for percentage, count the number of instances of a phenomenon, or determine an average. Descriptive questions may only include one variable, such as ours about student debt load, or they may include multiple variables. Because these are descriptive questions, our purpose is not to investigate causal relationships between variables. To do that, we need to use a quantitative explanatory question.

Quantitative explanatory questions

Most studies you read in the academic literature will be quantitative and explanatory. Why is that? If you recall from Chapter 2 , explanatory research tries to build nomothetic causal relationships. They are generalizable across space and time, so they are applicable to a wide audience. The editorial board of a journal wants to make sure their content will be useful to as many people as possible, so it’s not surprising that quantitative research dominates the academic literature.

Structurally, quantitative explanatory questions must contain an independent variable and dependent variable. Questions should ask about the relationship between these variables. The standard format I was taught in graduate school for an explanatory quantitative research question is: “What is the relationship between [independent variable] and [dependent variable] for [target population]?” You should play with the wording for your research question, revising that standard format to match what you really want to know about your topic.

Let’s take a look at a few more examples of possible research questions and consider the relative strengths and weaknesses of each. Table 9.1 does just that. While reading the table, keep in mind that I have only noted what I view to be the most relevant strengths and weaknesses of each question. Certainly each question may have additional strengths and weaknesses not noted in the table. Each of these questions is drawn from student projects in my research methods classes and reflects the work of many students on their research question over many weeks.

Making it more specific

A good research question should also be specific and clear about the concepts it addresses. A student investigating gender and household tasks knows what they mean by “household tasks.” You likely also have an impression of what “household tasks” means. But are your definition and the student’s definition the same? A participant in their study may think that managing finances and performing home maintenance are household tasks, but the researcher may be interested in other tasks like childcare or cleaning. The only way to ensure your study stays focused and clear is to be specific about what you mean by a concept. The student in our example could pick a specific household task that was interesting to them or that the literature indicated was important—for example, childcare. Or, the student could have a broader view of household tasks, one that encompasses childcare, food preparation, financial management, home repair, and care for relatives. Any option is probably okay, as long as the researcher is clear on what they mean by “household tasks.” Clarifying these distinctions is important as we look ahead to specifying how your variables will be measured in Chapter 11 .

Table 9.2 contains some “watch words” that indicate you may need to be more specific about the concepts in your research question.

It can be challenging to be this specific in social work research, particularly when you are just starting out your project and still reading the literature. If you’ve only read one or two articles on your topic, it can be hard to know what you are interested in studying. Broad questions like “What are the causes of chronic homelessness, and what can be done to prevent it?” are common at the beginning stages of a research project as working questions. However, moving from working questions to research questions in your research proposal requires that you examine the literature on the topic and refine your question over time to be more specific and clear. Perhaps you want to study the effect of a specific anti-homelessness program that you found in the literature. Maybe there is a particular model to fighting homelessness, like Housing First or transitional housing, that you want to investigate further. You may want to focus on a potential cause of homelessness such as LGBTQ+ discrimination that you find interesting or relevant to your practice. As you can see, the possibilities for making your question more specific are almost infinite.

Quantitative exploratory questions

In exploratory research, the researcher doesn’t quite know the lay of the land yet. If someone is proposing to conduct an exploratory quantitative project, the watch words highlighted in Table 9.2 are not problematic at all. In fact, questions such as “What factors influence the removal of children in child welfare cases?” are good because they will explore a variety of factors or causes. In this question, the independent variable is less clearly written, but the dependent variable, family preservation outcomes, is quite clearly written. The inverse can also be true. If we were to ask, “What outcomes are associated with family preservation services in child welfare?”, we would have a clear independent variable, family preservation services, but an unclear dependent variable, outcomes. Because we are only conducting exploratory research on a topic, we may not have an idea of what concepts may comprise our “outcomes” or “factors.” Only after interacting with our participants will we be able to understand which concepts are important.

Remember that exploratory research is appropriate only when the researcher does not know much about topic because there is very little scholarly research. In our examples above, there is extensive literature on the outcomes in family reunification programs and risk factors for child removal in child welfare. Make sure you’ve done a thorough literature review to ensure there is little relevant research to guide you towards a more explanatory question.

• Descriptive quantitative research questions are helpful for community scans but cannot investigate causal relationships between variables.
• Explanatory quantitative research questions must include an independent and dependent variable.
• Exploratory quantitative research questions should only be considered when there is very little previous research on your topic.
• Identify the type of research you are engaged in (descriptive, explanatory, or exploratory).
• Create a quantitative research question for your project that matches with the type of research you are engaged in.

Preferably, you should be creating an explanatory research question for quantitative research.

9.4 Qualitative research questions

• List the key terms associated with qualitative research questions
• Distinguish between qualitative and quantitative research questions

Qualitative research questions differ from quantitative research questions. Because qualitative research questions seek to explore or describe phenomena, not provide a neat nomothetic explanation, they are often more general and openly worded. They may include only one concept, though many include more than one. Instead of asking how one variable causes changes in another, we are instead trying to understand the experiences ,  understandings , and  meanings that people have about the concepts in our research question. These keywords often make an appearance in qualitative research questions.

Let’s work through an example from our last section. In Table 9.1, a student asked, “What is the relationship between sexual orientation or gender identity and homelessness for late adolescents in foster care?” In this question, it is pretty clear that the student believes that adolescents in foster care who identify as LGBTQ+ may be at greater risk for homelessness. This is a nomothetic causal relationship—LGBTQ+ status causes changes in homelessness.

However, what if the student were less interested in  predicting  homelessness based on LGBTQ+ status and more interested in  understanding  the stories of foster care youth who identify as LGBTQ+ and may be at risk for homelessness? In that case, the researcher would be building an idiographic causal explanation . The youths whom the researcher interviews may share stories of how their foster families, caseworkers, and others treated them. They may share stories about how they thought of their own sexuality or gender identity and how it changed over time. They may have different ideas about what it means to transition out of foster care.

Because qualitative questions usually center on idiographic causal relationships, they look different than quantitative questions. Table 9.3 below takes the final research questions from Table 9.1 and adapts them for qualitative research. The guidelines for research questions previously described in this chapter still apply, but there are some new elements to qualitative research questions that are not present in quantitative questions.

• Qualitative research questions often ask about lived experience, personal experience, understanding, meaning, and stories.
• Qualitative research questions may be more general and less specific.
• Qualitative research questions may also contain only one variable, rather than asking about relationships between multiple variables.

Qualitative research questions have one final feature that distinguishes them from quantitative research questions: they can change over the course of a study. Qualitative research is a reflexive process, one in which the researcher adapts their approach based on what participants say and do. The researcher must constantly evaluate whether their question is important and relevant to the participants. As the researcher gains information from participants, it is normal for the focus of the inquiry to shift.

For example, a qualitative researcher may want to study how a new truancy rule impacts youth at risk of expulsion. However, after interviewing some of the youth in their community, a researcher might find that the rule is actually irrelevant to their behavior and thoughts. Instead, their participants will direct the discussion to their frustration with the school administrators or the lack of job opportunities in the area. This is a natural part of qualitative research, and it is normal for research questions and hypothesis to evolve based on information gleaned from participants.

However, this reflexivity and openness unacceptable in quantitative research for good reasons. Researchers using quantitative methods are testing a hypothesis, and if they could revise that hypothesis to match what they found, they could never be wrong! Indeed, an important component of open science and reproducability is the preregistration of a researcher’s hypotheses and data analysis plan in a central repository that can be verified and replicated by reviewers and other researchers. This interactive graphic from 538 shows how an unscrupulous research could come up with a hypothesis and theoretical explanation  after collecting data by hunting for a combination of factors that results in a statistically significant relationship. This is an excellent example of how the positivist assumptions behind quantitative research and intepretivist assumptions behind qualitative research result in different approaches to social science.

• Qualitative research questions often contain words or phrases like “lived experience,” “personal experience,” “understanding,” “meaning,” and “stories.”
• Qualitative research questions can change and evolve over the course of the study.
• Using the guidance in this chapter, write a qualitative research question. You may want to use some of the keywords mentioned above.

9.5 Evaluating and updating your research questions

• Evaluate the feasibility and importance of your research questions
• Begin to match your research questions to specific designs that determine what the participants in your study will do

Feasibility and importance

As you are getting ready to finalize your research question and move into designing your research study, it is important to check whether your research question is feasible for you to answer and what importance your results will have in the community, among your participants, and in the scientific literature

Key questions to consider when evaluating your question’s feasibility include:

• Do you have access to the data you need?
• Will you be able to get consent from stakeholders, gatekeepers, and others?
• Does your project pose risk to individuals through direct harm, dual relationships, or breaches in confidentiality? (see Chapter 6 for more ethical considerations)
• Are you competent enough to complete the study?
• Do you have the resources and time needed to carry out the project?

Key questions to consider when evaluating the importance of your question include:

• Can we answer your research question simply by looking at the literature on your topic?
• How does your question add something new to the scholarly literature? (raises a new issue, addresses a controversy, studies a new population, etc.)
• How will your target population benefit, once you answer your research question?
• How will the community, social work practice, and the broader social world benefit, once you answer your research question?
• Using the questions above, check whether you think your project is feasible for you to complete, given the constrains that student projects face.
• Realistically, explore the potential impact of your project on the community and in the scientific literature. Make sure your question cannot be answered by simply reading more about your topic.

Matching your research question and study design

This chapter described how to create a good quantitative and qualitative research question. In Parts 3 and 4 of this textbook, we will detail some of the basic designs like surveys and interviews that social scientists use to answer their research questions. But which design should you choose?

As with most things, it all depends on your research question. If your research question involves, for example, testing a new intervention, you will likely want to use an experimental design. On the other hand, if you want to know the lived experience of people in a public housing building, you probably want to use an interview or focus group design.

We will learn more about each one of these designs in the remainder of this textbook. We will also learn about using data that already exists, studying an individual client inside clinical practice, and evaluating programs, which are other examples of designs. Below is a list of designs we will cover in this textbook:

• Surveys: online, phone, mail, in-person
• Experiments: classic, pre-experiments, quasi-experiments
• Interviews: in-person or via phone or videoconference
• Focus groups: in-person or via videoconference
• Content analysis of existing data
• Secondary data analysis of another researcher’s data
• Program evaluation

The design of your research study determines what you and your participants will do. In an experiment, for example, the researcher will introduce a stimulus or treatment to participants and measure their responses. In contrast, a content analysis may not have participants at all, and the researcher may simply read the marketing materials for a corporation or look at a politician’s speeches to conduct the data analysis for the study.

I imagine that a content analysis probably seems easier to accomplish than an experiment. However, as a researcher, you have to choose a research design that makes sense for your question and that is feasible to complete with the resources you have. All research projects require some resources to accomplish. Make sure your design is one you can carry out with the resources (time, money, staff, etc.) that you have.

There are so many different designs that exist in the social science literature that it would be impossible to include them all in this textbook. The purpose of the subsequent chapters is to help you understand the basic designs upon which these more advanced designs are built. As you learn more about research design, you will likely find yourself revising your research question to make sure it fits with the design. At the same time, your research question as it exists now should influence the design you end up choosing. There is no set order in which these should happen. Instead, your research project should be guided by whether you can feasibly carry it out and contribute new and important knowledge to the world.

• Research questions must be feasible and important.
• Research questions must match study design.
• Based on what you know about designs like surveys, experiments, and interviews, describe how you might use one of them to answer your research question.
• You may want to refer back to Chapter 2 which discusses how to get raw data about your topic and the common designs used in student research projects.
• Not familiar with SpongeBob SquarePants? You can learn more about him on Nickelodeon’s site dedicated to all things SpongeBob:  http://www.nick.com/spongebob-squarepants/ ↵
• Focus on the Family. (2005, January 26). Focus on SpongeBob.  Christianity Today . Retrieved from  http://www.christianitytoday.com/ct/2005/januaryweb-only/34.0c.html ↵
• BBC News. (2005, January 20). US right attacks SpongeBob video. Retrieved from:  http://news.bbc.co.uk/2/hi/americas/4190699.stm ↵
• In fact, an MA thesis examines representations of gender and relationships in the cartoon: Carter, A. C. (2010).  Constructing gender and   relationships in “SpongeBob SquarePants”: Who lives in a pineapple under the sea . MA thesis, Department of Communication, University of South Alabama, Mobile, AL. ↵
• Writing from an outline (10 minute read plus an 8 minute video, and then a 15 minute video)
• Writing your literature review (30 minute read)

Content warning: TBA

6.1: Writing from an outline

Learners will be able to...

• Integrate facts from the literature into scholarly writing
• Experiment with different approaches to integrating information that do not involve direct quotations from other authors

Congratulations! By now, you should have discovered, retrieved, evaluated, synthesized, and organized the information you need for your literature review. It’s now time to turn that stack of articles, papers, and notes into a literature review–it’s time to start writing!

The first step in research writing is outlining. In Chapter 4, we reviewed how to build a topical outline using quotations and facts from other authors. Use that outline (or one you write now) as a way to organize your thoughts.

Watch this video from Nicholas Cifuentes-Goodbody on Outlining . As he highlights, outlining is like building a mise en place before a meal--arranging your ingredients in an orderly way so you can create your masterpiece.

From quotations to original writing

Much like combining ingredients on a kitchen countertop, you will need to mix your ingredients together. That means you will not be relying extensively on quotations from other authors in your literature review. In moving from an outline to a literature review, the key intellectual move is relying on your own ideas about the literature, rather than quoting extensively from other sources.

Integrating ideas from other authors

Watch this video from Nicholas Cifuentes-Goodbody on using quotations in academic writing . In the video, he reviews a few different techniques to integrate quotations or ideas from other authors into your writing. All literature reviews use the ideas from other authors, but it's important not to overuse others' words. Your literature review is evaluated by your professor based on how well it shows  you are able to make connections between different facts in the scientific literature. The examples in this section should highlight how to get other people's words out of the way of your own. Use these strategies to diversify your writing and show your readers how your sources contributed to your work.

1. Make a claim without a quote

Claim ( Citation )

Some view cities as the storehouse of culture and creativity, and propose that urbanization is a consequence of the attractiveness of these social benefits ( Mumford, 1961 ).

More information

Oftentimes you do not need to directly quote a source to convey its conclusions or arguments – and some disciplines discourage quoting directly! Rather you can paraphrase the main point of a paper in your own words and provide an in-text citation. A benefit of using this strategy is that you can offer support for a claim without using a whole paragraph to introduce and frame a quote. You should make sure that you fully understand the paper's argument and that you are following university citation guidelines before attempting to paraphrase something from a paper.

2. Make a claim that is supported by two or more sources:

Claim ( Citation 1 ; Citation 2 ).

Reviews of this literature concede difficulty in making direct comparisons of emission levels across different sets of analysis ( Bader and Bleischwitz, 2009 ; Kennedy et al., 2009 ; Ramaswami et al., 2012 ).

Sometimes multiple sources support your claim, or there are two major publications that deserve credit for providing evidence on a topic. This is a perfect time to use multiple citations. You can cite two, three, or more sources in a single sentence!

Make a claim that has been supported in multiple contexts:

Context 1 ( Citation ), Context 2 ( Citation ), Context 3 ( Citation ).

These results are supported by more recent research on transportation energy consumption ( Liddle, 2014 ), electricity consumption in buildings ( Lariviere and Lafrance, 1999 ), and overall urban GHG emissions ( Marcotullio et al., 2013b ).

More information:

Use this citation strategy when you want to show that a body of research has found support for some claim across several different contexts. This can show the robustness of an effect or phenomenon and can give your claim some added validity

3. Quote important or unique terms

Claim " Term " ( Citation ).

The spatial implications of this thinking are manifest in the " concentric ring model " of urban expansion and its variants ( Harris and Ullman, 1945 ).

While block or even whole-sentence citations are rare in most research papers in the science and social science disciplines, there is often a need to quote specific terms or phrases that were first coined by a certain source or that were well-explained in a specific paper.

4. Quoting definitions

Contextualize quote , " important word or phrase ."

Role conflict is defined as "A situation in which contradictory, competing, or incompatible expectations are placed on an individual by two or more roles held at the same time" (Open Sociology Dictionary, 2023); whereas, role strain is defined as "a situation caused by higher-than-expected demands placed on an individual performing a specific role that leads to difficulty or stress" (Open Sociology Dictionary, 2023). In our study, we hypothesize that caregivers who reenter higher education experience role conflict between school work, paid work, and care work. Further, we hypothesize that this conflict is greater in individuals who had experienced role strain in employment or caregiving prior to entering college.

A direct quotation can bring attention to specific language in your source. When someone puts something perfectly, you can use a quotation to convey the identical meaning in your work. Definitions are an excellent example of when to use a quotation. In other cases, there may be quotations from important thinkers, clients or community members, and others whose specific wording is important.

I encourage you to use few, if any, direct quotations in your literature review. Personally, I think most students are scared of looking stupid and would rather use a good quotation than risk not getting it right. If you are a student who considers themselves a strong writer, this may not sound relevant to you. However, I'm willing to bet that there are many of your peers for whom this describes a particular bit of research anxiety.

When using quotations, make sure to only include the parts of the quotation that are necessary. You do not need to use quotation marks for statistics you use. And I encourage you to find ways to put others' statistics in  your sentences.

Why share information from other sources?

Now that you know some different sentence structures using APA citations, let's examine the purpose behind why you are sharing information from another source. Cited evidence can serve a wide range of purposes in academic papers. These examples will give you an idea of the different ways that you can use citations in your paper.

1. Summarize your source

The studies of Newman and Kenworthy ( 1989, 1999 ) demonstrate a negative relationship between population density and transportation fuel use .

You will help your reader understand your points better if you summarize the key points of a study. Describe the strengths or weaknesses a specific source that has been pivotal in your field. Describe the source's specific methodology, theory, or approach. Be sure to still include a citation. If you mention the name of the author in your text, you still need to provide the date of the study in a parenthetical citation.

2. Cite a method

Despite the popularity of the WUP indicators , they have been routinely criticized because the methodology relies on local- and country-specific definitions of bounding urban areas, resulting in of ten incomparable and widely divergent definitions of the population, density thresholds, or administrative/political units designated ( Satterthwaite, 2007 ).

This is an easy way to give credit to a source that has provided some evidence for the validity of a method or questionnaire. Readers can reference your citation if they are interested in knowing more about the method and its standing in the current literature.

3. Compare sources

Some evidence for this scaling relationship suggests that urban areas with larger population sizes have proportionally smaller energy infrastructures than smaller cities ( Bettencourt et al., 2007 ; Fragkias et al., 2013 ). Other evidence suggests that GHG emissions may increase more than proportionally to population size, such that larger cities exhibit proportionally higher energy demand as they grow than do smaller cities ( Marcotullio et al., 2013 ).

This is one of the most important techniques for creating an effective literature review. This allows you and your readers to consider controversies and discrepancies among the current literature, revealing gaps in the literature or points of contention for further study.

The examples in this guide come from:

Marcotullio, P. J., Hughes, S., Sarzynski, A., Pincetl, S., Sanchez Peña, L., Romero-Lankao, P., Runfola, D. and Seto, K. C. (2014), Urbanization and the carbon cycle: Contributions from social science. Earth's Future, 2: 496–514. doi:10.1002/2014EF000257

Avoiding plagiarism

The most difficult thing about avoiding plagiarism is that reading so much of other people's ideas can make them seem like your own after a while. We recommend you work through this interactive activity on determining how and when to cite other authors.

• Research writing requires outlining, which helps you arrange your facts neatly before writing. It's similar to arranging all of your ingredients before you start cooking.
• Eliminate quotations from your writing as much as possible. Your literature review needs to be your analysis of the literature, not just a summary of other people's good ideas.
• Experiment with the prompts in this chapter as you begin to write your research question. 

6.2 Writing your literature review

• Describe the components of a literature review
• Begin to write your literature review
• Identify the purpose of a problem statement
• Apply the components of a formal argument to your topic
• Use elements of formal writing style, including signposting and transitions
• Recognize commons errors in literature reviews

Writing about research is different than other types of writing. Research writing is not like a journal entry or opinion paper. The goal here is not to apply your research question to your life or growth as a practitioner. Research writing is about the provision and interpretation of facts. The tone should be objective and unbiased, and personal experiences and opinions are excluded. Particularly for students who are used to writing case notes, research writing can be a challenge. That's why its important to normalize getting help! If your professor has not built in peer review, consider setting up a peer review group among your peers. You should also reach out to your academic advisor to see if there are writing services on your campus available to graduate students. No one should feel bad for needing help with something they haven't done before, haven't done in a while, or were never taught how to do. 

If you’ve followed the steps in this chapter, you likely have an outline, summary table, and concept map from which you can begin the writing process. But what do you need to include in your literature review? We’ve mentioned it before, but to summarize, a literature review should:

• Introduce the topic and define its key terms.
• Establish the importance of the topic.
• Provide an overview of the important literature related to the concepts found in the research question.
• Identify gaps or controversies in the literature.
• Point out consistent findings across studies.
• Synthesize that which is known about a topic, rather than just provide a summary of the articles you read.
• Discuss possible implications and directions for future research.

Do you have enough facts and sources to accomplish these tasks? It’s a good time to consult your outlines and notes on each article you plan to include in your literature review. You may also want to consult with your professor on what is expected of you. If there is something you are missing, you may want to jump back to section 2.3 where we discussed how to search for literature. While you can always fill in material, there is the danger that you will start writing without really knowing what you are talking about or what you want to say. For example, if you don’t have a solid definition of your key concepts or a sense of how the literature has developed over time, it will be difficult to make coherent scholarly claims about your topic.

There is no magical point at which one is ready to write. As you consider whether you are ready, it may be useful to ask yourself these questions:

• How will my literature review be organized?
• What section headings will I be using?
• How do the various studies relate to each other?
• What contributions do they make to the field?
• Where are the gaps or limitations in existing research?
• And finally, but most importantly, how does my own research fit into what has already been done?

The problem statement

Scholarly works often begin with a problem statement, which serves two functions. First, it establishes why your topic is a social problem worth studying. Second, it pulls your reader into the literature review. Who would want to read about something unimportant?

A problem statement generally answers the following questions, though these are far from exhaustive:

• Why is this an important problem to study?
• How many people are affected by this problem?
• How does this problem impact other social issues relevant to social work?
• Why is your target population an important one to study?

A strong problem statement, like the rest of your literature review, should be filled with empirical results, theory, and arguments based on the extant literature. A research proposal differs significantly from other more reflective essays you’ve likely completed during your social work studies. If your topic were domestic violence in rural Appalachia, I’m sure you could come up with answers to the above questions without looking at a single source. However, the purpose of the literature review is not to test your intuition, personal experience, or empathy. Instead, research methods are about gaining specific and articulable knowledge to inform action. With a problem statement, you can take a “boring” topic like the color of rooms used in an inpatient psychiatric facility, transportation patterns in major cities, or the materials used to manufacture baby bottles, and help others see the topic as you see it—an important part of the social world that impacts social work practice.

The structure of a literature review

In general, the problem statement belongs at the beginning of the literature review. We usually advise students to spend no more than a paragraph or two for a problem statement. For the rest of your literature review, there is no set formula by which it needs to be organized. However, a literature review generally follows the format of any other essay—Introduction, Body, and Conclusion.

The introduction to the literature review contains a statement or statements about the overall topic. At a minimum, the introduction should define or identify the general topic, issue, or area of concern. You might consider presenting historical background, mentioning the results of a seminal study, and providing definitions of important terms. The introduction may also point to overall trends in what has been previously published on the topic or on conflicts in theory, methodology, evidence, conclusions, or gaps in research and scholarship. We also suggest putting in a few sentences that walk the reader through the rest of the literature review. Highlight your main arguments from the body of the literature review and preview your conclusion. An introduction should let the reader know what to expect from the rest of your review.

The body of your literature review is where you demonstrate your synthesis and analysis of the literature. Again, do not just summarize the literature. We would also caution against organizing your literature review by source—that is, one paragraph for source A, one paragraph for source B, etc. That structure will likely provide an adequate summary of the literature you’ve found, but it would give you almost no synthesis of the literature. That approach doesn’t tell your reader how to put those facts together, it doesn't highlight points of agreement or contention, or how each study builds on the work of others. In short, it does not demonstrate critical thinking.

Organize your review by argument

Instead, use your outlines and notes as a guide what you have to say about the important topics you need to cover. Literature reviews are written from the perspective of an expert in that field. After an exhaustive literature review, you should feel as though you are able to make strong claims about what is true—so make them! There is no need to hide behind “I believe” or “I think.” Put your voice out in front, loud and proud! But make sure you have facts and sources that back up your claims.

I’ve used the term “ argument ” here in a specific way. An argument in writing means more than simply disagreeing with what someone else said, as this classic Monty Python sketch demonstrates. Toulman, Rieke, and Janik (1984) identify six elements of an argument:

• Claim: the thesis statement—what you are trying to prove
• Grounds: theoretical or empirical evidence that supports your claim
• Warrant: your reasoning (rule or principle) connecting the claim and its grounds
• Backing: further facts used to support or legitimize the warrant
• Qualifier: acknowledging that the argument may not be true for all cases
• Rebuttal: considering both sides (as cited in Burnette, 2012) [1]

Let’s walk through an example. If I were writing a literature review on a negative income tax, a policy in which people in poverty receive an unconditional cash stipend from the government each month equal to the federal poverty level, I would want to lay out the following:

• Claim: the negative income tax is superior to other forms of anti-poverty assistance.
• Grounds: data comparing negative income tax recipients to people receiving anti-poverty assistance in existing programs, theory supporting a negative income tax, data from evaluations of existing anti-poverty programs, etc.
• Warrant: cash-based programs like the negative income tax are superior to existing anti-poverty programs because they allow the recipient greater self-determination over how to spend their money.
• Backing: data demonstrating the beneficial effects of self-determination on people in poverty.
• Qualifier: the negative income tax does not provide taxpayers and voters with enough control to make sure people in poverty are not wasting financial assistance on frivolous items.
• Rebuttal: policy should be about empowering the oppressed, not protecting the taxpayer, and there are ways of addressing taxpayer spending concerns through policy design.

Like any effective argument, your literature review must have some kind of structure. For example, it might begin by describing a phenomenon in a general way along with several studies that provide some detail, then describing two or more competing theories of the phenomenon, and finally presenting a hypothesis to test one or more of the theories. Or, it might describe one phenomenon, then describe another that seems inconsistent with the first, then propose a theory that resolves the inconsistency, and finally present a hypothesis to test that theory. In applied research, it might describe a phenomenon or theory, then describe how that phenomenon or theory applies to some important real-world situation, and finally, may suggest a way to test whether it does, in fact, apply to that situation.

Use signposts

Another important issue is  signposting . It may not be a term you are familiar with, but you are likely familiar with the concept. Signposting refers to the words used to identify the organization and structure of your literature review to your reader. The most basic form of signposting is using a topic sentence at the beginning of each paragraph. A topic sentence introduces the argument you plan to make in that paragraph. For example, you might start a paragraph stating, “There is strong disagreement in the literature as to whether psychedelic drugs cause people to develop psychotic disorders, or whether psychotic disorders cause people to use psychedelic drugs.” Within that paragraph, your reader would likely assume you will present evidence for both arguments. The concluding sentence of your paragraph should address the topic sentence, discussing how the facts and arguments from the paragraph you've written support a specific conclusion. To continue with our example, I might say, “There is likely a reciprocal effect in which both the use of psychedelic drugs worsens pre-psychotic symptoms and worsening psychosis increases the desire to use psychedelic drugs.”

Signposting also involves using headings and subheadings. Your literature review will use APA formatting, which means you need to follow their rules for bolding, capitalization, italicization, and indentation of headings. Headings help your reader understand the structure of your literature review. They can also help if the reader gets lost and needs to re-orient themselves within the document. We often tell our students to assume we know nothing (they don’t mind) and need to be shown exactly where they are addressing each part of the literature review. It’s like walking a small child around, telling them “First we’ll do this, then we’ll do that, and when we’re done, we’ll know this!”

Another way to use signposting is to open each paragraph with a sentence that links the topic of the paragraph with the one before it. Alternatively, one could end each paragraph with a sentence that links it with the next paragraph. For example, imagine we wanted to link a paragraph about barriers to accessing healthcare with one about the relationship between the patient and physician. We could use a transition sentence like this: “Even if patients overcome these barriers to accessing care, the physician-patient relationship can create new barriers to positive health outcomes.” A transition sentence like this builds a connection between two distinct topics. Transition sentences are also useful within paragraphs. They tell the reader how to consider one piece of information in light of previous information. Even simple transitional words like 'however' and 'similarly' can help demonstrate critical thinking and link each building block of your argument together.

Many beginning researchers have difficulty incorporating transitions into their writing. Let’s look at an example. Instead of beginning a sentence or paragraph by launching into a description of a study, such as “Williams (2004) found that…,” it is better to start by indicating something about why you are describing this particular study. Here are some simple examples:

• Another example of this phenomenon comes from the work of Williams (2004)...
• Williams (2004) offers one explanation of this phenomenon...
• An alternative perspective has been provided by Williams (2004)...

Now that we know to use signposts, the natural question is “What goes on the signposts?” First, it is important to start with an outline of the main points that you want to make, organized in the order you want to make them. The basic structure of your argument should then be apparent from the outline itself. Unfortunately, there is no formula we can give you that will work for everyone, but we can provide some general pointers on structuring your literature review.

The literature review tends to move from general to more specific ideas. You can build a review by identifying areas of consensus and areas of disagreement. You may choose to present historical studies—preferably seminal studies that are of significant importance—and close with the most recent research. Another approach is to start with the most distantly related facts and literature and then report on those most closely related to your research question. You could also compare and contrast valid approaches, features, characteristics, theories – that is, one approach, then a second approach, followed by a third approach.

Here are some additional tips for writing the body of your literature review:

• Start broad and then narrow down to more specific information.
• When appropriate, cite two or more sources for a single point, but avoid long strings of references for a single idea.
• Use quotes sparingly. Quotations for definitions are okay, but reserve quotes for when something is said so well you couldn’t possible phrase it differently. Never use quotes for statistics.
• Paraphrase when you need to relay the specific details within an article
• Include only the aspects of the study that are relevant to your literature review. Don’t insert extra facts about a study just to take up space.
• Avoid reflective, personal writing. It is traditional to avoid using first-person language (I, we, us, etc.).
• Avoid informal language like contractions, idioms, and rhetorical questions.
• Note any sections of your review that lack citations from the literature. Your arguments need to be based in empirical or theoretical facts. Do not approach this like a reflective journal entry.
• Point out consistent findings and emphasize stronger studies over weaker ones.
• Point out important strengths and weaknesses of research studies, as well as contradictions and inconsistent findings.
• Implications and suggestions for further research (where there are gaps in the current literature) should be specific.

The conclusion should summarize your literature review, discuss implications, and create a space for further research needed in this area. Your conclusion, like the rest of your literature review, should make a point. What are the important implications of your literature review? How do they inform the question you are trying to answer?

You should consult with your professor and the course syllabus about the final structure your literature review should take. Here is an example of one possible structure:

• Establish the importance of the topic
• Number and type of people affected
• Seriousness of the impact
• Physical, psychological, economic, social, or spiritual consequences of the problem
• Definitions of key terms
• Supporting evidence
• Common findings across studies, gaps in the literature
• Research question(s) and hypothesis(es)

Editing your literature review

Literature reviews are more than a summary of the publications you find on a topic. As you have seen in this brief introduction, literature reviews represent a very specific type of research, analysis, and writing. We will explore these topics further in upcoming chapters. As you begin your literature review, here are some common errors to avoid:

• Accepting a researcher’s finding as valid without evaluating methodology and data
• Ignoring contrary findings and alternative interpretations
• Using findings that are not clearly related to your own study or using findings that are too general
• Dedicating insufficient time to literature searching
• Reporting statistical results from a single study, rather than synthesizing the results of multiple studies to provide a comprehensive view of the literature on a topic
• Relying too heavily on secondary sources
• Overusing quotations
• Not justifying arguments using specific facts or theories from the literature

For your literature review, remember that your goal is to construct an argument for the importance of your research question. As you start editing your literature review, make sure it is balanced. Accurately report common findings, areas where studies contradict each other, new theories or perspectives, and how studies cause us to reaffirm or challenge our understanding of your topic.

It is acceptable to argue that the balance of the research supports the existence of a phenomenon or is consistent with a theory (and that is usually the best that researchers in social work can hope for), but it is not acceptable to ignore contradictory evidence. A large part of what makes a research question interesting is uncertainty about its answer (University of Minnesota, 2016). [2]

In addition to subjectivity and bias, writer's block can obstruct the completion of your literature review. Often times, writer’s block can stem from confusing the creating and editing parts of the writing process. Many writers often start by simply trying to type out what they want to say, regardless of how good it is. Author Anne Lamott (1995) [3] terms these “shitty first drafts,” and we all write them. They are a natural and important part of the writing process.

Even if you have a detailed outline from which to work, the words are not going to fall into place perfectly the first time you start writing. You should consider turning off the editing and critiquing part of your brain for a while and allow your thoughts to flow. Don’t worry about putting a correctly formatted internal citation (as long as  you know which source you used there) when you first write. Just get the information out. Only after you’ve reached a natural stopping point might you go back and edit your draft for grammar, APA style, organization, flow, and more. Divorcing the writing and editing process can go a long way to addressing writer’s block—as can picking a topic about which you have something to say!

As you are editing, keep in mind these questions adapted from Green (2012): [4]

• Content: Have I clearly stated the main idea or purpose of the paper? Is the thesis or focus clearly presented and appropriate for the reader?
• Organization: How well is it structured? Is the organization spelled out and easy to follow for the reader ?
• Flow: Is there a logical flow from section to section, paragraph to paragraph, sentence to sentence? Are there transitions between and within paragraphs that link ideas together?
• Development: Have I validated the main idea with supporting material? Are supporting data sufficient? Does the conclusion match the introduction?
• Form: Are there any APA style issues, redundancy, problematic wording and terminology (always know the definition of any word you use!), flawed sentence constructions and selection, spelling, and punctuation?

Social workers use the APA style guide to format and structure their literature reviews. Most students know APA style only as it relates to internal and external citations. If you are confused about them, consult this amazing APA style guide from the University of Texas-Arlington library. Your university's library likely has resources they created to help you with APA style, and you can meet with a librarian or your professor to talk about formatting questions you have. Make sure you budget in a few hours at the end of each project to build a correctly formatted references page and check your internal citations. The highest quality online source of information on APA style is the APA style blog, where you can search questions and answers from the

Of course, APA style is about much more than knowing there is a period after "et al." or citing the location a book was published. APA style is also about what the profession considers to be good writing. If you haven't picked up an APA publication manual because you use citation generators, know that I did the same thing when I was in school. Purchasing the APA manual can help you with a common problem we hear about from students. Every professor (and every website about APA style) seems to have their own peculiar idea of "correct" APA style that you can, if needed, demonstrate is not accurate.

• A literature review is not a book report. Do not organize it by article, with one paragraph for each source in your references. Instead, organize it based on the key ideas and arguments.
• The problem statement draws the reader into your topic by highlighting the importance of the topic to social work and to society overall.
• Signposting is an important component of academic writing that helps your reader follow the structure of your argument and of your literature review.
• Transitions demonstrate critical thinking and help guide your reader through your arguments.
• Editing and writing are separate processes.
• Consult with an APA style guide or a librarian to help you format your paper.

Look at your professor's prompt for the literature review component of your research proposal (or if you don't have one, use the example question provided in this section).

• Write 2-3 facts you would use to address each question or component in the prompt.
• Reflect on which questions you have a lot of information about and which you need to gather more information about in order to answer adequately.

Outline the structure of your literature review using your concept map from Section 5.2 as a guide.

• Identify the key arguments you will make and how they are related to each other.
• Reflect on topic sentences and concluding sentences you would use for each argument.
• Human subjects research (19 minute read)
• Specific ethical issues to consider (12 minute read)
• Benefits and harms of research across the ecosystem (7 minute read)
• Being an ethical researcher (8 minute read)

Content warning: examples in this chapter contain references to numerous incidents of unethical medical experimentation (e.g. intentionally injecting diseases into unknowing participants, withholding proven treatments), social experimentation under extreme conditions (e.g. being directed to deliver electric shocks to test obedience), violations of privacy, gender and racial inequality, research with people who are incarcerated or on parole, experimentation on animals, abuse of people with Autism, community interactions with law enforcement, WWII, the Holocaust, and Nazi activities (especially related to research on humans).

With your literature review underway, you are ready to begin thinking in more concrete terms about your research topic. Recall our discussion in Chapter 2 on practical and ethical considerations that emerge as part of the research process. In this chapter, we will expand on the ethical boundaries that social scientists must abide by when conducting human subjects research. As a result of reading this chapter, you should have a better sense of what is possible and ethical for the research project you create.

6.1 Human subjects research

• Understand what we mean by ethical research and why it is important
• Understand some of the egregious ethical violations that have occurred throughout history

While all research comes with its own set of ethical concerns, those associated with research conducted on human subjects vary dramatically from those of research conducted on nonliving entities. The US Department of Health and Human Services (USDHHS) defines a human subject as “a living individual about whom an investigator (whether professional or student) conducting research obtains (1) data through intervention or interaction with the individual, or (2) identifiable private information” (USDHHS, 1993, para. 1). [5] Some researchers prefer the term "participants" to "subjects'" as it acknowledges the agency of people who participate in the study. For our purposes, we will use the two terms interchangeably.

In some states, human subjects also include deceased individuals and human fetal materials. Nonhuman research subjects, on the other hand, are objects or entities that investigators manipulate or analyze in the process of conducting research. Nonhuman research subjects typically include sources such as newspapers, historical documents, pieces of clothing, television shows, buildings, and even garbage (to name just a few), that are analyzed for unobtrusive research projects. Unsurprisingly, research on human subjects is regulated much more heavily than research on nonhuman subjects. This is why many student research projects use data that is publicly available, rather than recruiting their own study participants. However, there are ethical considerations that all researchers must take into account, regardless of their research subject. We’ll discuss those considerations in addition to concerns that are unique to human subject research.

Why do research participants need protection?

First and foremost, we are professionally bound to engage in the ethical practice of research. This chapter discusses ethical research and will show you how to engage in research that is consistent with the NASW Code of Ethics as well as national and international ethical standards all researchers are accountable to. Before we begin, we need to understand the historical occurrences that were the catalyst for the formation of the current ethical standards . This chapter will enable you to view ethics from a micro, mezzo, and macro perspective.

The research process has led to many life-changing discoveries; these have improved life expectancy, improved living conditions, and helped us understand what contributes to certain social problems. That said, not all research has been conducted in respectful, responsible, or humane ways. Unfortunately, some research projects have dramatically marginalized, oppressed, and harmed participants and whole communities.

Would you believe that the following actions have been carried out in the name of research? I realize there was a content warning at the beginning of the chapter, but it is worth mentioning that the list below of research atrocities may be particularly upsetting or triggering.

• intentionally froze healthy body parts of prisoners to see if they could develop a treatment for freezing [6]
• scaled the body parts of prisoners to how best to treat soldiers who had injuries from being exposed to high temperatures [7]
• intentionally infected healthy individuals to see if they could design effective methods of treatment for infections [8]
• gave healthy people TB to see if they could treat it [9]
• attempted to transplant limbs, bones, and muscles to another person to see if this was possible [10]
• castrated and irradiated genitals to see if they could develop a faster method of sterilization [11]
• starved people and only allowed them to drink seawater to see if they could make saline water drinkable [12]
• artificially inseminated women with animal sperm to see what would happen [13]
• gassed living people to document how they would die [14]
• conducted cruel experiments on people and if they did not die, would kill them so they could undergo an autopsy [15]
• refused to treat syphilis in African American men (when treatment was available) because they wanted to track the progression of the illness [16]
• vivisected humans without anesthesia to see how illnesses that researches gave prisoners impacted their bodies [17]
• intentionally tried to infect prisoners with the Bubonic Plague [18]
• intentionally infected prisoners, prostitutes, soldiers, and children with syphilis to study the disease's progression [19]
• performed gynecological experiments on female slaves without anesthesia to investigate new surgical methods [20]

The sad fact is that not only did all of these occur, in many instances, these travesties continued for years until exposed and halted. Additionally, these examples have contributed to the formation of a legacy of distrust toward research. Specifically, many underrepresented groups have a deep distrust of agencies that implement research and are often skeptical of research findings. This has made it difficult for groups to support and have confidence in medical treatments, advances in social service programs, and evidence-informed policy changes. While the aforementioned unethical examples may have ended, this deep and painful wound on the public's trust remains. Consequently, we must be vigilant in our commitment to ethical research.

Many of the situations described may seem like extreme historical cases of misuse of power as researchers. However, ethical problems in research don't just happen in these extreme occurrences. None of us are immune to making unethical choices and the ethical practice of research requires conscientious mindful attention to what we are asking of our research participants. A few examples of less noticeable ethical issues might include: failing to fully explain to someone in advance what their participation might involve because you are in a rush to recruit a large enough sample; or only presenting findings that support your ideas to help secure a grant that is relevant to your research area. Remember, any time research is conducted with human beings, there is the chance that ethical violations may occur that pose social, emotional, and even physical risks for groups, and this is especially true when vulnerable or oppressed groups are involved.

A brief history of unethical social science research

Research on humans hasn’t always been regulated in the way it is today. The earliest documented cases of research using human subjects are of medical vaccination trials (Rothman, 1987). [21] One such case took place in the late 1700s, when scientist Edward Jenner exposed an 8-year-old boy to smallpox in order to identify a vaccine for the devastating disease. Medical research on human subjects continued without much law or policy intervention until the mid-1900s when, at the end of World War II, a number of Nazi doctors and scientists were put on trial for conducting human experimentation during the course of which they tortured and murdered many concentration camp inmates (Faden & Beauchamp, 1986). [22] The trials, conducted in Nuremberg, Germany, resulted in the creation of the Nuremberg Code , a 10-point set of research principles designed to guide doctors and scientists who conduct research on human subjects. Today, the Nuremberg Code guides medical and other research conducted on human subjects, including social scientific research.

Medical scientists are not the only researchers who have conducted questionable research on humans. In the 1960s, psychologist Stanley Milgram (1974) [23] conducted a series of experiments designed to understand obedience to authority in which he tricked subjects into believing they were administering an electric shock to other subjects. In fact, the shocks weren’t real at all, but some, though not many, of Milgram’s research participants experienced extreme emotional distress after the experiment (Ogden, 2008). [24] A reaction of emotional distress is understandable. The realization that one is willing to administer painful shocks to another human being just because someone who looks authoritative has told you to do so might indeed be traumatizing—even if you later learn that the shocks weren’t real.

Around the same time that Milgram conducted his experiments, sociology graduate student Laud Humphreys (1970) [25] was collecting data for his dissertation on the tearoom trade, which was the practice of men engaging in anonymous sexual encounters in public restrooms. Humphreys wished to understand who these men were and why they participated in the trade. To conduct his research, Humphreys offered to serve as a “watch queen,” in a local park restroom where the tearoom trade was known to occur. His role would be to keep an eye out for police while also getting the benefit of being able to watch the sexual encounters. What Humphreys did not do was identify himself as a researcher to his research subjects. Instead, he watched his subjects for several months, getting to know several of them, learning more about the tearoom trade practice and, without the knowledge of his research subjects, jotting down their license plate numbers as they pulled into or out of the parking lot near the restroom.

Sometime after participating as a watch queen, with the help of several insiders who had access to motor vehicle registration information, Humphreys used those license plate numbers to obtain the names and home addresses of his research subjects. Then, disguised as a public health researcher, Humphreys visited his subjects in their homes and interviewed them about their lives and their health. Humphreys’ research dispelled a good number of myths and stereotypes about the tearoom trade and its participants. He learned, for example, that over half of his subjects were married to women and many of them did not identify as gay or bisexual. [26]

Once Humphreys’ work became public, there was some major controversy at his home university (e.g., the chancellor tried to have his degree revoked), among scientists in general, and among members of the public, as it raised public concerns about the purpose and conduct of social science research. In addition, the Washington   Post  journalist Nicholas von Hoffman wrote the following warning about “sociological snoopers”:

We’re so preoccupied with defending our privacy against insurance investigators, dope sleuths, counterespionage men, divorce detectives and credit checkers, that we overlook the social scientists behind the hunting blinds who’re also peeping into what we thought were our most private and secret lives. But they are there, studying us, taking notes, getting to know us, as indifferent as everybody else to the feeling that to be a complete human involves having an aspect of ourselves that’s unknown (von Hoffman, 1970). [27]

In the original version of his report, Humphreys defended the ethics of his actions. In 2008 [28] , years after Humphreys’ death, his book was reprinted with the addition of a retrospect on the ethical implications of his work. In his written reflections on his research and the fallout from it, Humphreys maintained that his tearoom observations constituted ethical research on the grounds that those interactions occurred in public places. But Humphreys added that he would conduct the second part of his research differently. Rather than trace license numbers and interview unwitting tearoom participants in their homes under the guise of public health research, Humphreys instead would spend more time in the field and work to cultivate a pool of informants. Those informants would know that he was a researcher and would be able to fully consent to being interviewed. In the end, Humphreys concluded “there is no reason to believe that any research subjects have suffered because of my efforts, or that the resultant demystification of impersonal sex has harmed society” (Humphreys, 2008, p. 231). [29]

Today, given increasing regulation of social scientific research, chances are slim that a researcher would be allowed to conduct a project similar to Humphreys’. Some argue that Humphreys’ research was deceptive, put his subjects at risk of losing their families and their positions in society, and was therefore unethical (Warwick, 1973; Warwick, 1982). [30] Others suggest that Humphreys’ research “did not violate any premise of either beneficence or the sociological interest in social justice” and that the benefits of Humphreys’ research, namely the dissolution of myths about the tearoom trade specifically and human sexual practice more generally, outweigh the potential risks associated with the work (Lenza, 2004, p. 23). [31] What do you think, and why?

These and other studies (Reverby, 2009) [32] led to increasing public awareness of and concern about research on human subjects. In 1974, the US Congress enacted the National Research Act , which created the National Commission for the Protection of Human Subjects in Biomedical and Behavioral Research. The commission produced  The Belmont Report , a document outlining basic ethical principles for research on human subjects (National Commission for the Protection of Human Subjects in Biomedical and Behavioral Research, 1979). [33] The National Research Act (1974) [34] also required that all institutions receiving federal support establish institutional review boards (IRBs) to protect the rights of human research subjects. Since that time, many organizations that do not receive federal support but where research is conducted have also established review boards to evaluate the ethics of the research that they conduct. IRBs are overseen by the federal Office of Human Research Protections .

The Belmont Report

As mentioned above, The Belmont Report is a federal document that outlines the foundational principles that guide the ethical practice of research in the United States. These ethical principles include: respect for persons, beneficence, and justice. Each of these terms has specific implications as they are applied to the practice of research. These three principles arose as a response to many of the mistreatment and abuses that have been previously discussed and provide important guidance as researchers consider how they will construct and conduct their research studies. As you are crafting your research proposal, makes sure you are mindful of these important ethical guidelines.

Respect for Persons

As social workers, our professional code of ethics requires that we recognize and respect the "inherent dignity and worth of the person." [35] This is very similar to the ethical research principle of r espect for persons . According to this principle, as researchers, we need to treat all research participants with respect, dignity and inherent autonomy. This is reflected by ensuring that participants have self-determination to make informed decisions about their participation in research, that they have a clear understanding of what they will be asked to do and any risks involved, and that their participation is voluntary and can be stopped at any time. Furthermore, for those persons who may have diminished autonomy (e.g. children, people who are incarcerated), extra protections must be built in to these research studies to ensure that respect for persons continues to be demonstrated towards these groups, as they may be especially vulnerable to exploitation and coercion through the research process. A critical tool in establishing respect for persons in your research is the informed consent process, which will be discussed in more detail below.

Beneficence

You may not be familiar with this word yet, but the concept is pretty straightforward. The main idea with beneficence is that the intent of research is to do good. As researchers, to accomplish this, we seek to maximize benefits and minimize risks. Benefits may be something good or advantageous directly received by the research participant, or they may represent a broader good to a wider group of people or the scientific community at large (such as increasing knowledge about the topic or social problem that you are studying). Risks are potential physical, social, or emotional harm that may come about as a response to participation in a study. These risks may be more immediate (e.g. risk of identifying information about a participant being shared, or a participant being upset or triggered by a particular question), or long-term (e.g. some aspect about the person could be shared that could lead to long-term stigmatization). As researchers, we need to think about risk that might be experienced by the individual, but also risks that might be directed towards the community or population(s) the individual may represent. For instance, if our study is specifically focused on surveying single parents, we need to consider how the sharing of our findings might impact this group and how they are perceived. It is a very rare study in which there is no risk to participants. However, a well-designed and ethically sound study will seek to minimize these risks, provide resources to anticipate and address them, and maximize the benefits that are gained through the study.

The final ethical principle we need to cover is justice. While you likely have some idea what justice is, for the purposes of research, justice is the idea that the benefits and the burdens of research are distributed fairly across populations and groups. To help illustrate the concept of justice in research, research in the area of mental health and psychology has historically been critiqued as failing to adequately represent women and people of diverse racial and ethnic groups in their samples (Cundiff, 2012). [36] This has created a body of knowledge that is overly representative of the white male experience, further reinforcing systems of power and privilege. In addition, consider the influence of language as it relates to research justice. If we create studies that only recruit participants fluent in English, which many studies do, we are often failing to satisfy the ethical principle of justice as it applies to people who don't speak English. It is unrealistic to think that we can represent all people in all studies. However, we do need to thoughtfully acknowledge voices that might not be reflected in our samples and attempt to recruit diverse and representative samples whenever possible.

These three principles provide the foundation for the oversight work that is carried out by Institutional Review Boards, our next topic.

Institutional review boards

Institutional review boards, or IRBs, are tasked with ensuring that the rights and welfare of human research subjects will be protected at all institutions, including universities, hospitals, nonprofit research institutions, and other organizations, that receive federal support for research. IRBs typically consist of members from a variety of disciplines, such as sociology, economics, education, social work, and communications (to name a few). Most IRBs also include representatives from the community in which they reside. For example, representatives from nearby prisons, hospitals, or treatment centers might sit on the IRBs of university campuses near them. The diversity of membership helps to ensure that the many and complex ethical issues that may arise from human subjects research will be considered fully and by a knowledgeable and experienced panel. Investigators conducting research on human subjects are required to submit proposals outlining their research plans to IRBs for review and approval prior to beginning their research. Even students who conduct research on human subjects must have their proposed work reviewed and approved by the IRB before beginning any research (though, on some campuses, exceptions are made for student projects that will not be shared outside of the classroom).

The IRB has three levels of review, defined in statute by the USDHHS.

Exempt review is the lowest level of review. Studies that are considered exempt expose participants to the least potential for harm and often involve little participation by human subjects. In social work, exempt studies often examine data that is publicly available or secondary data from another researcher that has been de-identified by the person who collected it.

Expedited review is the middle level of review. Studies considered under expedited review do not have to go before the full IRB board because they expose participants to minimal risk. However, the studies must be thoroughly reviewed by a member of the IRB committee. While there are many types of studies that qualify for expedited review, the most relevant to social workers include the use of existing medical records, recordings (such as interviews) gathered for research purposes, and research on individual group characteristics or behavior.

Finally, the highest level of review is called a  full board review . A full board review will involve multiple members of the IRB evaluating your proposal. When researchers submit a proposal under full board review, the full IRB board will meet, discuss any questions or concerns with the study, invite the researcher to answer questions and defend their proposal, and vote to approve the study or send it back for revision. Full board proposals pose greater than minimal risk to participants. They may also involve the participation of  vulnerable populations , or people who need additional protection from the IRB. Vulnerable populations include prisoners, children, people with cognitive impairments, people with physical disabilities, employees, and students. While some of these populations can fall under expedited review in some cases, they will often require the full IRB to approve their study.

It may surprise you to hear that IRBs are not always popular or appreciated by researchers. Who wouldn’t want to conduct ethical research, you ask? In some cases, the concern is that IRBs are most well-versed in reviewing biomedical and experimental research, neither of which is particularly common within social work. Much social work research, especially qualitative research, is open-ended in nature, a fact that can be problematic for IRBs. The members of IRBs often want to know in advance exactly who will be observed, where, when, and for how long, whether and how they will be approached, exactly what questions they will be asked, and what predictions the researcher has for their findings. Providing this level of detail for a year-long participant observation within an activist group of 200-plus members, for example, would be extraordinarily frustrating for the researcher in the best case and most likely would prove to be impossible. Of course, IRBs do not intend to have researchers avoid studying controversial topics or avoid using certain methodologically sound data collection techniques, but unfortunately, that is sometimes the result. The solution is not to eradicate review boards, which serve a necessary and important function, but instead to help educate IRB members about the variety of social scientific research methods and topics covered by social workers and other social scientists.

What we have provided here is only a short summary of federal regulations and international agreements that provide the boundaries between ethical and unethical research.

Here are a few more detailed guides for continued learning about research ethics and human research protections.

• University of California, San Francisco: Levels of IRB Review
• United States Department of Health and Human Services: The Belmont Report
• NIH, National Institute of Environmental Health Sciences: What is Ethics in Research & Why is it important 
• NIH: Guiding Principles for Ethical Research
• Council on Social Work Education: National Statement on Research Integrity in Social Work
• Butler, I. (2002). A code of ethics for social work and social care research.  British Journal of Social Work ,  32 (2), 239-248
• Research on human subjects presents a unique set of challenges and opportunities when it comes to conducting ethical research.
• Research on human subjects has not always been regulated to the extent that it is today.
• All institutions receiving federal support for research must have an IRB. Organizations that do not receive federal support but where research is conducted also often include IRBs as part of their organizational structure.
• Researchers submit studies for IRB review at one of three different levels, depending on the level of harm the study may cause.
• Recall whether your project will gather data from human subjects and sketch out what the data collection process might look like.
• Identify which level of IRB review is most appropriate for your project.
• For many students, your professors may have existing agreements with your university's IRB that allow students to conduct research projects outside the supervision of the IRB. Make sure that your project falls squarely within those parameters. If you feel you may be outside of such an agreement, consult with your professor to see if you will need to submit your study for IRB review before starting your project.

6.2 Specific ethical issues to consider

• Define informed consent, and describe how it works
• Identify the unique concerns related to the study of vulnerable populations
• Differentiate between anonymity and confidentiality
• Explain the ethical responsibilities of social workers conducting research

As should be clear by now, conducting research on humans presents a number of unique ethical considerations. Human research subjects must be given the opportunity to consent to their participation in research, and be fully informed of the study’s risks, benefits, and purpose. Further, subjects’ identities and the information they share should be protected by researchers. Of course, how consent and identity protection are defined may vary by individual researcher, institution, or academic discipline. In this section, we’ll take a look at a few specific topics that individual researchers must consider before embarking on research with human subjects.

Informed consent

An expectation of voluntary participation is presumed in all social work research projects. In other words, we cannot force anyone to participate in our research without that person’s knowledge or consent. Researchers must therefore design procedures to obtain subjects’ informed consent to participate in their research. This specifically relates back to the ethical principle of respect for persons outlined in The Belmont Report . Informed consent  is defined as a subject’s voluntary agreement to participate in research based on a full understanding of the research and of the possible risks and benefits involved. Although it sounds simple, ensuring that one has actually obtained informed consent is a much more complex process than you might initially presume.

The first requirement is that, in giving their informed consent, subjects may neither waive nor even  appear  to waive any of their legal rights. Subjects also cannot release a researcher, her sponsor, or institution from any legal liability should something go wrong during the course of their participation in the research (USDHHS,2009). [37] Because social work research does not typically involve asking subjects to place themselves at risk of physical harm by, for example, taking untested drugs or consenting to new medical procedures, social work researchers do not often worry about potential liability associated with their research projects. However, their research may involve other types of risks.

For example, what if a social work researcher fails to sufficiently conceal the identity of a subject who admits to participating in a local swinger’s club? In this case, a violation of confidentiality may negatively affect the participant’s social standing, marriage, custody rights, or employment. Social work research may also involve asking about intimately personal topics that may be difficult for participants to discuss, such as trauma or suicide. Participants may re-experience traumatic events and symptoms when they participate in your study. Even if you are careful to fully inform your participants of all risks before they consent to the research process, I’m sure you can empathize with thinking you could bear talking about a difficult topic and then finding it too overwhelming once you start. In cases like these, it is important for a social work researcher to have a plan to provide supports. This may mean providing referrals to counseling supports in the community or even calling the police if the participant is an imminent danger to himself or others.

It is vital that social work researchers explain their mandatory reporting duties in the consent form and ensure participants understand them before they participate. Researchers should also emphasize to participants that they can stop the research process at any time or decide to withdraw from the research study for any reason. Importantly, it is not the job of the social work researcher to act as a clinician to the participant. While a supportive role is certainly appropriate for someone experiencing a mental health crisis, social workers must ethically avoid dual roles. Referring a participant in crisis to other mental health professionals who may be better able to help them is the expectation.

Beyond the legal issues, most IRBs require researchers to share some details about the purpose of the research, possible benefits of participation, and, most importantly, possible risks associated with participating in that research with their subjects. In addition, researchers must describe how they will protect subjects’ identities, how, where, and for how long any data collected will be stored, how findings may be shared, and whom to contact for additional information about the study or about subjects’ rights. All this information is typically shared in an informed consent form that researchers provide to subjects. In some cases, subjects are asked to sign the consent form indicating that they have read it and fully understand its contents. In other cases, subjects are simply provided a copy of the consent form and researchers are responsible for making sure that subjects have read and understand the form before proceeding with any kind of data collection. Your IRB will often provide guidance or even templates for what they expect to see included in an informed consent form. This is a document that they will inspect very closely. Table 6.1 outlines elements to include in your informed consent. While these offer a guideline for you, you should always visit your schools, IRB website to see what guidance they offer. They often provide a template that they prefer researchers to use. Using these templates ensures that you are using the language that the IRB reviewers expect to see and this can also save you time.

One last point to consider when preparing to obtain informed consent is that not all potential research subjects are considered equally competent or legally allowed to consent to participate in research. Subjects from vulnerable populations may be at risk of experiencing undue influence or coercion (USDHHS, 2009). [38] The rules for consent are more stringent for vulnerable populations. For example, minors must have the consent of a legal guardian in order to participate in research. In some cases, the minors themselves are also asked to participate in the consent process by signing special, age-appropriate assent forms designed specifically for them. Prisoners and parolees also qualify as vulnerable populations. Concern about the vulnerability of these subjects comes from the very real possibility that prisoners and parolees could perceive that they will receive some highly desired reward, such as early release, if they participate in research or that there could be punitive consequences if they choose not to participate. When a participant faces undue or excess pressure to participate by either favorable or unfavorable means, this is known as coercion and must be avoided by researchers.

Another potential concern regarding vulnerable populations is that they may be underrepresented or left out of research opportunities, specifically because of concerns about their ability to consent. So, on the one hand, researchers must take extra care to ensure that their procedures for obtaining consent from vulnerable populations are not coercive. The procedures for receiving approval to conduct research with these groups may be more rigorous than that for non-vulnerable populations. On the other hand, researchers must work to avoid excluding members of vulnerable populations from participation simply on the grounds that they are vulnerable or that obtaining their consent may be more complex. While there is no easy solution to this ethical research dilemma, an awareness of the potential concerns associated with research on vulnerable populations is important for identifying whatever solution is most appropriate for a specific case.

Protection of identities

As mentioned earlier, the informed consent process includes the requirement that researchers outline how they will protect the identities of subjects. This aspect of the research process, however, is one of the most commonly misunderstood. Furthermore, failing to protect identities is one of the greatest risks to participants in social work research studies.

In protecting subjects’ identities, researchers typically promise to maintain either the anonymity or confidentiality of their research subjects. These are two distinctly different terms and they are NOT interchangeable. Anonymity is the more stringent of the two and is very hard to guarantee in most research studies. When a researcher promises anonymity to participants, not even the researcher is able to link participants’ data with their identities. Anonymity may be impossible for some social work researchers to promise due to the modes of data collection many social workers employ. Face-to-face interviewing means that subjects will be visible to researchers and will hold a conversation, making anonymity impossible. In other cases, the researcher may have a signed consent form or obtain personal information on a survey and will therefore know the identities of their research participants. In these cases, a researcher should be able to at least promise confidentiality to participants.

Offering  confidentiality means that some identifying information is known at some time by the research team, but only the research team has access to this identifying information and this information will not be linked with their data in any publicly accessible way. Confidentiality in research is quite similar to confidentiality in clinical practice. You know who your clients are, but others do not. You agree to keep their information and identity private. As you can see under the “Risks” section of the consent form in Figure 5.1, sometimes it is not even possible to promise that a subject’s confidentiality will be maintained. This is the case if data are collected in public or in the presence of other research participants in the course of a focus group, for example. Participants who social work researchers deem to be of imminent danger to self or others or those that disclose abuse of children and other vulnerable populations fall under a social worker’s duty to report. Researchers must then violate confidentiality to fulfill their legal obligations.

There are a number of steps that researchers can take to protect the identities of research participants. These include, but are not limited to:

• Collecting data in private spaces
• Not requesting information that will uniquely identify or "out" that person as a participant
• Assigning study identification codes or pseudonyms
• Keeping signed informed consent forms separate from other data provided by the participant
• Making sure that physical data is kept in a locked and secured location, and the virtual data is encrypted or password-protected
• Reporting data in aggregate (only discussing the data collectively, not by individual responses)

Protecting research participants’ identities is not always a simple prospect, especially for those conducting research on stigmatized groups or illegal behaviors. Sociologist Scott DeMuth learned that all too well when conducting his dissertation research on a group of animal rights activists. As a participant observer, DeMuth knew the identities of his research subjects. So when some of his research subjects vandalized facilities and removed animals from several research labs at the University of Iowa, a grand jury called on Mr. DeMuth to reveal the identities of the participants in the raid. When DeMuth refused to do so, he was jailed briefly and then charged with conspiracy to commit animal enterprise terrorism and cause damage to the animal enterprise (Jaschik, 2009). [39]

Publicly, DeMuth’s case raised many of the same questions as Laud Humphreys’ work 40 years earlier. What do social scientists owe the public? Is DeMuth, by protecting his research subjects, harming those whose labs were vandalized? Is he harming the taxpayers who funded those labs? Or is it more important that DeMuth emphasize what he owes his research subjects, who were told their identities would be protected? DeMuth’s case also sparked controversy among academics, some of whom thought that as an academic himself, DeMuth should have been more sympathetic to the plight of the faculty and students who lost years of research as a result of the attack on their labs. Many others stood by DeMuth, arguing that the personal and academic freedom of scholars must be protected whether we support their research topics and subjects or not. DeMuth’s academic adviser even created a new group, Scholars for Academic Justice , to support DeMuth and other academics who face persecution or prosecution as a result of the research they conduct. What do you think? Should DeMuth have revealed the identities of his research subjects? Why or why not?

Discipline-specific considerations

Often times, specific disciplines will provide their own set of guidelines for protecting research subjects and, more generally, for conducting ethical research. For social workers, the National Association of Social Workers (NASW) Code of Ethics section 5.02 describes the responsibilities of social workers in conducting research. Summarized below, these responsibilities are framed as part of a social worker’s responsibility to the profession. As representative of the social work profession, it is your responsibility to conduct and use research in an ethical manner.

A social worker should:

• Monitor and evaluate policies, programs, and practice interventions
• Contribute to the development of knowledge through research
• Keep current with the best available research evidence to inform practice
• Ensure voluntary and fully informed consent of all participants
• Not engage in any deception in the research process
• Allow participants to withdraw from the study at any time
• Provide access to appropriate supportive services for participants
• Protect research participants from harm
• Maintain confidentiality
• Report findings accurately
• Disclose any conflicts of interest
• Researchers must obtain the informed consent of research participants.
• Social workers must take steps to minimize the harms that could arise during the research process.
• If anonymity is promised, individual participants cannot be linked with their data.
• If confidentiality is promised, the identities of research participants cannot be revealed, even if individual participants can be linked with their data.
• The NASW Code of Ethics includes specific responsibilities for social work researchers.
• Talk with your professor to see if an informed consent form is required for your research project. If documentation is required, customize the template provided by your professor or the IRB at your school, using the details of your study. If documentation on consent is not required, for example if consent is given verbally, use the templates as guides to create a guide for what you will say to participants regarding informed consent.
• Identify whether your data will be confidential or anonymous. Describe the measures you will take to protect the identities of individuals in your study. How will you store the data? How will you ensure that no one can identify participants based on what you report in papers and presentations? Be sure to think carefully. People can be identified by characteristics such as age, gender, disability status, location, etc.

6.3 Benefits and harms of research across the ecosystem

• Identify and distinguish between micro-, mezzo-, and macro-level considerations with respect to the ethical conduct of social scientific research

This chapter began with a long list of harmful acts that researchers engaged in while conducting studies on human subjects. Indeed, even the last section on informed consent and protection of confidential information can be seen in light of minimizing harm and maximizing benefits. The benefits of your study should be greater than the harms. But who benefits from your research study, and who might be harmed? The first person who benefits is, most clearly, you as the researcher. You need a project to complete, be it for a grade, a grant, an academic responsibility, etc. However you need to make sure that your benefit does not come at the expense of harming others. Furthermore, research requires resources, including resources from the communities we work with. Part of being good stewards of these resources as social work researchers means that we need to engage in research that benefits the people we serve in meaningful and relevant ways. We need to consider how others are impacted by our research.

Micro-, mezzo-, and macro-level concerns

One useful way to think about the breadth of ethical questions that might arise out of any research project is to think about potential issues from the perspective of different analytical levels that are familiar to us as social workers. In Chapter 1 , you learned about the micro-, mezzo-, and macro-levels of inquiry and how a researcher’s specific point of focus might vary depending on her level of inquiry. Here we’ll apply this ecological framework to a discussion of research ethics. Within most research projects, there are specific questions that arise for researchers at each of these three levels.

At the micro-level, researchers must consider their own conduct and the impact on individual research participants. For example, did Stanley Milgram behave ethically when he allowed research participants to think that they were administering electric shocks to fellow participants? Did Laud Humphreys behave ethically when he deceived his research subjects about his own identity? Were the rights of individuals in these studies protected? How did these participants benefit themselves from the research that was conducted? While not social workers by trade, would the actions of these two researchers hold up against our professional NASW Code of Ethics? The questions posed here are the sort that you will want to ask yourself as a researcher when considering ethics at the micro-level.

At the mezzo-level, researchers should think about their duty to the community. How will the results of your study impact your target population? Ideally, your results will benefit your target population by identifying important areas for social workers to intervene and to better understand the experiences of the communities they serve. However, it is possible that your study may perpetuate negative stereotypes about your target population or damage its reputation. Indigenous people in particular have highlighted how historically social science has furthered marginalization of indigenous peoples (Smith, 2013). [40] Mezzo-level concerns should also address other groups or organizations that are connected to your target population. This may include the human service agencies with whom you've partnered for your study as well as the communities and peoples they serve. If your study reflected negatively on a particular housing project in your area, for example, will community members seek to remove it from their community? Or might it draw increased law enforcement presence that is unwanted by participants or community members? Research is a powerful tool and can be used for many purposes, not all of them altruistic. In addition, research findings can have many implications, intended and unintended. As responsible researchers, we need to do our best to thoughtfully anticipate these consequences.

Finally, at the macro-level, a researcher should consider duty to, and the expectations of, society. Perhaps the most high-profile case involving macro-level questions of research ethics comes from debates over whether to use data gathered by, or cite published studies based on data gathered from, the Nazis in the course of their unethical and horrendous experiments on humans during World War II (Moe, 1984). [41] Some argue that because the data were gathered in such an unquestionably unethical manner, they should never be used. The data, say these people, are neither valid nor reliable and should therefore not be used in any current scientific investigation (Berger, 1990). [42]

On the other hand, some people argue that data themselves are neutral; that “information gathered is independent of the ethics of the methods and that the two are not linked together” (Pozos, 1992, p. 104). [43] Others point out that not using the data could inadvertently strengthen the claims of those who deny that the Holocaust ever happened. In his striking statement in support of publishing the data, medical ethics professor Velvl Greene (1992) says,

Instead of banning the Nazi data or assigning it to some archivist or custodial committee, I maintain that it be exhumed, printed, and disseminated to every medical school in the world along with the details of methodology and the names of the doctors who did it, whether or not they were indicted, acquitted, or hanged.…Let the students and the residents and the young doctors know that this was not ancient history or an episode from a horror movie where the actors get up after filming and prepare for another role. It was real. It happened yesterday (p. 169–170). [44]

While debates about the use of data collected by the Nazis are typically centered on medical scientists’ use of them, there are conceivable circumstances under which these data might be used by social scientists. Perhaps, for example, a social scientist might wish to examine contemporary reactions to the experiments. Or perhaps the data could be used in a study of the sociology of science. What do you think? Should data gathered by the Nazis be used or cited today? What arguments can you make in support of your position, and how would you respond to those who disagree?

Additionally at the macro-level, you must also consider your responsibilities to the profession of social work. When you engage in social work research, you stand on the reputation the profession has built for over a century. Since research is public-facing, meaning that research findings are intended to be shared publicly, you are an ambassador for the profession. How you conduct yourself as a social work researcher has potential implications for how the public perceives both social work and research. As a social worker, you have a responsibility to work towards greater social, environmental, and economic justice and human rights. Your research should reflect this responsibility. Attending to research ethics helps to fulfill your responsibilities to the profession, in addition to your target population.

Table 6.2 summarizes the key questions that researchers might ask themselves about the ethics of their research at each level of inquiry.

• At the micro-level, researchers should consider their own conduct and the rights of individual research participants.
• At the mezzo-level, researchers should consider the expectations of their profession, any organizations that may have funded their research, and the communities affected by their research.
• At the macro-level, researchers should consider their duty to and the expectations of society with respect to social science research.
• Summarize the benefits and harms at the micro-, mezzo-, and macro-level of inquiry. At which level of inquiry is your research project?
• In a few sentences, identify whether the benefits of your study outweigh the potential harms.

6.4 Being an ethical researcher

• Identify why researchers must provide a detailed description of methodology
• Describe what it means to use science in an ethical way

Research ethics has to do with both how research is conducted and how findings from that research are used. In this section, we’ll consider research ethics from both angles.

Doing science the ethical way

As you should now be aware, researchers must consider their own personal ethical principles in addition to following those of their institution, their discipline, and their community. We’ve already considered many of the ways that social workers strive to ensure the ethical practice of research, such as informing and protecting subjects. But the practice of ethical research doesn’t end once subjects have been identified and data have been collected. Social workers must also fully disclose their research procedures and findings. This means being honest about how research subjects were identified and recruited, how exactly data were collected and analyzed, and ultimately, what findings were reached.

If researchers fully disclose how they conducted their research, then those who use their work to build research projects, create social policies, or make treatment decisions can have greater confidence in the work. By sharing how research was conducted, a researcher helps assure readers they have conducted legitimate research and didn’t simply come to whatever conclusions they wanted   to find. A description or presentation of research findings that is not accompanied by information about research methodology is missing relevant information. Sometimes methodological details are left out because there isn’t time or space to share them. This is often the case with news reports of research findings. Other times, there may be a more insidious reason that important information is missing. This may be the case if sharing methodological details would call the legitimacy of a study into question. As researchers, it is our ethical responsibility to fully disclose our research procedures. As consumers of research, it is our ethical responsibility to pay attention to such details. We’ll discuss this more in the next section.

There’s a New Yorker cartoon that depicts a set of filing cabinets that aptly demonstrates what we don’t want to see happen with research. Each filing cabinet drawer in the cartoon is labeled differently. The labels include such headings as, “Our Facts,” “Their Facts,” “Neutral Facts,” “Disputable Facts,” “Absolute Facts,” “Bare Facts,” “Unsubstantiated Facts,” and “Indisputable Facts.” The implication of this cartoon is that one might just choose to open the file drawer of her choice and pick whichever facts one likes best. While this may occur if we use some of the unscientific ways of knowing described in Chapter 1 , it is fortunately not how the discovery of knowledge in social work, or in any other science for that matter, takes place. There actually is a method to this madness we call research. At its best, research reflects a systematic, transparent, informative process.

Honesty in research is facilitated by the scientific principle of replication . Ideally, this means that one scientist could repeat another’s study with relative ease. By replicating a study, we may become more (or less) confident in the original study’s findings. Replication is far more difficult (perhaps impossible) to achieve in the case of many qualitative studies, as our purpose is often a deep understanding of very specific circumstances, rather than the broad, generalizable knowledge we traditionally seek in quantitative studies. Nevertheless, transparency in the research process is an important standard for all social scientific researchers—that we provide as much detail as possible about the processes by which we reach our conclusions. This allows the quality of our research to be evaluated. Along with replication, peer review is another important principle of the scientific process. Peer review involves other knowledgeable researchers in our field of study to evaluate our research and to determine if it is of sufficient quality to share with the public. There are valid critiques of the peer review process: that it is biased towards studies with positive findings, that it may reinforce systemic barriers to oppressed groups accessing and leveraging knowledge, that it is far more subjective and/or unreliable than it claims to be. Despite these critiques, peer review remains a foundational concept for how scientific knowledge is generated.

Full disclosure also includes the need to be honest about a study’s strengths and weaknesses, both with oneself and with others. Being aware of the strengths and weaknesses of your own work can help a researcher make reasonable recommendations about the next steps other researchers might consider taking in their inquiries. Awareness and disclosure of a study’s strengths and weaknesses can also help highlight the theoretical or policy implications of one’s work. In addition, openness about strengths and weaknesses helps those reading the research better evaluate the work and decide for themselves how or whether to rely on its findings. Finally, openness about a study’s sponsors is crucial. How can we effectively evaluate research without knowing who paid the bills? This allows us to assess for potential conflicts of interest that may compromise the integrity of the research.

The standard of replicability, the peer-review process, and openness about a study’s strengths, weaknesses, and funding sources enables those who read the research to evaluate it fairly and completely. Knowledge of funding sources is often raised as an issue in medical research. Understandably, independent studies of new drugs may be more compelling to the Food and Drug Administration (FDA) than studies touting the virtues of a new drug that happen to have been funded by the company who created that drug. But medical researchers aren’t the only ones who need to be honest about their funding. If we know, for example, that a political think tank with ties to a particular party has funded some research, we can take that knowledge into consideration when reviewing the study’s findings and stated policy implications. Lastly, and related to this point, we must consider how, by whom, and for what purpose research may be used.

Using science the ethical way

Science has many uses. By “use” I mean the ways that science is understood and applied (as opposed to the way it is conducted). Some use science to create laws and social policies; others use it to understand themselves and those around them. Some people rely on science to improve their life conditions or those of other people, while still others use it to improve their businesses or other undertakings. In each case, the most ethical way for us to use science is to educate ourselves about the design and purpose of any studies we may wish to use. This helps us to more adequately critique the value of this research, to recognize its strengths and limitations.

As part of my research course, students are asked to critique a research article. I often find in this assignment that students often have very lofty expectations for everything that 'should' be included in the journal article they are reviewing. While I appreciate the high standards, I often give them feedback that it is perhaps unrealistic (even unattainable) for a research study to be perfectly designed and described for public consumption. All research has limitations; this may be a consequence of limited resources, issues related to feasibility, and unanticipated roadblocks or problems as we are carrying out our research. Furthermore, the ways we disseminate or share our research often has restrictions on what and how we can share our findings. This doesn't mean that a study with limitations has no value—every study has limitations! However, as we are reviewing research, we should look for an open discussion about methodology , strengths, and weaknesses of the study that helps us to interpret what took place and in what ways it may be important.

For instance, this can be especially important to think about in terms of a study's sample. It can be challenging to recruit a diverse and representative sample for your study (however, that doesn't mean we shouldn't try!). The next time you are reading research studies that were used to help establish an evidence based practice (EBP), make sure to look at the description of the sample. We cannot assume that what works for one group of people will uniformly work with all groups of people with very different life experiences; however, historically much of our intervention repertoire has been both created by and evaluated on white men. If research studies don't obtain a diverse sample, for whatever reason, we would expect that the authors would identify this as a limitation and an area requiring further study. We need to challenge our profession to provide practices, strategies, models, interventions, and policies that have been evaluated and tested for their efficacy with the diverse range of people that we work with as social workers.

Social scientists who conduct research on behalf of organizations and agencies may face additional ethical questions about the use of their research, particularly when the organization for which a study is conducted controls the final report and the publicity it receives. There is a potential conflict of interest for evaluation researchers who are employees of the agency being evaluated. A similar conflict of interest might exist between independent researchers whose work is being funded by some government agency or private foundation.

So who decides what constitutes ethical conduct or use of research? Perhaps we all do. What qualifies as ethical research may shift over time and across cultures as individual researchers, disciplinary organizations, members of society, and regulatory entities, such as institutional review boards, courts, and lawmakers, all work to define the boundaries between ethical and unethical research.

• Conducting research ethically requires that researchers be ethical not only in their data collection procedures but also in reporting their methods and findings.
• The ethical use of research requires an effort to understand research, an awareness of your own limitations in terms of knowledge and understanding, and the honest application of research findings.
• Think about your research hypothesis at this point. What would happen if your results revealed information that could harm the population you are studying? What are your ethical responsibilities as far as reporting about your research?
• Ultimately, we cannot control how others will use the results of our research. What are the implications of this for how you report on your research?
• Reading the results of empirical studies (16 minute read)
• Annotating empirical journal articles (15 minute read)
• Generalizability and transferability of empirical results (15 minute read)

Content warning: examples in this chapter contain references to domestic violence and details on types of abuse, drug use, poverty, mental health, sexual harassment and details on harassing behaviors, children’s mental health, LGBTQ+ oppression and suicide, obesity, anti-poverty stigma, and psychotic disorders.

5.1 Reading the results of empirical studies

• Describe how statistical significance and confidence intervals demonstrate which results are most important
• Differentiate between qualitative and quantitative results in an empirical journal article

If you recall from section 3.1 , empirical journal articles are those that report the results of quantitative or qualitative data analyzed by the author. They follow a set structure—introduction, methods, results, discussion/conclusions. This section is about reading the most challenging section: results.

I want to normalize not understanding statistics terms and symbols. However, a basic understanding of a results section goes a very long way to understanding the key results in an article. This will take you beyond the two or three sentences in the abstract that summarize the study's results and into the nitty-gritty of what they found for each concept they studied.

Read beyond the abstract

At this point, I have read hundreds of literature reviews written by students. One of the challenges I have noted is that students will report the results as summarized in the abstract, rather than the detailed findings laid out in the results section of the article. This poses a problem when you are writing a literature review because you need to provide specific and clear facts that support your reading of the literature. The abstract may say something like: “we found that poverty is associated with mental health status.” For your literature review, you want the details, not the summary. In the results section of the article, you may find a sentence that states: “children living in households experiencing poverty are three times more likely to have a mental health diagnosis.” This more specific statistical information provides a stronger basis on which to build the arguments in your literature review.

Using the summarized results in an abstract is an understandable mistake to make. The results section often contains figures and tables that may be challenging to understand. Often, without having completed more advanced coursework on statistical or qualitative analysis, some of the terminology, symbols, or diagrams may be difficult to comprehend. This section is all about how to read and interpret the results of an empirical (quantitative or qualitative) journal article. Our discussion here will be basic, and in parts three and four of the textbook, you will learn more about how to interpret results from statistical tests and qualitative data analysis.

Remember, this section only addresses empirical articles. Non-empirical articles (e.g., theoretical articles, literature reviews) don't have results. They cite the analysis of raw data completed by other authors, not the person writing the journal article who is merely summarizing others' work.

Quantitative results

Quantitative articles often contain tables, and scanning them is a good way to begin reading the results. A table usually provides a quick, condensed summary of the report’s key findings. Tables are a concise way to report large amounts of data. Some tables present descriptive information about a researcher’s sample (often the first table in a results section). These tables will likely contain frequencies (N) and percentages (%). For example, if gender happened to be an important variable for the researcher’s analysis, a descriptive table would show how many and what percent of all study participants are of a particular gender. Frequencies or “how many” will probably be listed as N, while the percent symbol (%) might be used to indicate percentages.

In a table presenting a causal relationship, two sets of variables are represented. The independent variable , or cause, and the dependent variable , the effect. We will discuss these further when we review quantitative conceptualization and measurement. Independent variable attributes are typically presented in the table’s columns, while dependent variable attributes are presented in rows. This allows the reader to scan a table’s rows to see how values on the dependent variable change as the independent variable values change (i.e., changes in the dependent variable depend on changes in the independent variable). Tables displaying results of quantitative analysis will also likely include some information about which relationships are significant or not. We will discuss the details of significance and p-values later in this section.

Let’s look at a specific example: Table 5.1. It presents the causal relationship between gender and experiencing harassing behaviors at work. In this example, gender is the independent variable (the cause) and the harassing behaviors listed are the dependent variables (the effects). [46] Therefore, we place gender in the table’s columns and harassing behaviors in the table’s rows.

Reading across the table’s top row, we see that 2.9% of women in the sample reported experiencing subtle or obvious threats to their safety at work, while 4.7% of men in the sample reported the same. We can read across each of the rows of the table in this way. Reading across the bottom row, we see that 9.4% of women in the sample reported experiencing staring or invasion of their personal space at work while just 2.3% of men in the sample reported having the same experience. We’ll discuss  p values later in this section.

While you can certainly scan tables for key results, they are often difficult to understand without reading the text of the article. The article and table were meant to complement each other, and the text should provide information on how the authors interpret their findings. The table is not redundant with the text of the results section. Additionally, the first table in most results sections is a summary of the study's sample, which provides more background information on the study than information about hypotheses and findings. It is also a good idea to look back at the methods section of the article as the data analysis plan the authors outline should walk you through the steps they took to analyze their data which will inform how they report them in the results section.

Statistical significance

The statistics reported in Table 5.1 represent what the researchers found in their sample. The purpose of statistical analysis is usually to generalize from a the small number of people in a study's sample to a larger population of people. Thus, the researchers intend to make causal arguments about harassing behaviors at workplaces beyond those covered in the sample.

Generalizing is key to understanding statistical significance . According to Cassidy and colleagues, (2019) [47] 89% of research methods textbooks in psychology define statistical significance incorrectly. This includes an early draft of this textbook which defined statistical significance as "the likelihood that the relationships we observe could be caused by something other than chance." If you have previously had a research methods class, this might sound familiar to you. It certainly did to me!

But statistical significance is less about "random chance" than more about the null hypothesis . Basically, at the beginning of a study a researcher develops a hypothesis about what they expect to find, usually that there is a statistical relationship between two or more variables . The null hypothesis is the opposite. It is the hypothesis that there is no relationship between the variables in a research study. Researchers then can hopefully reject the null hypothesis because they find a relationship between the variables.

For example, in Table 5.1 researchers were examining whether gender impacts harassment. Of course, researchers assumed that women were more likely to experience harassment than men. The null hypothesis, then, would be that gender has no impact on harassment. Once we conduct the study, our results will hopefully lead us to reject the null hypothesis because we find that gender impacts harassment. We would then generalize from our study's sample to the larger population of people in the workplace.

Statistical significance is calculated using a p-value which is obtained by comparing the statistical results with a hypothetical set of results if the researchers re-ran their study a large number of times. Keeping with our example, imagine we re-ran our study with different men and women from different workplaces hundreds and hundred of times and we assume that the null hypothesis is true that gender has no impact on harassment. If results like ours come up pretty often when the null hypothesis is true, our results probably don't mean much. "The smaller the p-value, the greater the statistical incompatibility with the null hypothesis" (Wasserstein & Lazar, 2016, p. 131). [48] Generally, researchers in the social sciences have used 0.05 as the value at which a result is significant (p is less than 0.05) or not significant (p is greater than 0.05). The p-value 0.05 refers to if 5% of those hypothetical results from re-running our study show the same or more extreme relationships when the null hypothesis is true. Researchers, however, may choose a stricter standard such as 0.01 in which only 1% of those hypothetical results are more extreme or a more lenient standard like 0.1 in which 10% of those hypothetical results are more extreme than what was found in the study.

Let's look back at Table 5.1. Which one of the relationships between gender and harassing behaviors is statistically significant? It's the last one in the table, "staring or invasion of personal space," whose p-value is 0.039 (under the p<0.05 standard to establish statistical significance). Again, this indicates that if we re-ran our study over and over again and gender did not  impact staring/invasion of space (i.e., the null hypothesis was true), only 3.9% of the time would we find similar or more extreme differences between men and women than what we observed in our study. Thus, we conclude that for staring or invasion of space only , there is a statistically significant relationship.

For contrast, let's look at "being pushed, hit, or grabbed" and run through the same analysis to see if it is statistically significant. If we re-ran our study over and over again and the null hypothesis was true, 48% of the time (p=.48) we would find similar or more extreme differences between men and women. That means these results are not statistically significant.

This discussion should also highlight a point we discussed previously: that it is important to read the full results section, rather than simply relying on the summary in the abstract. If the abstract stated that most tests revealed no statistically significant relationships between gender and harassment, you would have missed the detail on which behaviors were and were not associated with gender. Read the full results section! And don't be afraid to ask for help from a professor in understanding what you are reading, as results sections are often not written to be easily understood.

Statistical significance and p-values have been critiqued recently for a number of reasons, including that they are misused and misinterpreted (Wasserstein & Lazar, 2016) [49] , that researchers deliberately manipulate their analyses to have significant results (Head et al., 2015) [50] , and factor into the difficulty scientists have today in reproducing many of the results of previous social science studies (Peng, 2015). [51] For this reason, we share these principles, adapted from those put forth by the American Statistical Association, [52]  for understanding and using p-values in social science:

• P-values provide evidence against a null hypothesis.
• P-values do not indicate whether the results were produced by random chance alone or if the researcher's hypothesis is true, though both are common misconceptions.
• Statistical significance can be detected in minuscule differences that have very little effect on the real world.
• Nuance is needed to interpret scientific findings, as a conclusion does not become true or false when the p-value passes from p=0.051 to p=0.049.
• Real-world decision-making must use more than reported p-values. It's easy to run analyses of large datasets and only report the significant findings.
• Greater confidence can be placed in studies that pre-register their hypotheses and share their data and methods openly with the public.
• "By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis. For example, a p-value near 0.05 taken by itself offers only weak evidence against the null hypothesis. Likewise, a relatively large p-value does not imply evidence in favor of the null hypothesis; many other hypotheses may be equally or more consistent with the observed data" (Wasserstein & Lazar, 2016, p. 132).

Confidence intervals

Because of the limitations of p-values, scientists can use other methods to determine whether their models of the world are true. One common approach is to use a confidence interval , or a range of values in which the true value is likely to be found. Confidence intervals are helpful because, as principal #5 above points out, p-values do not measure the size of an effect (Greenland et al., 2016). [53] Remember, something that has very little impact on the world can be statistically significant, and the values in a confidence interval would be helpful. In our example from Table 5.1, imagine our analysis produced a confidence interval that women are 1.2-3.4x more likely to experience "staring or invasion of personal space" than men. As with p-values, calculation for a confidence interval compares what was found in one study with a hypothetical set of results if we repeated the study over and over again. If we calculated 95% confidence intervals for all of the hypothetical set of hundreds and hundreds of studies, that would be our confidence interval. 

Confidence intervals are pretty intuitive. As of this writing, my wife and are expecting our second child. The doctor told us our due date was December 11th. But the doctor also told us that December 11th was only their best estimate. They were actually 95% sure our baby might be born any time in the 30-day period between November 27th and December 25th. Confidence intervals are often listed with a percentage, like 90% or 95%, and a range of values, such as between November 27th and December 25th. You can read that as: "we are 95% sure your baby will be born between November 27th and December 25th because we've studied hundreds of thousands of fetuses and mothers, and we're 95% sure your baby will be within these two dates."

Notice that we're hedging our bets here by using words like "best estimate." When testing hypotheses, social scientists generally phrase their findings in a tentative way, talking about what results "indicate" or "support," rather than making bold statements about what their results "prove." Social scientists have humility because they understand the limitations of their knowledge. In a literature review, using a single study or fact to "prove" an argument right or wrong is often a signal to the person reading your literature review (usually your professor) that you may not have appreciated the limitations of that study or its place in the broader literature on the topic. Strong arguments in a literature review include multiple facts and ideas that span across multiple studies.

You can learn more about creating tables, reading tables, and tests of statistical significance in a class focused exclusively on statistical analysis. We provide links to many free and openly licensed resources on statistics in Chapter 16 . For now, we hope this brief introduction to reading tables will improve your confidence in reading and understanding the results sections in quantitative empirical articles.

Qualitative results

Quantitative articles will contain a lot of numbers and the results of statistical tests demonstrating associations between those numbers. Qualitative articles, on the other hand, will consist mostly of quotations from participants. For most qualitative articles, the authors want to put their results in the words of their participants, as they are the experts. Articles that lack quotations make it difficult to assess whether the researcher interpreted the data in a trustworthy, unbiased manner. These types of articles may also indicate how often particular themes or ideas came up in the data, potentially reflective of how important they were to participants.

Authors often organize qualitative results by themes and subthemes. For example, see this snippet from the results section in Bonanno and Veselak (2019) [54] discussion parents' attitudes towards child mental health information sources.

Data analysis revealed four themes related to participants’ abilities to access mental health help and information for their children, and parents’ levels of trust in these sources. These themes are: others’ firsthand experiences family and friends with professional experience, protecting privacy, and uncertainty about schools as information sources. Trust emerged as an overarching and unifying concept for all of these themes. Others’ firsthand experiences. Several participants reported seeking information from other parents who had experienced mental health struggles similar to their own children. They often referenced friends or family members who had been or would be good sources of information due to their own personal experiences. The following quote from Adrienne demonstrates the importance of firsthand experience: [I would only feel comfortable sharing concerns or asking for advice] if I knew that they had been in the same situation. (Adrienne) Similarly, Michelle said: And I talked to a friend of mine who has kids who have IEPs in the district to see, kind of, how did she go about it. (Michelle) ... Friends/family with professional experience . Several respondents referred to friends or family members who had professional experience with or knowledge of child mental health and suggested that these individuals would be good sources of information. For example, Hannah said: Well, what happened with me was I have an uncle who’s a psychiatrist. Sometimes if he’s up in (a city to the north), he’s retired, I can call him sometimes and get information. (Hannah) Michelle, who was in nursing school, echoed this sentiment: At this point, [if my child’s behavioral difficulties continued], I would probably call one of my [nursing] professors. That’s what I’ve done in the past when I’ve needed help with certain things...I have a professor who I would probably consider a friend who I would probably talk to first. She has a big adolescent practice. (Michelle) (p. 402-403)

The terms in bold above refer to the key themes (i.e., qualitative results) that were present in the data. Researchers will state the process by which they interpret each theme, providing a definition and usually some quotations from research participants. Researchers will also draw connections between themes, note consensus or conflict over themes, and situate the themes within the study context.

Qualitative results are specific to the time, place, and culture in which they arise, so you will have to use your best judgment to determine whether these results are relevant to your study. For example, students in my class at Radford University in Southwest Virginia may be studying rural populations. Would a study on group homes in a large urban city transfer well to group homes in a rural area?

Maybe. But even if you were using data from a qualitative study in another rural area, are all rural areas the same? How is the client population and sociocultural context in the article similar or different to the one in your study? Qualitative studies have tremendous depth, but researchers must be intentional about drawing conclusions about one context based on a study in another context. To make conclusions about how a study applies in another context, researchers need to examine each component of an empirical journal article--they need to annotate!

• The results section of empirical articles are often the most difficult to understand.
• To understand a quantitative results section, look for results that were statistically significant and examine the confidence interval, if provided.
• To understand a qualitative results section, look for definitions of themes or codes and use the quotations provided to understand the participants’ perspective.

Select a quantitative empirical article related to your topic.

• Write down the results the authors identify as statistically significant in the results section.
• How do the authors interpret their results in the discussion section?
• Do the authors provide enough information in the introduction for you to understand their results?

Select a qualitative empirical article relevant to your topic.

• Write down the key themes the authors identify and how they were defined by the participants.

5.2 Annotating empirical journal articles

• Define annotation and describe how to use it to identify, extract, and reflect on the information you need from an article

Annotation refers to the process of writing notes on an article. There are many ways to do this. The most basic technique is to print out the article and build a binder related to your topic. Raul Pacheco-Vega's excellent blog has a post on his approach to taking physical notes. Honestly, while you are there, browse around that website. It is full of amazing tips for students conducting a literature review and graduate research projects. I see a lot of benefits to the paper, pen, and highlighter approach to annotating articles. Personally though, I prefer to use a computer to write notes on an article because my handwriting is terrible and typing notes allows me search for keywords. For other students, electronic notes work best because they cannot afford to print every article that they will use in their paper. No matter what you use, the point is that you need to write notes when you're reading. Reading is research!

There are a number of free software tools you can use to help you annotate a journal article. Most PDF readers like Adobe Acrobat have a commenting and highlighting feature, though the PDF readers included with internet browsers like Google Chrome, Microsoft Edge, and Safari do not have this feature. The best approach may be to use a citation manager like Zotero. Using a citation manager, you can build a library of articles, save your annotations, and link annotations across PDFs using keywords. They also provide integration with word processing programs to help with citations in a reference list

Of course, I don't follow this advice because I have a system that works well for me. I have a PDF open in one computer window and a Word document open in a window next to it. I type notes and copy quotes, listing the page number for each note I take. It's a bit low-tech, but it does make my notes searchable. This way, when I am looking for a concept or quote, I can simply search my notes using the Find feature in Word and get to the information I need.

Annotation and reviewing literature does not have to be a solo project. If are working in a group, you can use the Hypothes.is web browser extension to annotate articles collaboratively. You can also use Google Docs to collaboratively annotate a shared PDF using the commenting feature and write collaborative notes in a shared document. By sharing your highlights and comments, you can split the work of getting the most out of each article you read and build off one another's ideas.

Common annotations

In this section, we present common annotations people make when reading journal articles. These annotations are adapted from Craig Whippo and Raul Pacheco-Vega . If you are annotating on paper, I suggest using different color highlighters for each type of annotation listed below. If you are annotating electronically, you can use the names below as tags to easily find information later. For example, if you are searching for definitions of key concepts, you can either click on the tag for [definitions] in your PDF reader or thumb through a printed copy of article for whatever color or tag you used to indicate definitions of key terms. Most of all, you want to avoid reading through all of your sources again just to find that one thing you know you read somewhere . Time is a graduate student's most valuable resource, so our goal here is to help you spend your time reading the literature wisely.

Personal reflections

Personal reflections are all about you. What do you think? Are there any areas you are confused about? Any new ideas or reflections come to mind while you're reading? Treat these annotations as a means of capturing your first reflections about an article. Write down any questions or thoughts that come to mind as you read. If you think the author says something inaccurate or unsubstantiated, write that down. If you don't understand something, make a note about it and ask your professor. Don't feel bad! Journal articles are hard to understand sometimes, even for professors. Your goal is to critically read the literature, so write down what you think while reading! Table 4.2 contains some questions that might stimulate your thoughts.

Definitions

Note definitions of key terms for your topic. At minimum, you should include a scholarly definition for the concepts represented in your working question. If your working question asks about the process of leaving a relationship with domestic violence, your research proposal will have to explain how you define domestic violence, as well as how you define "leaving" an abusive relationship. While you may already know what you mean by domestic violence, the person reading your research proposal does not.

Annotating definitions also helps you engage with the scholarly debate around your topic. Definitions are often contested among scholars. Some definitions of domestic violence will be more comprehensive, including things such economic abuse or forcing the victim to problematically use substances. Other definitions will be less comprehensive, covering only physical, verbal, and sexual abuse. Often, how someone defines something conceptually is highly related to how they measure it in their study. Since you will have to do both of these things, find a definition that feels right to you or create your own, noting the ways in which it is similar or different from those in the literature.

Definitions are also an important way of dealing with jargon. Becoming familiar with a new content area involves learning the jargon experts use. For example, in the last paragraph I used the term economic abuse, but that's probably not a term you've heard before. If you were conducting a literature review on domestic violence, you would want to search for keywords like economic abuse if they are relevant to your working question. You will also want to know what they mean so you can use them appropriately in designing your study and writing your literature review.

Theoretical perspective

Noting the theoretical perspective of the article can help you interpret the data in the same manner as the author. For example, articles on supervised injection facilities for people who use intravenous drugs most likely come from a harm reduction perspective, and understanding the theory behind harm reduction is important to make sense of empirical results. Articles should be grounded in a theoretical perspective that helps the author conceptualize and understand the data. As we discussed in Chapter 3 , some journal articles are entirely theoretical and help you understand the theories or conceptual models related to your topic. We will help you determine a theoretical perspective for your project in Chapter 7 . For now, it's a good idea to note what theories authors mention when talking about your topic area. Some articles are better about this than others, and many authors make it a bit challenging to find theory (if mentioned at all). In other articles, it may help to note which social work theories are missing  from the literature. For example, a study's findings might address issues of oppression and discrimination, but the authors may not use critical theory to make sense of what happened.

Background knowledge

It's a good idea to note any relevant information the author relies on for background. When an author cites facts or opinions from others, you are subsequently able to get information from multiple articles simultaneously. For example, if we were looking at this meta-analysis about domestic violence , in the introduction section, the authors provide facts from many other sources. These facts will likely be relevant to your inquiry on domestic violence, as well.

As you are looking at background information, you should also note any subtopics or concepts about which there is controversy or consensus. The author may present one viewpoint and then an opposing viewpoint, something you may do in your literature review as well. Similarly, they may present facts that scholars in the field have come to consensus on and describe the ways in which different sources support these conclusions.

Sources of interest

Note any relevant sources the author cites. If there is any background information you plan to use, note the original source of that information. When you write your literature review, cite the original source of a piece of information you are using, which may not be where you initially read it . Remember that you should read and refer to the primary source . If you are reading Article A and the author cites a fact from Article B, you should note Article B in your annotations and use Article B when you cite the fact in your paper. You should also make sure Article A interpreted Article B correctly and scan Article B for any other useful facts.

Research question/Purpose

Authors should be clear about the purpose of their article. Charitable authors will give you a sentence that starts with something like this:

• "The purpose of this research project was..."
• "Our research question was..."
• "The research project was designed to test the following hypothesis..."

Unfortunately, not all authors are so clear, and you may to hunt around for the research question or hypothesis. Generally, in an empirical article, the research question or hypothesis is at the end of the introduction. In non-empirical articles, the author will likely discuss the purpose of the article in the abstract or introduction.

We will discuss in greater detail how to read the results of empirical articles in Chapter 5 . For now, just know that you should highlight any of the key findings of an article. They will be described very briefly in the abstract, and in much more detail in the article itself. In an empirical article, you should look at both the 'Results' and 'Discussion' sections. For a non-empirical article, the key findings will likely be in the conclusion. You can also find them in the topic or concluding sentences in a paragraph within the body of the article.

How do researchers know something when they see it? Found in the 'Methods' section of empirical articles, the measures section is where researchers spell out the tools, or measures, they used to gather data. For quantitative studies, you will want to get familiar with the questions researchers typically use to measure key variables. For example, to measure domestic violence, researchers often use the Conflict Tactics Scale . The more frequently used and cited a measure is, the more we know about how well it works (or not). Qualitative studies will often provide at least some of the interview or focus group questions they used with research participants. They will also include information about how their inquiry and hypotheses may have evolved over time. Keep in mind however, sometimes important information is cut out of an article during editing. If you need more information, consider reaching out to the author directly. Before you do so, check if the author provided an appendix with the information you need or if the article links to a their data and measures as part open data sharing practices.

Who exactly were the study participants and how were they recruited? In quantitative studies, you will want to pay attention to the sample size. Generally, the larger the sample, the greater the study's explanatory power. Additionally, randomly drawn samples are desirable because they leave any variation up to chance. Samples that are conducted out of convenience can be biased and non-representative of the larger population. In qualitative studies, non-random sampling is appropriate but consider this: how well does what we find for this group of people transfer to the people who will be in your study? For qualitative studies and quantitative studies, look for how well the sample is described and whether there are important characteristics missing from the article that you would need to determine the quality of the sample.

Limitations

Honest authors will include these at the end of each article. But you should also note any additional limitations you find with their work as well.

Your annotations

These are just a few suggested annotations, but you can come up with your own. For example, maybe there are annotations you would use for different assignments or for the problem statement in your research proposal. If you have an argument or idea that keeps coming to mind when you read, consider creating an annotation for it so you can remember which part of each article supports your ideas. Whatever works for you. The goal with annotation is to extract as much information from each article while reading, so you don't have to go back through everything again. It's useless to read an article and forget most of what you read. Annotate!

• Begin your search by reading thorough and cohesive literature reviews. Review articles are great sources of information to get a broad perspective of your topic.
• Don’t read an article just to say you’ve read it. Annotate and take notes so you don’t have to re-read it later.
• Use software or paper-and-pencil approaches to write notes on articles.
• Annotation is best used when closely reading an empirical study highly similar to your research project.
• Select an empirical article highly related to the study you would like to conduct.
• Annotate the article using the aforementioned annotations and create some of your own.
• Create the first draft of a summary table with key information from this empirical study that you would like to compare to other empirical studies you closely read.

5.3 Generalizability and transferability of empirical results

• Define generalizability and transferability.
• Assess the generalizability and transferability to how researchers use the results from empirical research studies to make arguments about what is objectively true.
• Relate both concepts to the hierarchy of evidence and the types of articles in the scholarly literature

Now that you have read an empirical article in detail, it's important to put its results in conversation with the broader literature on your topic. In this chapter we discuss two important concepts-- generalizability and   transferability --and the interrelationship between the two. We also explain how these two properties of empirical data impact your literature review and evidence-based practice.

Generalizability

The figure below provides a common approach to assessing empirical evidence. As you move up the pyramid below, you can be more sure that the data contained in those studies generalizes to all people who experience the issue.

As we reviewed in Chapter 1, objective truth is true for everyone, regardless of context. In other words, objective truths generalize beyond the sample of people from whom data were collected to the larger population of people who experience the issue under examination. You can be much more sure that information from a systematic review or meta-analysis will generalize than something from a case study of a single person, pilot projects, and other studies that do not seek to establish generalizability.

The type of article listed here is also related to the types of research methods the authors used. While we cover many of these approaches in this textbook, some of them (like cohort studies) are somewhat less common in social work. Additionally, there is one important research method, survey design, that does not appear in this diagram. Finally, social work research uses many different types of qualitative research--some of which generates more generalizable data than others.

For a refresher on the different types of evidence available in each type of article, refer back to section 4.1. You'll recall the hierarchy of evidence as described by McNeese & Thyer (2004) [55]

• Systematic reviews and meta-analyses
• Randomized controlled trials
• Quasi-experimental studies
• Case-control and cohort studies
• Pre-experimental (or non-experimental) group studies
• Qualitative studies

Because there is further variation in the types of studies used by social work researchers, I expanded the hierarchy of evidence to cover a greater breadth of research methods in Figure 5.3.

Refined information from multiple sources

The top of the hierarchy represents refined scientific information or meta-research . Meta-research uses the scientific method to analyze and improve the scientific production of knowledge. For example, meta-analyses pull together samples of people from all high-quality studies on a given topic area creating a super-study with far more people than any single researcher could feasibly collect data from. Because scientists (and clinical experts) refine data across multiple studies, these represent the most generalizable research findings.

Of course, not all meta-analyses or systematic reviews are of good quality. As a peer reviewer for a scholarly journal, I have seen poor quality systematic reviews that make methodological mistakes—like not including relevant keywords—that lead to incorrect conclusions. Unfortunately, not all errors are caught in the peer review process, and not all limitations are acknowledged by the authors. Just because you are looking at a systematic review does not mean you are looking at THE OBJECTIVE TRUTH. Nevertheless, you can be pretty sure that results from these studies are generalizable to the population in the study’s research question.

A good way to visualize the process of sampling is by examining the procedure used for systematic reviews and meta-analyses to scientifically search for articles. In Figure 5.4 below, you can see how researchers conducting a systematic review identified a large pool of potentially relevant articles, downloaded and analyzed them for relevance, and in the end, analyzed only 71 articles in their systematic review out of a total of 1,589 potentially relevant articles. Because systematic reviews or meta-analyses are intended to make strong, generalizable conclusions, they often exclude studies that still contain good information.

In the process of selecting articles for a meta-analysis and systematic review, researchers may exclude articles with important information for a number of good reasons. No study is perfect, and all research methods decisions come with limitations--including meta-research. Authors conducting a meta-analysis cannot include a study unless researchers provide data for the authors to include in their meta-analysis, and many empirical journal articles do not make their data available. Additionally, a study’s intervention or measures may be a bit different than what researchers want to make conclusions about. This is a key truth applicable across all articles you read—who or what gets selected for analysis in a research project determines how well the project’s results generalize to everyone.

We will talk about this in future chapters as sampling, and in those chapters, we will learn which sampling approaches are intended to support generalizability and which are used for other purposes. For example, availability or convenience sampling is often used to get quick information while random sampling approaches are intended to support generalizability. It is impossible to know everything about your article right now, but by the end of this course, you will have the information you need to critically examine the generalizability of a sample.

Primary sources (empirical studies)

Because refined sources like systematic reviews exclude good studies, they are only a first step in getting to know a topic area. You will need to examine primary sources--the reports of researchers who conducted empirical studies--to make evidence-based conclusions about your topic. Figure 5.3 describes three different types of data and ranks them vertically based on how well you can be sure the information generalizes.

As we will discuss further in our chapter on causal explanations, a key factor in scientifically assessing what happened first. Researchers conducting intervention studies are causing change by providing therapy, housing, or whatever the intervention is and measuring the outcomes of that intervention after they happen. This is unlike survey researchers, who do not introduce an intervention but ask people to self-report information on a questionnaire. Longitudinal surveys are particularly helpful because they can provide a clearer picture of whether the cause came before the effect in a causal relationship, but because they are expensive and time-consuming to conduct, longitudinal studies are relatively rare in the literature and most surveys measure people at only one point in time. Thus, because researchers cannot tightly control the causal variable (an intervention, an experience of abuse, etc.) we can be somewhat less certain of the conclusions of surveys than experiments. At the same time, because surveys measure people in their naturalistic environment rather than in a laboratory or artificial setting, they may do a better job at reducing the potential for the researcher to influence the data a participant provides. Surveys also provide descriptive information--like the number of people with a diagnosis or risk factor--that experiments cannot provide.

Surveys and experiments are commonly used in social work, and we will describe the methods they use in future chapters. When assessing the generalizability of a given survey or experiment, you are looking at whether the methods used by the researchers improve generalizability (or, at least that those methods are intended to improve generalizability). Specifically, there are sampling, measurement, and design decisions that researchers make that can improve generalizability. And once the study is conducted, whether those methods worked as intended also impact generalizability.

We address sampling, measurement, and design in the coming chapters, and you will need more in-depth knowledge of research methods to assess the generalizability of the results you are reading. In the meantime, Figure 5.3 is organized by design, and this is a good starting point for your inquiry since it only requires you to identify the design in each empirical article--which should be included in the abstract and described in detail in the methods section. For more information on how to conduct sampling, measurement, and design in a way that maximizes generalizability, read Part 2 of this textbook.

When searching for design of a study, look for specific keywords that indicate the researcher used methods that do not generalize well like pilot study, pre-experiment, non-experiment, convenience sample, availability sample, and exploratory study. When researchers are seeking to perform a pilot study, they are optimizing for time, not generalizability. Their results may still be useful to you! But, you should not generalize from their study to all people with the issue under analysis without a lot of caution and additional supporting evidence. Instead, you should see whether the lessons from this study might transfer to the context in which you are researching--our next topic.

Qualitative studies use sampling, measures, and designs that do not try to optimize generalizability. Thus, if the results of a qualitative study indicate 10 out of 50 students who participated in the focus group found the mandatory training on harassment to be unhelpful, does that mean 20% of all college students at this university find it unhelpful? Because focus groups and interviews (and other qualitative methods we will discuss) use qualitative methods, they are not concerned with generalizability. It would not make sense to generalize from focus groups to all people in a population. Instead, focus groups methods optimize for trustworthy and authentic research projects that make sure, for example, all themes and quotes in the researcher's report are traceable to quotes from focus group participants. Instead of providing what is generally true, qualitative research provides a thick description of people's experiences so you can understand them. S ubjective inquiry is less generalizable but provides greater depth in understanding people's feelings, beliefs, and decision-making processes within their context. 

In Figure 5.3, you will note that some qualitative studies are ranked higher than others in terms of generalizability. Meta-syntheses are ranked highest because they are meta-research, pooling together the themes and raw data from multiple qualitative studies into a super-study. A meta-synthesis is the qualitative equivalent of a meta-analysis, which analyzes quantitative data. Because the researchers conducting the meta-syntheses aim to make more broad generalizations across research studies, even though generalizability is not strictly the goal. In a similar way, grounded theory studies (a type of qualitative design) aim to produce a testable hypothesis that could generalize. At the bottom of the hierarchy are individual case studies, which report what happens with a single person, organization, or event. It's best not to think too long about the generalizability of qualitative results. When examining qualitative articles, you should be examining their transferability, our topic for the next subsection.

Transferability

Generalizability asks one question: How well does the sample of people in this study represent everyone with this issue? If you read in a study that 50% of people in the sample experienced depression, does that mean 50% of everyone experiences depression? We previewed future discussions in this textbook that will discuss the specific quantitative research methods used to optimize the generalizability of results. By adhering strictly to best practices in sampling, measurement, and design, researchers can provide you with good evidence for the generalizability of their study's results.

Of course, generalizability is not the only question worth asking. Just because a study's sample represents a broader population does not mean it is helpful for making conclusions about your working question. In assessing a study's transferability, you are making a weaker but compelling argument that the conclusions of one study can be applied to understanding the people in your working question and research project. Generalizable results may be applicable because they are broadly transferable across situations, and you can be confident in that when they follow the best practices in this textbook for improving generalizability. However, there may be aspects of a study that make its results difficult to transfer to your topic area.

When evaluating the transferability of a research result to your working question, consider the sample, measures, and design. That is, how data was collected from individuals, who those individuals are, and what researchers did with them. You may find that the samples in generalizable studies do not talk about the specific ethnic, cultural, or geographic group that is in your working question. Similarly, studies that measure the outcomes of substance use treatment by measuring sobriety may not match your working question on moderation, medication adherence, or substitution as an outcome in substance use treatment. Evaluating the transferability of designs may help you identify whether the methods the authors used would be similar to those you might use if you were to conduct a study gathering and collecting your own raw data.

Assessing transferability is more subjective. You are using your knowledge of your topic area and research methods (which are always improving!) to make a reasonable argument about why a given piece of evidence from a primary source helps you understand something. Look back at Table 5.2, your annotations, and the researchers' sampling, data analysis, results, and design. Using your critical thinking (and the knowledge you can in Part 2 and Part 3 of this textbook) you will need to make a reasonable argument that these results transfer to the people, places, and culture that you are talking about in your working question.

In the final chapter of Part 1, we will discuss how to assemble the facts you have taken from journal articles into a literature review that represents what  you think about the topic.

• Developing your theoretical framework
• Conceptual definitions
• Inductive & deductive reasoning

Nomothetic causal explanations

Content warning: examples in this chapter include references to sexual harassment, domestic violence, gender-based violence, the child welfare system, substance use disorders, neonatal abstinence syndrome, child abuse, racism, and sexism.

11.1 Developing your theoretical framework

• Differentiate between theories that explain specific parts of the social world versus those that are more broad and sweeping in their conclusions
• Identify the theoretical perspectives that are relevant to your project and inform your thinking about it
• Define key concepts in your working question and develop a theoretical framework for how you understand your topic.

Theories provide a way of looking at the world and of understanding human interaction. Paradigms are grounded in big assumptions about the world—what is real, how do we create knowledge—whereas theories describe more specific phenomena. Well, we are still oversimplifying a bit. Some theories try to explain the whole world, while others only try to explain a small part. Some theories can be grouped together based on common ideas but retain their own individual and unique features. Our goal is to help you find a theoretical framework that helps you understand your topic more deeply and answer your working question.

Theories: Big and small

In your human behavior and the social environment (HBSE) class, you were introduced to the major theoretical perspectives that are commonly used in social work. These are what we like to call big-T 'T'heories. When you read about systems theory, you are actually reading a synthesis of decades of distinct, overlapping, and conflicting theories that can be broadly classified within systems theory. For example, within systems theory, some approaches focus more on family systems while others focus on environmental systems, though the core concepts remain similar.

Different theorists define concepts in their own way, and as a result, their theories may explore different relationships with those concepts. For example, Deci and Ryan's (1985) [56] self-determination theory discusses motivation and establishes that it is contingent on meeting one's needs for autonomy, competency, and relatedness. By contrast, ecological self-determination theory, as written by Abery & Stancliffe (1996), [57] argues that self-determination is the amount of control exercised by an individual over aspects of their lives they deem important across the micro, meso, and macro levels. If self-determination were an important concept in your study, you would need to figure out which of the many theories related to self-determination helps you address your working question.

Theories can provide a broad perspective on the key concepts and relationships in the world or more specific and applied concepts and perspectives. Table 7.2 summarizes two commonly used lists of big-T Theoretical perspectives in social work. See if you can locate some of the theories that might inform your project.

Competing theoretical explanations

Within each area of specialization in social work, there are many other theories that aim to explain more specific types of interactions. For example, within the study of sexual harassment, different theories posit different explanations for why harassment occurs.

One theory, first developed by criminologists, is called routine activities theory. It posits that sexual harassment is most likely to occur when a workplace lacks unified groups and when potentially vulnerable targets and motivated offenders are both present (DeCoster, Estes, & Mueller, 1999). [60]

Other theories of sexual harassment, called relational theories, suggest that one's existing relationships are the key to understanding why and how workplace sexual harassment occurs and how people will respond when it does occur (Morgan, 1999). [61] Relational theories focus on the power that different social relationships provide (e.g., married people who have supportive partners at home might be more likely than those who lack support at home to report sexual harassment when it occurs).

Finally, feminist theories of sexual harassment take a different stance. These theories posit that the organization of our current gender system, wherein those who are the most masculine have the most power, best explains the occurrence of workplace sexual harassment (MacKinnon, 1979). [62] As you might imagine, which theory a researcher uses to examine the topic of sexual harassment will shape the questions asked about harassment. It will also shape the explanations the researcher provides for why harassment occurs.

For a graduate student beginning their study of a new topic, it may be intimidating to learn that there are so many theories beyond what you’ve learned in your theory classes. What’s worse is that there is no central database of theories on your topic. However, as you review the literature in your area, you will learn more about the theories scientists have created to explain how your topic works in the real world. There are other good sources for theories, in addition to journal articles. Books often contain works of theoretical and philosophical importance that are beyond the scope of an academic journal. Do a search in your university library for books on your topic, and you are likely to find theorists talking about how to make sense of your topic. You don't necessarily have to agree with the prevailing theories about your topic, but you do need to be aware of them so you can apply theoretical ideas to your project.

Applying big-T theories to your topic

The key to applying theories to your topic is learning the key concepts associated with that theory and the relationships between those concepts, or propositions . Again, your HBSE class should have prepared you with some of the most important concepts from the theoretical perspectives listed in Table 7.2. For example, the conflict perspective sees the world as divided into dominant and oppressed groups who engage in conflict over resources. If you were applying these theoretical ideas to your project, you would need to identify which groups in your project are considered dominant or oppressed groups, and which resources they were struggling over. This is a very general example. Challenge yourself to find small-t theories about your topic that will help you understand it in much greater detail and specificity. If you have chosen a topic that is relevant to your life and future practice, you will be doing valuable work shaping your ideas towards social work practice.

Integrating theory into your project can be easy, or it can take a bit more effort. Some people have a strong and explicit theoretical perspective that they carry with them at all times. For me, you'll probably see my work drawing from exchange and choice, social constructionist, and critical theory. Maybe you have theoretical perspectives you naturally employ, like Afrocentric theory or person-centered practice. If so, that's a great place to start since you might already be using that theory (even subconsciously) to inform your understanding of your topic. But if you aren't aware of whether you are using a theoretical perspective when you think about your topic, try writing a paragraph off the top of your head or talking with a friend explaining what you think about that topic. Try matching it with some of the ideas from the broad theoretical perspectives from Table 7.2. This can ground you as you search for more specific theories. Some studies are designed to test whether theories apply the real world while others are designed to create new theories or variations on existing theories. Consider which feels more appropriate for your project and what you want to know.

Another way to easily identify the theories associated with your topic is to look at the concepts in your working question. Are these concepts commonly found in any of the theoretical perspectives in Table 7.2? Take a look at the Payne and Hutchison texts and see if any of those look like the concepts and relationships in your working question or if any of them match with how you think about your topic. Even if they don't possess the exact same wording, similar theories can help serve as a starting point to finding other theories that can inform your project. Remember, HBSE textbooks will give you not only the broad statements of theories but also sources from specific theorists and sub-theories that might be more applicable to your topic. Skim the references and suggestions for further reading once you find something that applies well.

Choose a theoretical perspective from Hutchison, Payne, or another theory textbook that is relevant to your project. Using their textbooks or other reputable sources, identify :

• At least five important concepts from the theory
• What relationships the theory establishes between these important concepts (e.g., as x increases, the y decreases)
• How you can use this theory to better understand the concepts and variables in your project?

Developing your own theoretical framework

Hutchison's and Payne's frameworks are helpful for surveying the whole body of literature relevant to social work, which is why they are so widely used. They are one framework, or way of thinking, about all of the theories social workers will encounter that are relevant to practice. Social work researchers should delve further and develop a theoretical or conceptual framework of their own based on their reading of the literature. In Chapter 8 , we will develop your theoretical framework further, identifying the cause-and-effect relationships that answer your working question. Developing a theoretical framework is also instructive for revising and clarifying your working question and identifying concepts that serve as keywords for additional literature searching. The greater clarity you have with your theoretical perspective, the easier each subsequent step in the research process will be.

Getting acquainted with the important theoretical concepts in a new area can be challenging. While social work education provides a broad overview of social theory, you will find much greater fulfillment out of reading about the theories related to your topic area. We discussed some strategies for finding theoretical information in Chapter 3 as part of literature searching. To extend that conversation a bit, some strategies for searching for theories in the literature include:

• Consider searching for these keywords in the title or abstract, specifically
• Looking at the references and cited by links within theoretical articles and textbooks
• Looking at books, edited volumes, and textbooks that discuss theory
• Talking with a scholar on your topic, or asking a professor if they can help connect you to someone
• Nice authors are clear about how they use theory to inform their research project, usually in the introduction and discussion section.
• For example, from the broad umbrella of systems theory, you might pick out family systems theory if you want to understand the effectiveness of a family counseling program.

It's important to remember that knowledge arises within disciplines, and that disciplines have different theoretical frameworks for explaining the same topic. While it is certainly important for the social work perspective to be a part of your analysis, social workers benefit from searching across disciplines to come to a more comprehensive understanding of the topic. Reaching across disciplines can provide uncommon insights during conceptualization, and once the study is completed, a multidisciplinary researcher will be able to share results in a way that speaks to a variety of audiences. A study by An and colleagues (2015) [63] uses game theory from the discipline of economics to understand problems in the Temporary Assistance for Needy Families (TANF) program. In order to receive TANF benefits, mothers must cooperate with paternity and child support requirements unless they have "good cause," as in cases of domestic violence, in which providing that information would put the mother at greater risk of violence. Game theory can help us understand how TANF recipients and caseworkers respond to the incentives in their environment, and highlight why the design of the "good cause" waiver program may not achieve its intended outcome of increasing access to benefits for survivors of family abuse.

Of course, there are natural limits on the depth with which student researchers can and should engage in a search for theory about their topic. At minimum, you should be able to draw connections across studies and be able to assess the relative importance of each theory within the literature. Just because you found one article applying your theory (like game theory, in our example above) does not mean it is important or often used in the domestic violence literature. Indeed, it would be much more common in the family violence literature to find psychological theories of trauma, feminist theories of power and control, and similar theoretical perspectives used to inform research projects rather than game theory, which is equally applicable to survivors of family violence as workers and bosses at a corporation. Consider using the Cited By feature to identify articles, books, and other sources of theoretical information that are seminal or well-cited in the literature. Similarly, by using the name of a theory in the keywords of a search query (along with keywords related to your topic), you can get a sense of how often the theory is used in your topic area. You should have a sense of what theories are commonly used to analyze your topic, even if you end up choosing a different one to inform your project.

Theories that are not cited or used as often are still immensely valuable. As we saw before with TANF and "good cause" waivers, using theories from other disciplines can produce uncommon insights and help you make a new contribution to the social work literature. Given the privileged position that the social work curriculum places on theories developed by white men, students may want to explore Afrocentricity as a social work practice theory (Pellebon, 2007) [64] or abolitionist social work (Jacobs et al., 2021) [65] when deciding on a theoretical framework for their research project that addresses concepts of racial justice. Start with your working question, and explain how each theory helps you answer your question. Some explanations are going to feel right, and some concepts will feel more salient to you than others. Keep in mind that this is an iterative process. Your theoretical framework will likely change as you continue to conceptualize your research project, revise your research question, and design your study.

By trying on many different theoretical explanations for your topic area, you can better clarify your own theoretical framework. Some of you may be fortunate enough to find theories that match perfectly with how you think about your topic, are used often in the literature, and are therefore relatively straightforward to apply. However, many of you may find that a combination of theoretical perspectives is most helpful for you to investigate your project. For example, maybe the group counseling program for which you are evaluating client outcomes draws from both motivational interviewing and cognitive behavioral therapy. In order to understand the change happening in the client population, you would need to know each theory separately as well as how they work in tandem with one another. Because theoretical explanations and even the definitions of concepts are debated by scientists, it may be helpful to find a specific social scientist or group of scientists whose perspective on the topic you find matches with your understanding of the topic. Of course, it is also perfectly acceptable to develop your own theoretical framework, though you should be able to articulate how your framework fills a gap within the literature.

If you are adapting theoretical perspectives in your study, it is important to clarify the original authors' definitions of each concept. Jabareen (2009) [66] offers that conceptual frameworks are not merely collections of concepts but, rather, constructs in which each concept plays an integral role. [67] A conceptual framework is a network of linked concepts that together provide a comprehensive understanding of a phenomenon. Each concept in a conceptual framework plays an ontological or epistemological role in the framework, and it is important to assess whether the concepts and relationships in your framework make sense together. As your framework takes shape, you will find yourself integrating and grouping together concepts, thinking about the most important or least important concepts, and how each concept is causally related to others.

Much like paradigm, theory plays a supporting role for the conceptualization of your research project. Recall the ice float from Figure 7.1. Theoretical explanations support the design and methods you use to answer your research question. In student projects that lack a theoretical framework, I often see the biases and errors in reasoning that we discussed in Chapter 1 that get in the way of good social science. That's because theories mark which concepts are important, provide a framework for understanding them, and measure their interrelationships. If you are missing this foundation, you will operate on informal observation, messages from authority, and other forms of unsystematic and unscientific thinking we reviewed in Chapter 1 .

Theory-informed inquiry is incredibly helpful for identifying key concepts and how to measure them in your research project, but there is a risk in aligning research too closely with theory. The theory-ladenness of facts and observations produced by social science research means that we may be making our ideas real through research. This is a potential source of confirmation bias in social science. Moreover, as Tan (2016) [68] demonstrates, social science often proceeds by adopting as true the perspective of Western and Global North countries, and cross-cultural research is often when ethnocentric and biased ideas are most visible . In her example, a researcher from the West studying teacher-centric classrooms in China that rely partially on rote memorization may view them as less advanced than student-centered classrooms developed in a Western country simply because of Western philosophical assumptions about the importance of individualism and self-determination. Developing a clear theoretical framework is a way to guard against biased research, and it will establish a firm foundation on which you will develop the design and methods for your study.

• Just as empirical evidence is important for conceptualizing a research project, so too are the key concepts and relationships identified by social work theory.
• Using theory your theory textbook will provide you with a sense of the broad theoretical perspectives in social work that might be relevant to your project.
• Try to find small-t theories that are more specific to your topic area and relevant to your working question.
• In Chapter 2 , you developed a concept map for your proposal. Take a moment to revisit your concept map now as your theoretical framework is taking shape. Make any updates to the key concepts and relationships in your concept map. . If you need a refresher, we have embedded a short how-to video from the University of Guelph Library (CC-BY-NC-SA 4.0) that we also used in Chapter 2 .

11.2 Conceptual definitions

• Define measurement and conceptualization
• Apply Kaplan’s three categories to determine the complexity of measuring a given variable
• Identify the role previous research and theory play in defining concepts
• Distinguish between unidimensional and multidimensional concepts
• Critically apply reification to how you conceptualize the key variables in your research project

In social science, when we use the term  measurement , we mean the process by which we describe and ascribe meaning to the key facts, concepts, or other phenomena that we are investigating. At its core, measurement is about defining one’s terms in as clear and precise a way as possible. Of course, measurement in social science isn’t quite as simple as using a measuring cup or spoon, but there are some basic tenets on which most social scientists agree when it comes to measurement. We’ll explore those, as well as some of the ways that measurement might vary depending on your unique approach to the study of your topic.

An important point here is that measurement does not require any particular instruments or procedures. What it does require is a systematic procedure for assigning scores, meanings, and descriptions to individuals or objects so that those scores represent the characteristic of interest. You can measure phenomena in many different ways, but you must be sure that how you choose to measure gives you information and data that lets you answer your research question. If you're looking for information about a person's income, but your main points of measurement have to do with the money they have in the bank, you're not really going to find the information you're looking for!

The question of what social scientists measure can be answered by asking yourself what social scientists study. Think about the topics you’ve learned about in other social work classes you’ve taken or the topics you’ve considered investigating yourself. Let’s consider Melissa Milkie and Catharine Warner’s study (2011) [69] of first graders’ mental health. In order to conduct that study, Milkie and Warner needed to have some idea about how they were going to measure mental health. What does mental health mean, exactly? And how do we know when we’re observing someone whose mental health is good and when we see someone whose mental health is compromised? Understanding how measurement works in research methods helps us answer these sorts of questions.

As you might have guessed, social scientists will measure just about anything that they have an interest in investigating. For example, those who are interested in learning something about the correlation between social class and levels of happiness must develop some way to measure both social class and happiness. Those who wish to understand how well immigrants cope in their new locations must measure immigrant status and coping. Those who wish to understand how a person’s gender shapes their workplace experiences must measure gender and workplace experiences (and get more specific about which experiences are under examination). You get the idea. Social scientists can and do measure just about anything you can imagine observing or wanting to study. Of course, some things are easier to observe or measure than others.

Observing your variables

In 1964, philosopher Abraham Kaplan (1964) [70] wrote The   Conduct of Inquiry,  which has since become a classic work in research methodology (Babbie, 2010). [71] In his text, Kaplan describes different categories of things that behavioral scientists observe. One of those categories, which Kaplan called “observational terms,” is probably the simplest to measure in social science. Observational terms are the sorts of things that we can see with the naked eye simply by looking at them. Kaplan roughly defines them as conditions that are easy to identify and verify through direct observation. If, for example, we wanted to know how the conditions of playgrounds differ across different neighborhoods, we could directly observe the variety, amount, and condition of equipment at various playgrounds.

Indirect observables , on the other hand, are less straightforward to assess. In Kaplan's framework, they are conditions that are subtle and complex that we must use existing knowledge and intuition to define. If we conducted a study for which we wished to know a person’s income, we’d probably have to ask them their income, perhaps in an interview or a survey. Thus, we have observed income, even if it has only been observed indirectly. Birthplace might be another indirect observable. We can ask study participants where they were born, but chances are good we won’t have directly observed any of those people being born in the locations they report.

Sometimes the measures that we are interested in are more complex and more abstract than observational terms or indirect observables. Think about some of the concepts you’ve learned about in other social work classes—for example, ethnocentrism. What is ethnocentrism? Well, from completing an introduction to social work class you might know that it has something to do with the way a person judges another’s culture. But how would you  measure  it? Here’s another construct: bureaucracy. We know this term has something to do with organizations and how they operate but measuring such a construct is trickier than measuring something like a person’s income. The theoretical concepts of ethnocentrism and bureaucracy represent ideas whose meanings we have come to agree on. Though we may not be able to observe these abstractions directly, we can observe their components.

Kaplan referred to these more abstract things that behavioral scientists measure as constructs.  Constructs  are “not observational either directly or indirectly” (Kaplan, 1964, p. 55), [72] but they can be defined based on observables. For example, the construct of bureaucracy could be measured by counting the number of supervisors that need to approve routine spending by public administrators. The greater the number of administrators that must sign off on routine matters, the greater the degree of bureaucracy. Similarly, we might be able to ask a person the degree to which they trust people from different cultures around the world and then assess the ethnocentrism inherent in their answers. We can measure constructs like bureaucracy and ethnocentrism by defining them in terms of what we can observe. [73]

The idea of coming up with your own measurement tool might sound pretty intimidating at this point. The good news is that if you find something in the literature that works for you, you can use it (with proper attribution, of course). If there are only pieces of it that you like, you can reuse those pieces (with proper attribution and describing/justifying any changes). You don't always have to start from scratch!

Look at the variables in your research question.

• Classify them as direct observables, indirect observables, or constructs.
• Do you think measuring them will be easy or hard?
• What are your first thoughts about how to measure each variable? No wrong answers here, just write down a thought about each variable.

Measurement starts with conceptualization

In order to measure the concepts in your research question, we first have to understand what we think about them. As an aside, the word concept  has come up quite a bit, and it is important to be sure we have a shared understanding of that term. A  concept is the notion or image that we conjure up when we think of some cluster of related observations or ideas. For example, masculinity is a concept. What do you think of when you hear that word? Presumably, you imagine some set of behaviors and perhaps even a particular style of self-presentation. Of course, we can’t necessarily assume that everyone conjures up the same set of ideas or images when they hear the word  masculinity . While there are many possible ways to define the term and some may be more common or have more support than others, there is no universal definition of masculinity. What counts as masculine may shift over time, from culture to culture, and even from individual to individual (Kimmel, 2008). This is why defining our concepts is so important.\

Not all researchers clearly explain their theoretical or conceptual framework for their study, but they should! Without understanding how a researcher has defined their key concepts, it would be nearly impossible to understand the meaning of that researcher’s findings and conclusions. Back in Chapter 7 , you developed a theoretical framework for your study based on a survey of the theoretical literature in your topic area. If you haven't done that yet, consider flipping back to that section to familiarize yourself with some of the techniques for finding and using theories relevant to your research question. Continuing with our example on masculinity, we would need to survey the literature on theories of masculinity. After a few queries on masculinity, I found a wonderful article by Wong (2010) [74] that analyzed eight years of the journal Psychology of Men & Masculinity and analyzed how often different theories of masculinity were used . Not only can I get a sense of which theories are more accepted and which are more marginal in the social science on masculinity, I am able to identify a range of options from which I can find the theory or theories that will inform my project. 

Identify a specific theory (or more than one theory) and how it helps you understand...

• Your independent variable(s).
• Your dependent variable(s).
• The relationship between your independent and dependent variables.

Rather than completing this exercise from scratch, build from your theoretical or conceptual framework developed in previous chapters.

In quantitative methods, conceptualization involves writing out clear, concise definitions for our key concepts. These are the kind of definitions you are used to, like the ones in a dictionary. A conceptual definition involves defining a concept in terms of other concepts, usually by making reference to how other social scientists and theorists have defined those concepts in the past. Of course, new conceptual definitions are created all the time because our conceptual understanding of the world is always evolving.

Conceptualization is deceptively challenging—spelling out exactly what the concepts in your research question mean to you. Following along with our example, think about what comes to mind when you read the term masculinity. How do you know masculinity when you see it? Does it have something to do with men or with social norms? If so, perhaps we could define masculinity as the social norms that men are expected to follow. That seems like a reasonable start, and at this early stage of conceptualization, brainstorming about the images conjured up by concepts and playing around with possible definitions is appropriate. However, this is just the first step. At this point, you should be beyond brainstorming for your key variables because you have read a good amount of research about them

In addition, we should consult previous research and theory to understand the definitions that other scholars have already given for the concepts we are interested in. This doesn’t mean we must use their definitions, but understanding how concepts have been defined in the past will help us to compare our conceptualizations with how other scholars define and relate concepts. Understanding prior definitions of our key concepts will also help us decide whether we plan to challenge those conceptualizations or rely on them for our own work. Finally, working on conceptualization is likely to help in the process of refining your research question to one that is specific and clear in what it asks. Conceptualization and operationalization (next section) are where "the rubber meets the road," so to speak, and you have to specify what you mean by the question you are asking. As your conceptualization deepens, you will often find that your research question becomes more specific and clear.

If we turn to the literature on masculinity, we will surely come across work by Michael Kimmel , one of the preeminent masculinity scholars in the United States. After consulting Kimmel’s prior work (2000; 2008), [75] we might tweak our initial definition of masculinity. Rather than defining masculinity as “the social norms that men are expected to follow,” perhaps instead we’ll define it as “the social roles, behaviors, and meanings prescribed for men in any given society at any one time” (Kimmel & Aronson, 2004, p. 503). [76] Our revised definition is more precise and complex because it goes beyond addressing one aspect of men’s lives (norms), and addresses three aspects: roles, behaviors, and meanings. It also implies that roles, behaviors, and meanings may vary across societies and over time. Using definitions developed by theorists and scholars is a good idea, though you may find that you want to define things your own way.

As you can see, conceptualization isn’t as simple as applying any random definition that we come up with to a term. Defining our terms may involve some brainstorming at the very beginning. But conceptualization must go beyond that, to engage with or critique existing definitions and conceptualizations in the literature. Once we’ve brainstormed about the images associated with a particular word, we should also consult prior work to understand how others define the term in question. After we’ve identified a clear definition that we’re happy with, we should make sure that every term used in our definition will make sense to others. Are there terms used within our definition that also need to be defined? If so, our conceptualization is not yet complete. Our definition includes the concept of "social roles," so we should have a definition for what those mean and become familiar with role theory to help us with our conceptualization. If we don't know what roles are, how can we study them?

Let's say we do all of that. We have a clear definition of the term masculinity with reference to previous literature and we also have a good understanding of the terms in our conceptual definition...then we're done, right? Not so fast. You’ve likely met more than one man in your life, and you’ve probably noticed that they are not the same, even if they live in the same society during the same historical time period. This could mean there are dimensions of masculinity. In terms of social scientific measurement, concepts can be said to have multiple dimensions  when there are multiple elements that make up a single concept. With respect to the term  masculinity , dimensions could based on gender identity, gender performance, sexual orientation, etc.. In any of these cases, the concept of masculinity would be considered to have multiple dimensions.

While you do not need to spell out every possible dimension of the concepts you wish to measure, it is important to identify whether your concepts are unidimensional (and therefore relatively easy to define and measure) or multidimensional (and therefore require multi-part definitions and measures). In this way, how you conceptualize your variables determines how you will measure them in your study. Unidimensional concepts are those that are expected to have a single underlying dimension. These concepts can be measured using a single measure or test. Examples include simple concepts such as a person’s weight, time spent sleeping, and so forth. 

One frustrating this is that there is no clear demarcation between concepts that are inherently unidimensional or multidimensional. Even something as simple as age could be broken down into multiple dimensions including mental age and chronological age, so where does conceptualization stop? How far down the dimensional rabbit hole do we have to go? Researchers should consider two things. First, how important is this variable in your study? If age is not important in your study (maybe it is a control variable), it seems like a waste of time to do a lot of work drawing from developmental theory to conceptualize this variable. A unidimensional measure from zero to dead is all the detail we need. On the other hand, if we were measuring the impact of age on masculinity, conceptualizing our independent variable (age) as multidimensional may provide a richer understanding of its impact on masculinity. Finally, your conceptualization will lead directly to your operationalization of the variable, and once your operationalization is complete, make sure someone reading your study could follow how your conceptual definitions informed the measures you chose for your variables. 

Write a conceptual definition for your independent and dependent variables.

• Cite and attribute definitions to other scholars, if you use their words.
• Describe how your definitions are informed by your theoretical framework.
• Place your definition in conversation with other theories and conceptual definitions commonly used in the literature.
• Are there multiple dimensions of your variables?
• Are any of these dimensions important for you to measure?

Do researchers actually know what we're talking about?

Conceptualization proceeds differently in qualitative research compared to quantitative research. Since qualitative researchers are interested in the understandings and experiences of their participants, it is less important for them to find one fixed definition for a concept before starting to interview or interact with participants. The researcher’s job is to accurately and completely represent how their participants understand a concept, not to test their own definition of that concept.

If you were conducting qualitative research on masculinity, you would likely consult previous literature like Kimmel’s work mentioned above. From your literature review, you may come up with a  working definition  for the terms you plan to use in your study, which can change over the course of the investigation. However, the definition that matters is the definition that your participants share during data collection. A working definition is merely a place to start, and researchers should take care not to think it is the only or best definition out there.

In qualitative inquiry, your participants are the experts (sound familiar, social workers?) on the concepts that arise during the research study. Your job as the researcher is to accurately and reliably collect and interpret their understanding of the concepts they describe while answering your questions. Conceptualization of concepts is likely to change over the course of qualitative inquiry, as you learn more information from your participants. Indeed, getting participants to comment on, extend, or challenge the definitions and understandings of other participants is a hallmark of qualitative research. This is the opposite of quantitative research, in which definitions must be completely set in stone before the inquiry can begin.

The contrast between qualitative and quantitative conceptualization is instructive for understanding how quantitative methods (and positivist research in general) privilege the knowledge of the researcher over the knowledge of study participants and community members. Positivism holds that the researcher is the "expert," and can define concepts based on their expert knowledge of the scientific literature. This knowledge is in contrast to the lived experience that participants possess from experiencing the topic under examination day-in, day-out. For this reason, it would be wise to remind ourselves not to take our definitions too seriously and be critical about the limitations of our knowledge.

Conceptualization must be open to revisions, even radical revisions, as scientific knowledge progresses. While I’ve suggested consulting prior scholarly definitions of our concepts, you should not assume that prior, scholarly definitions are more real than the definitions we create. Likewise, we should not think that our own made-up definitions are any more real than any other definition. It would also be wrong to assume that just because definitions exist for some concept that the concept itself exists beyond some abstract idea in our heads. Building on the paradigmatic ideas behind interpretivism and the critical paradigm, researchers call the assumption that our abstract concepts exist in some concrete, tangible way is known as reification . It explores the power dynamics behind how we can create reality by how we define it.

Returning again to our example of masculinity. Think about our how our notions of masculinity have developed over the past few decades, and how different and yet so similar they are to patriarchal definitions throughout history. Conceptual definitions become more or less popular based on the power arrangements inside of social science the broader world. Western knowledge systems are privileged, while others are viewed as unscientific and marginal. The historical domination of social science by white men from WEIRD countries meant that definitions of masculinity were imbued their cultural biases and were designed explicitly and implicitly to preserve their power. This has inspired movements for cognitive justice as we seek to use social science to achieve global development.

• Measurement is the process by which we describe and ascribe meaning to the key facts, concepts, or other phenomena that we are investigating.
• Kaplan identified three categories of things that social scientists measure including observational terms, indirect observables, and constructs.
• Some concepts have multiple elements or dimensions.
• Researchers often use measures previously developed and studied by other researchers.
• Conceptualization is a process that involves coming up with clear, concise definitions.
• Conceptual definitions are based on the theoretical framework you are using for your study (and the paradigmatic assumptions underlying those theories).
• Whether your conceptual definitions come from your own ideas or the literature, you should be able to situate them in terms of other commonly used conceptual definitions.
• Researchers should acknowledge the limited explanatory power of their definitions for concepts and how oppression can shape what explanations are considered true or scientific.

Think historically about the variables in your research question.

• How has our conceptual definition of your topic changed over time?
• What scholars or social forces were responsible for this change?

Take a critical look at your conceptual definitions.

• How participants might define terms for themselves differently, in terms of their daily experience?
• On what cultural assumptions are your conceptual definitions based?
• Are your conceptual definitions applicable across all cultures that will be represented in your sample?

11.3 Inductive and deductive reasoning

• Describe inductive and deductive reasoning and provide examples of each
• Identify how inductive and deductive reasoning are complementary

Congratulations! You survived the chapter on theories and paradigms. My experience has been that many students have a difficult time thinking about theories and paradigms because they perceive them as "intangible" and thereby hard to connect to social work research. I even had one student who said she got frustrated just reading the word "philosophy."

Rest assured, you do not need to become a theorist or philosopher to be an effective social worker or researcher. However, you should have a good sense of what theory or theories will be relevant to your project, as well as how this theory, along with your working question, fit within the three broad research paradigms we reviewed. If you don't have a good idea about those at this point, it may be a good opportunity to pause and read more about the theories related to your topic area.

Theories structure and inform social work research. The converse is also true: research can structure and inform theory. The reciprocal relationship between theory and research often becomes evident to students when they consider the relationships between theory and research in inductive and deductive approaches to research. In both cases, theory is crucial. But the relationship between theory and research differs for each approach.

While inductive and deductive approaches to research are quite different, they can also be complementary. Let’s start by looking at each one and how they differ from one another. Then we’ll move on to thinking about how they complement one another.

Inductive reasoning

A researcher using inductive reasoning begins by collecting data that is relevant to their topic of interest. Once a substantial amount of data have been collected, the researcher will then step back from data collection to get a bird’s eye view of their data. At this stage, the researcher looks for patterns in the data, working to develop a theory that could explain those patterns. Thus, when researchers take an inductive approach, they start with a particular set of observations and move to a more general set of propositions about those experiences. In other words, they move from data to theory, or from the specific to the general. Figure 8.1 outlines the steps involved with an inductive approach to research.

There are many good examples of inductive research, but we’ll look at just a few here. One fascinating study in which the researchers took an inductive approach is Katherine Allen, Christine Kaestle, and Abbie Goldberg’s (2011) [77] study of how boys and young men learn about menstruation. To understand this process, Allen and her colleagues analyzed the written narratives of 23 young cisgender men in which the men described how they learned about menstruation, what they thought of it when they first learned about it, and what they think of it now. By looking for patterns across all 23 cisgender men’s narratives, the researchers were able to develop a general theory of how boys and young men learn about this aspect of girls’ and women’s biology. They conclude that sisters play an important role in boys’ early understanding of menstruation, that menstruation makes boys feel somewhat separated from girls, and that as they enter young adulthood and form romantic relationships, young men develop more mature attitudes about menstruation. Note how this study began with the data—men’s narratives of learning about menstruation—and worked to develop a theory.

In another inductive study, Kristin Ferguson and colleagues (Ferguson, Kim, & McCoy, 2011) [78] analyzed empirical data to better understand how to meet the needs of young people who are homeless. The authors analyzed focus group data from 20 youth at a homeless shelter. From these data they developed a set of recommendations for those interested in applied interventions that serve homeless youth. The researchers also developed hypotheses for others who might wish to conduct further investigation of the topic. Though Ferguson and her colleagues did not test their hypotheses, their study ends where most deductive investigations begin: with a theory and a hypothesis derived from that theory. Section 8.4 discusses the use of mixed methods research as a way for researchers to test hypotheses created in a previous component of the same research project.

You will notice from both of these examples that inductive reasoning is most commonly found in studies using qualitative methods, such as focus groups and interviews. Because inductive reasoning involves the creation of a new theory, researchers need very nuanced data on how the key concepts in their working question operate in the real world. Qualitative data is often drawn from lengthy interactions and observations with the individuals and phenomena under examination. For this reason, inductive reasoning is most often associated with qualitative methods, though it is used in both quantitative and qualitative research.

Deductive reasoning

If inductive reasoning is about creating theories from raw data, deductive reasoning is about testing theories using data. Researchers using deductive reasoning take the steps described earlier for inductive research and reverse their order. They start with a compelling social theory, create a hypothesis about how the world should work, collect raw data, and analyze whether their hypothesis was confirmed or not. That is, deductive approaches move from a more general level (theory) to a more specific (data); whereas inductive approaches move from the specific (data) to general (theory).

A deductive approach to research is the one that people typically associate with scientific investigation. Students in English-dominant countries that may be confused by inductive vs. deductive research can rest part of the blame on Sir Arthur Conan Doyle, creator of the Sherlock Holmes character. As Craig Vasey points out in his breezy introduction to logic book chapter , Sherlock Holmes more often used inductive rather than deductive reasoning (despite claiming to use the powers of deduction to solve crimes). By noticing subtle details in how people act, behave, and dress, Holmes finds patterns that others miss. Using those patterns, he creates a theory of how the crime occurred, dramatically revealed to the authorities just in time to arrest the suspect. Indeed, it is these flashes of insight into the patterns of data that make Holmes such a keen inductive reasoner. In social work practice, rather than detective work, inductive reasoning is supported by the intuitions and practice wisdom of social workers, just as Holmes' reasoning is sharpened by his experience as a detective.

So, if deductive reasoning isn't Sherlock Holmes' observation and pattern-finding, how does it work? It starts with what you have already done in Chapters 3 and 4, reading and evaluating what others have done to study your topic. It continued with Chapter 5, discovering what theories already try to explain how the concepts in your working question operate in the real world. Tapping into this foundation of knowledge on their topic, the researcher studies what others have done, reads existing theories of whatever phenomenon they are studying, and then tests hypotheses that emerge from those theories. Figure 8.2 outlines the steps involved with a deductive approach to research.

While not all researchers follow a deductive approach, many do. We’ll now take a look at a couple excellent recent examples of deductive research. 

In a study of US law enforcement responses to hate crimes, Ryan King and colleagues (King, Messner, & Baller, 2009) [79] hypothesized that law enforcement’s response would be less vigorous in areas of the country that had a stronger history of racial violence. The authors developed their hypothesis from prior research and theories on the topic. They tested the hypothesis by analyzing data on states’ lynching histories and hate crime responses. Overall, the authors found support for their hypothesis and illustrated an important application of critical race theory.

In another recent deductive study, Melissa Milkie and Catharine Warner (2011) [80] studied the effects of different classroom environments on first graders’ mental health. Based on prior research and theory, Milkie and Warner hypothesized that negative classroom features, such as a lack of basic supplies and heat, would be associated with emotional and behavioral problems in children. One might associate this research with Maslow's hierarchy of needs or systems theory. The researchers found support for their hypothesis, demonstrating that policymakers should be paying more attention to the mental health outcomes of children’s school experiences, just as they track academic outcomes (American Sociological Association, 2011). [81]

Complementary approaches

While inductive and deductive approaches to research seem quite different, they can actually be rather complementary. In some cases, researchers will plan for their study to include multiple components, one inductive and the other deductive. In other cases, a researcher might begin a study with the plan to conduct either inductive or deductive research, but then discovers along the way that the other approach is needed to help illuminate findings. Here is an example of each such case.

Dr. Amy Blackstone (n.d.), author of Principles of sociological inquiry: Qualitative and quantitative methods , relates a story about her mixed methods research on sexual harassment.

We began the study knowing that we would like to take both a deductive and an inductive approach in our work. We therefore administered a quantitative survey, the responses to which we could analyze in order to test hypotheses, and also conducted qualitative interviews with a number of the survey participants. The survey data were well suited to a deductive approach; we could analyze those data to test hypotheses that were generated based on theories of harassment. The interview data were well suited to an inductive approach; we looked for patterns across the interviews and then tried to make sense of those patterns by theorizing about them. For one paper (Uggen & Blackstone, 2004) [82] , we began with a prominent feminist theory of the sexual harassment of adult women and developed a set of hypotheses outlining how we expected the theory to apply in the case of younger women’s and men’s harassment experiences. We then tested our hypotheses by analyzing the survey data. In general, we found support for the theory that posited that the current gender system, in which heteronormative men wield the most power in the workplace, explained workplace sexual harassment—not just of adult women but of younger women and men as well. In a more recent paper (Blackstone, Houle, & Uggen, 2006), [83] we did not hypothesize about what we might find but instead inductively analyzed interview data, looking for patterns that might tell us something about how or whether workers’ perceptions of harassment change as they age and gain workplace experience. From this analysis, we determined that workers’ perceptions of harassment did indeed shift as they gained experience and that their later definitions of harassment were more stringent than those they held during adolescence. Overall, our desire to understand young workers’ harassment experiences fully—in terms of their objective workplace experiences, their perceptions of those experiences, and their stories of their experiences—led us to adopt both deductive and inductive approaches in the work. (Blackstone, n.d., p. 21) [84]

Researchers may not always set out to employ both approaches in their work but sometimes find that their use of one approach leads them to the other. One such example is described eloquently in Russell Schutt’s  Investigating the Social World (2006). [85] As Schutt describes, researchers Sherman and Berk (1984) [86] conducted an experiment to test two competing theories of the effects of punishment on deterring deviance (in this case, domestic violence).Specifically, Sherman and Berk hypothesized that deterrence   theory (see Williams, 2005 [87] for more information on that theory) would provide a better explanation of the effects of arresting accused batterers than labeling theory . Deterrence theory predicts that arresting an accused spouse batterer will  reduce  future incidents of violence. Conversely, labeling theory predicts that arresting accused spouse batterers will  increase  future incidents (see Policastro & Payne, 2013 [88] for more information on that theory). Figure 8.3 summarizes the two competing theories and the hypotheses Sherman and Berk set out to test.

Research from these follow-up studies were mixed. In some cases, arrest deterred future incidents of violence. In other cases, it did not. This left the researchers with new data that they needed to explain. The researchers therefore took an inductive approach in an effort to make sense of their latest empirical observations. The new studies revealed that arrest seemed to have a deterrent effect for those who were married and employed, but that it led to increased offenses for those who were unmarried and unemployed. Researchers thus turned to control theory, which posits that having some stake in conformity through the social ties provided by marriage and employment, as the better explanation (see Davis et al., 2000 [90] for more information on this theory).

What the original Sherman and Berk study, along with the follow-up studies, show us is that we might start with a deductive approach to research, but then, if confronted by new data we must make sense of, we may move to an inductive approach. We will expand on these possibilities in section 8.4 when we discuss mixed methods research.

Ethical and critical considerations

Deductive and inductive reasoning, just like other components of the research process comes with ethical and cultural considerations for researchers. Specifically, deductive research is limited by existing theory. Because scientific inquiry has been shaped by oppressive forces such as sexism, racism, and colonialism, what is considered theory is largely based in Western, white-male-dominant culture. Thus, researchers doing deductive research may artificially limit themselves to ideas that were derived from this context. Non-Western researchers, international social workers, and practitioners working with non-dominant groups may find deductive reasoning of limited help if theories do not adequately describe other cultures.

While these flaws in deductive research may make inductive reasoning seem more appealing, on closer inspection you'll find similar issues apply. A researcher using inductive reasoning applies their intuition and lived experience when analyzing participant data. They will take note of particular themes, conceptualize their definition, and frame the project using their unique psychology. Since everyone's internal world is shaped by their cultural and environmental context, inductive reasoning conducted by Western researchers may unintentionally reinforcing lines of inquiry that derive from cultural oppression.

Inductive reasoning is also shaped by those invited to provide the data to be analyzed. For example, I recently worked with a student who wanted to understand the impact of child welfare supervision on children born dependent on opiates and methamphetamine. Due to the potential harm that could come from interviewing families and children who are in foster care or under child welfare supervision, the researcher decided to use inductive reasoning and to only interview child welfare workers.

Talking to practitioners is a good idea for feasibility, as they are less vulnerable than clients. However, any theory that emerges out of these observations will be substantially limited, as it would be devoid of the perspectives of parents, children, and other community members who could provide a more comprehensive picture of the impact of child welfare involvement on children. Notice that each of these groups has less power than child welfare workers in the service relationship. Attending to which groups were used to inform the creation of a theory and the power of those groups is an important critical consideration for social work researchers.

As you can see, when researchers apply theory to research they must wrestle with the history and hierarchy around knowledge creation in that area. In deductive studies, the researcher is positioned as the expert, similar to the positivist paradigm presented in Chapter 5. We've discussed a few of the limitations on the knowledge of researchers in this subsection, but the position of the "researcher as expert" is inherently problematic. However, it should also not be taken to an extreme. A researcher who approaches inductive inquiry as a naïve learner is also inherently problematic. Just as competence in social work practice requires a baseline of knowledge prior to entering practice, so does competence in social work research. Because a truly naïve intellectual position is impossible—we all have preexisting ways we view the world and are not fully aware of how they may impact our thoughts—researchers should be well-read in the topic area of their research study but humble enough to know that there is always much more to learn.

• Inductive reasoning begins with a set of empirical observations, seeking patterns in those observations, and then theorizing about those patterns.
• Deductive reasoning begins with a theory, developing hypotheses from that theory, and then collecting and analyzing data to test the truth of those hypotheses.
• Inductive and deductive reasoning can be employed together for a more complete understanding of the research topic.
• Though researchers don’t always set out to use both inductive and deductive reasoning in their work, they sometimes find that new questions arise in the course of an investigation that can best be answered by employing both approaches.
• Identify one theory and how it helps you understand your topic and working question.

I encourage you to find a specific theory from your topic area, rather than relying only on the broad theoretical perspectives like systems theory or the strengths perspective. Those broad theoretical perspectives are okay...but I promise that searching for theories about your topic will help you conceptualize and design your research project.

• Using the theory you identified, describe what you expect the answer to be to your working question.
• Define and provide an example of idiographic causal relationships
• Describe the role of causality in quantitative research as compared to qualitative research
• Identify, define, and describe each of the main criteria for nomothetic causal relationships
• Describe the difference between and provide examples of independent, dependent, and control variables
• Define hypothesis, state a clear hypothesis, and discuss the respective roles of quantitative and qualitative research when it comes to hypotheses

Causality  refers to the idea that one event, behavior, or belief will result in the occurrence of another, subsequent event, behavior, or belief. In other words, it is about cause and effect. It seems simple, but you may be surprised to learn there is more than one way to explain how one thing causes another. How can that be? How could there be many ways to understand causality?

Think back to our discussion in Section 5.3 on paradigms [insert chapter link plus link to section 1.2]. You’ll remember the positivist paradigm as the one that believes in objectivity. Positivists look for causal explanations that are universally true for everyone, everywhere  because they seek objective truth. Interpretivists, on the other hand, look for causal explanations that are true for individuals or groups in a specific time and place because they seek subjective truths. Remember that for interpretivists, there is not one singular truth that is true for everyone, but many truths created and shared by others.

"Are you trying to generalize or nah?"

One of my favorite classroom moments occurred in the early days of my teaching career. Students were providing peer feedback on their working questions. I overheard one group who was helping someone rephrase their research question. A student asked, “Are you trying to generalize or nah?” Teaching is full of fun moments like that one. Answering that one question can help you understand how to conceptualize and design your research project.

Nomothetic causal explanations are incredibly powerful. They allow scientists to make predictions about what will happen in the future, with a certain margin of error. Moreover, they allow scientists to generalize —that is, make claims about a large population based on a smaller sample of people or items. Generalizing is important. We clearly do not have time to ask everyone their opinion on a topic or test a new intervention on every person. We need a type of causal explanation that helps us predict and estimate truth in all situations.

Generally, nomothetic causal relationships work best for explanatory research projects [INSERT SECTION LINK]. They also tend to use quantitative research: by boiling things down to numbers, one can use the universal language of mathematics to use statistics to explore those relationships. On the other hand, descriptive and exploratory projects often fit better with idiographic causality. These projects do not usually try to generalize, but instead investigate what is true for individuals, small groups, or communities at a specific point in time. You will learn about this type of causality in the next section. Here, we will assume you have an explanatory working question. For example, you may want to know about the risk and protective factors for a specific diagnosis or how a specific therapy impacts client outcomes.

What do nomothetic causal explanations look like?

Nomothetic causal explanations express relationships between variables . The term variable has a scientific definition. This one from Gillespie & Wagner (2018) "a logical grouping of attributes that can be observed and measured and is expected to vary from person to person in a population" (p. 9). [91] More practically, variables are the key concepts in your working question. You know, the things you plan to observe when you actually do your research project, conduct your surveys, complete your interviews, etc. These things have two key properties. First, they vary , as in they do not remain constant. "Age" varies by number. "Gender" varies by category. But they both vary. Second, they have attributes . So the variable "health professions" has attributes or categories, such as social worker, nurse, counselor, etc.

It's also worth reviewing what is  not a variable. Well, things that don't change (or vary) aren't variables. If you planned to do a study on how gender impacts earnings but your study only contained women, that concept would not vary . Instead, it would be a constant . Another common mistake I see in students' explanatory questions is mistaking an attribute for a variable. "Men" is not a variable. "Gender" is a variable. "Virginia" is not a variable. The variable is the "state or territory" in which someone or something is physically located.

When one variable causes another, we have what researchers call independent and dependent variables. For example, in a study investigating the impact of spanking on aggressive behavior, spanking would be the independent variable and aggressive behavior would be the dependent variable. An independent variable is the cause, and a  dependent variable  is the effect. Why are they called that? Dependent variables  depend on independent variables. If all of that gets confusing, just remember the graphical relationship in Figure 8.5.

Write out your working question, as it exists now. As we said previously in the subsection, we assume you have an explanatory research question for learning this section.

• Write out a diagram similar to Figure 8.5.
• Put your independent variable on the left and the dependent variable on the right.
• Can your variables vary?
• Do they have different attributes or categories that vary from person to person?
• How does the theory you identified in section 8.1 help you understand this causal relationship?

If the theory you've identified isn't much help to you or seems unrelated, it's a good indication that you need to read more literature about the theories related to your topic.

For some students, your working question may not be specific enough to list an independent or dependent variable clearly. You may have "risk factors" in place of an independent variable, for example. Or "effects" as a dependent variable. If that applies to your research question, get specific for a minute even if you have to revise this later. Think about which specific risk factors or effects you are interested in. Consider a few options for your independent and dependent variable and create diagrams similar to Figure 8.5.

Finally, you are likely to revisit your working question so you may have to come back to this exercise to clarify the causal relationship you want to investigate.

For a ten-cent word like "nomothetic," these causal relationships should look pretty basic to you. They should look like "x causes y." Indeed, you may be looking at your causal explanation and thinking, "wow, there are so many other things I'm missing in here." In fact, maybe my dependent variable sometimes causes changes in my independent variable! For example, a working question asking about poverty and education might ask how poverty makes it more difficult to graduate college or how high college debt impacts income inequality after graduation. Nomothetic causal relationships are slices of reality. They boil things down to two (or often more) key variables and assert a one-way causal explanation between them. This is by design, as they are trying to generalize across all people to all situations. The more complicated, circular, and often contradictory causal explanations are idiographic, which we will cover in the next section of this chapter.

Developing a hypothesis

A hypothesis   is a statement describing a researcher’s expectation regarding what they anticipate finding. Hypotheses in quantitative research are a nomothetic causal relationship that the researcher expects to determine is true or false. A hypothesis is written to describe the expected relationship between the independent and dependent variables. In other words, write the answer to your working question using your variables. That's your hypothesis! Make sure you haven't introduced new variables into your hypothesis that are not in your research question. If you have, write out your hypothesis as in Figure 8.5.

A good hypothesis should be testable using social science research methods. That is, you can use a social science research project (like a survey or experiment) to test whether it is true or not. A good hypothesis is also  specific about the relationship it explores. For example, a student project that hypothesizes, "families involved with child welfare agencies will benefit from Early Intervention programs," is not specific about what benefits it plans to investigate. For this student, I advised her to take a look at the empirical literature and theory about Early Intervention and see what outcomes are associated with these programs. This way, she could  more clearly state the dependent variable in her hypothesis, perhaps looking at reunification, attachment, or developmental milestone achievement in children and families under child welfare supervision.

Your hypothesis should be an informed prediction based on a theory or model of the social world. For example, you may hypothesize that treating mental health clients with warmth and positive regard is likely to help them achieve their therapeutic goals. That hypothesis would be based on the humanistic practice models of Carl Rogers. Using previous theories to generate hypotheses is an example of deductive research. If Rogers’ theory of unconditional positive regard is accurate, a study comparing clinicians who used it versus those who did not would show more favorable treatment outcomes for clients receiving unconditional positive regard.

Let’s consider a couple of examples. In research on sexual harassment (Uggen & Blackstone, 2004), [92] one might hypothesize, based on feminist theories of sexual harassment, that more females than males will experience specific sexually harassing behaviors. What is the causal relationship being predicted here? Which is the independent and which is the dependent variable? In this case, researchers hypothesized that a person’s sex (independent variable) would predict their likelihood to experience sexual harassment (dependent variable).

Sometimes researchers will hypothesize that a relationship will take a specific direction. As a result, an increase or decrease in one area might be said to cause an increase or decrease in another. For example, you might choose to study the relationship between age and support for legalization of marijuana. Perhaps you’ve taken a sociology class and, based on the theories you’ve read, you hypothesize that age is negatively related to support for marijuana legalization. [93] What have you just hypothesized?

You have hypothesized that as people get older, the likelihood of their supporting marijuana legalization decreases. Thus, as age (your independent variable) moves in one direction (up), support for marijuana legalization (your dependent variable) moves in another direction (down). So, a direct relationship (or positive correlation) involve two variables going in the same direction and an inverse relationship (or negative correlation) involve two variables going in opposite directions. If writing hypotheses feels tricky, it is sometimes helpful to draw them out and depict each of the two hypotheses we have just discussed.

It’s important to note that once a study starts, it is unethical to change your hypothesis to match the data you find. For example, what happens if you conduct a study to test the hypothesis from Figure 8.7 on support for marijuana legalization, but you find no relationship between age and support for legalization? It means that your hypothesis was incorrect, but that’s still valuable information. It would challenge what the existing literature says on your topic, demonstrating that more research needs to be done to figure out the factors that impact support for marijuana legalization. Don’t be embarrassed by negative results, and definitely don’t change your hypothesis to make it appear correct all along!

Criteria for establishing a nomothetic causal relationship

Let’s say you conduct your study and you find evidence that supports your hypothesis, as age increases, support for marijuana legalization decreases. Success! Causal explanation complete, right? Not quite.

You’ve only established one of the criteria for causality. The criteria for causality must include all of the following: covariation, plausibility, temporality, and nonspuriousness. In our example from Figure 8.7, we have established only one criteria—covariation. When variables covary , they vary together. Both age and support for marijuana legalization vary in our study. Our sample contains people of varying ages and varying levels of support for marijuana legalization. If, for example, we only included 16-year-olds in our study, age would be a  constant , not a variable.

Just because there might be some correlation between two variables does not mean that a causal relationship between the two is really plausible. Plausibility means that in order to make the claim that one event, behavior, or belief causes another, the claim has to make sense. It makes sense that people from previous generations would have different attitudes towards marijuana than younger generations. People who grew up in the time of Reefer Madness or the hippies may hold different views than those raised in an era of legalized medicinal and recreational use of marijuana. Plausibility is of course helped by basing your causal explanation in existing theoretical and empirical findings.

Once we’ve established that there is a plausible relationship between the two variables, we also need to establish whether the cause occurred before the effect, the criterion of temporality . A person’s age is a quality that appears long before any opinions on drug policy, so temporally the cause comes before the effect. It wouldn’t make any sense to say that support for marijuana legalization makes a person’s age increase. Even if you could predict someone’s age based on their support for marijuana legalization, you couldn’t say someone’s age was caused by their support for legalization of marijuana.

Finally, scientists must establish nonspuriousness. A spurious relationship is one in which an association between two variables appears to be causal but can in fact be explained by some third variable. This third variable is often called a confound or confounding variable because it clouds and confuses the relationship between your independent and dependent variable, making it difficult to discern the true causal relationship is.

Continuing with our example, we could point to the fact that older adults are less likely to have used marijuana recreationally. Maybe it is actually recreational use of marijuana that leads people to be more open to legalization, not their age. In this case, our confounding variable would be recreational marijuana use. Perhaps the relationship between age and attitudes towards legalization is a spurious relationship that is accounted for by previous use. This is also referred to as the third variable problem , where a seemingly true causal relationship is actually caused by a third variable not in the hypothesis. In this example, the relationship between age and support for legalization could be more about having tried marijuana than the age of the person.

Quantitative researchers are sensitive to the effects of potentially spurious relationships. As a result, they will often measure these third variables in their study, so they can control for their effects in their statistical analysis. These are called  control variables , and they refer to potentially confounding variables whose effects are controlled for mathematically in the data analysis process. Control variables can be a bit confusing, and we will discuss them more in Chapter 10, but think about it as an argument between you, the researcher, and a critic.

Researcher: “The older a person is, the less likely they are to support marijuana legalization.” Critic: “Actually, it’s more about whether a person has used marijuana before. That is what truly determines whether someone supports marijuana legalization.” Researcher: “Well, I measured previous marijuana use in my study and mathematically controlled for its effects in my analysis. Age explains most of the variation in attitudes towards marijuana legalization.”

Let’s consider a few additional, real-world examples of spuriousness. Did you know, for example, that high rates of ice cream sales have been shown to cause drowning? Of course, that’s not really true, but there is a positive relationship between the two. In this case, the third variable that causes both high ice cream sales and increased deaths by drowning is time of year, as the summer season sees increases in both (Babbie, 2010). [94]

Here’s another good one: it is true that as the salaries of Presbyterian ministers in Massachusetts rise, so too does the price of rum in Havana, Cuba. Well, duh, you might be saying to yourself. Everyone knows how much ministers in Massachusetts love their rum, right? Not so fast. Both salaries and rum prices have increased, true, but so has the price of just about everything else (Huff & Geis, 1993). [95]

Finally, research shows that the more firefighters present at a fire, the more damage is done at the scene. What this statement leaves out, of course, is that as the size of a fire increases so too does the amount of damage caused as does the number of firefighters called on to help (Frankfort-Nachmias & Leon-Guerrero, 2011). [96] In each of these examples, it is the presence of a confounding variable that explains the apparent relationship between the two original variables.

In sum, the following criteria must be met for a nomothetic causal relationship:

• The two variables must vary together.
• The relationship must be plausible.
• The cause must precede the effect in time.
• The relationship must be nonspurious (not due to a confounding variable).

The hypothetico-dedutive method

The primary way that researchers in the positivist paradigm use theories is sometimes called the hypothetico-deductive method (although this term is much more likely to be used by philosophers of science than by scientists themselves). Researchers choose an existing theory. Then, they make a prediction about some new phenomenon that should be observed if the theory is correct. Again, this prediction is called a hypothesis. The researchers then conduct an empirical study to test the hypothesis. Finally, they reevaluate the theory in light of the new results and revise it if necessary.

This process is usually conceptualized as a cycle because the researchers can then derive a new hypothesis from the revised theory, conduct a new empirical study to test the hypothesis, and so on. As Figure 8.8 shows, this approach meshes nicely with the process of conducting a research project—creating a more detailed model of “theoretically motivated” or “theory-driven” research. Together, they form a model of theoretically motivated research. 

Keep in mind the hypothetico-deductive method is only one way of using social theory to inform social science research. It starts with describing one or more existing theories, deriving a hypothesis from one of those theories, testing your hypothesis in a new study, and finally reevaluating the theory based on the results data analyses. This format works well when there is an existing theory that addresses the research question—especially if the resulting hypothesis is surprising or conflicts with a hypothesis derived from a different theory.

But what if your research question is more interpretive? What if it is less about theory-testing and more about theory-building? This is what our next chapters will cover: the process of inductively deriving theory from people's stories and experiences. This process looks different than that depicted in Figure 8.8. It still starts with your research question and answering that question by conducting a research study. But instead of testing a hypothesis you created based on a theory, you will create a theory of your own that explain the data you collected. This format works well for qualitative research questions and for research questions that existing theories do not address.

• In positivist and quantitative studies, the goal is often to understand the more general causes of some phenomenon rather than the idiosyncrasies of one particular instance, as in an idiographic causal relationship.
• Nomothetic causal explanations focus on objectivity, prediction, and generalization.
• Criteria for nomothetic causal relationships require the relationship be plausible and nonspurious; and that the cause must precede the effect in time.
• In a nomothetic causal relationship, the independent variable causes changes in the dependent variable.
• Hypotheses are statements, drawn from theory, which describe a researcher’s expectation about a relationship between two or more variables.
• Write out your working question and hypothesis.
• Defend your hypothesis in a short paragraph, using arguments based on the theory you identified in section 8.1.
• Review the criteria for a nomothetic causal relationship. Critique your short paragraph about your hypothesis using these criteria.
• Are there potentially confounding variables, issues with time order, or other problems you can identify in your reasoning?

Inductive & deductive (deductive focus)

9. Writing your research question Copyright © 2020 by Matthew DeCarlo is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

10 Qualities of a Good Researcher: Quest for Excellence

• Post author By admin
• November 9, 2023

Discover the essential 10 qualities of a good researcher! Uncover the traits that drive success in the world of research. Learn what it takes to excel in the quest for knowledge and innovation

Suppose a vast landscape of knowledge, uncharted and waiting to be discovered. Research is the compass guiding us through this territory, and at the helm of every great exploration stands a good researcher.

But what sets them apart? It’s not just knowledge; it’s a unique set of qualities that propel them towards understanding.

In this journey, we’ll uncover the very essence of a good researcher. We’ll delve into the top 10 qualities that define them. From unquenchable curiosity to unwavering perseverance, these qualities are the secret sauce behind their success in academia and exploration.

Whether you’re already treading the path of research or gearing up for the adventure, understanding and embracing these qualities will transform you into a research dynamo. So, let’s embark on this quest to unravel what makes a good researcher tick.

Table of Contents

10 Qualities of a Good Researcher

Check out the 10 qualities of a good researcher:-

1. Inquisitiveness: The Craving for Knowledge

Think of a good researcher as that friend who’s always full of questions. They’re the eternal curious cats of the academic world, forever wondering, forever seeking, and forever hungry for knowledge. It’s like they have a built-in “Why?” button that never switches off.

A good researcher’s inquisitiveness is like the spark that lights up a dark room. It’s what pushes them to ask the questions no one else has thought of and venture into uncharted territories. They’re the ultimate seekers, the champions of “What if?” and “Why not?” It’s this insatiable curiosity that keeps their research fresh, exciting, and always on the hunt for more knowledge.

2. Patience: Sifting Through Data

Imagine a good researcher as a treasure hunter in the vast desert of data. Research can sometimes feel like slogging through quicksand – slow, meticulous, and demanding. But here’s the thing: good researchers have an incredible treasure map, and it’s called “patience.”

They understand that research isn’t a race; it’s a journey. It’s about sifting through tons of data, the way a prospector pans for gold. Every grain of information matters, and they’re willing to invest the time needed to collect, analyze, and interpret data accurately.

This patience isn’t about twiddling thumbs; it’s about meticulously building the puzzle of knowledge, piece by piece. They understand that no detail is too small to be overlooked, and in the end, it’s these small pieces that complete the big picture.

Good researchers don’t rush; they savor the journey, knowing that the best discoveries often lie in the details. They are the patient architects of knowledge, and it’s their patience that ensures that no gem of information goes undiscovered.

3. Attention to Detail: Devil in the Details

In research, it’s the little things that matter most. A good researcher understands this like no other. They’re the ones who spot the faintest footprints in the sand and the almost invisible fingerprints on the glass because they know that in research, the devil truly lies in the details.

For them, every piece of information is a precious puzzle piece. They’re like puzzle enthusiasts, and they’re determined to find and fit every piece perfectly. Because, in their world, even the tiniest detail holds the potential to make or break a study.

In a realm where precision reigns supreme, good researchers are the vigilant guardians of information. They’re the ones who make sure no stone is left unturned, no detail is too minor, and it’s this unwavering attention to detail that transforms their research into something truly extraordinary.

4. Critical Thinking: Questioning the Norm

Let’s picture a good researcher as the ultimate rebel of the research realm. They don’t just follow the herd; they’re the ones breaking the mold, challenging established theories, and stirring up the intellectual pot. Their secret weapon? It’s called critical thinking.

Critical thinking is like their sidekick, the Watson to their Holmes. It’s their power to look at information with a discerning eye, to cut through the noise, and make informed judgments. Good researchers? They’ve got critical thinking in their toolkit, and they’re not afraid to use it.

They’re not content with nodding along to the norm. No, they’re the ones who dare to ask, “Why?” and “What if?” They’re the Sherlock Holmes of academia, seeking the hidden clues that others might overlook. They’re the explorers who venture beyond the boundaries of convention.

For them, curiosity isn’t just a casual interest; it’s a full-blown investigation. They’re the skeptics, the truth-seekers, and the challengers of the status quo. Because they know that the road to enlightenment is paved with skepticism and paved with profound insights.

In a world where knowledge is the ultimate treasure, good researchers are the rebels with a cause. They’re the ones who question, challenge, and redefine the norm, making the pursuit of knowledge a thrilling adventure.

5. Organization: Chaos to Clarity

Let’s paint a mental picture of a good researcher as the master organizer of the research universe. Picture this: researchers often find themselves wading through mountains of data, like explorers in an information jungle.

But what sets good researchers apart is their exceptional skill in turning chaos into clarity through one magic word – organization.

These researchers are like the conductors of a grand symphony, where data plays the melodious tunes. They understand that without a meticulously organized score, the music may fall into chaos.

This is why they keep their work structured and well-organized. It’s like having a treasure map to navigate through the data wilderness.

For them, organization isn’t just a preference; it’s a necessity. It ensures that every piece of data, every note in the symphony, can be easily accessed and referenced when needed. It’s the librarian’s skill of categorizing, labeling, and arranging knowledge in a way that makes sense.

In a world where data can be overwhelming, good researchers are the navigators who chart the course from chaos to clarity. They bring order to the information realm, making sure that every piece of data finds its place in the grand mosaic of knowledge.

6. Effective Communication: Sharing Insights

Imagine a good researcher as not just a discoverer of hidden treasures but also a gifted storyteller. Research isn’t merely about uncovering the unknown; it’s about sharing those discoveries with the world. Good researchers possess a unique superpower – effective communication.

They are the bards of academia, able to weave intricate tales of data and insight. It’s not enough to gather knowledge; they understand the importance of conveying it to their peers and the wider community. They’re like skilled translators, turning complex data into understandable narratives.

For them, research isn’t a solitary endeavor but a communal one. They can articulate their findings, transforming raw data into gems of wisdom. They speak not just to fellow researchers but to anyone who seeks understanding.

In a world where information is abundant but understanding can be scarce, good researchers are the bridges that connect data to meaning. They’re the ones who bring clarity to complexity, ensuring that their discoveries benefit not just themselves but all who thirst for knowledge.

7. Ethical Integrity: The Moral Compass

Picture a good researcher as a moral compass, always pointing in the direction of what’s right. In their world, there’s no room for ethical shortcuts; they’re the guardians of integrity, setting the highest standards.

Ethical conduct is their unwavering principle, not a mere guideline. These researchers tread the path of knowledge with profound respect for all beings, be it humans, animals, or the environment.

They understand that research isn’t just about facts and figures; it’s about the impact on the world.

They are the ethical warriors who ensure that every discovery is made with the utmost respect for boundaries. They’re the ones who hold the torch of integrity, even when the road gets dark and uncertain.

In a world where ethical dilemmas can cloud the way, good researchers are the beacons of moral clarity. They remind us that the pursuit of knowledge should always be illuminated by the light of ethics, leaving a positive and lasting legacy.

8. Adaptability: Rolling with Research’s Twists

Now, picture a good researcher as the ultimate research ninja. They know that in the world of research, surprises are the name of the game. What makes them exceptional? Their uncanny ability to adapt.

In their world, every research project is like a thrilling rollercoaster ride. They’re fully aware that not everything will go as planned.

But instead of dreading the unexpected, they welcome it with open arms. It’s not about dodging hurdles; it’s about using them as springboards for new discoveries.

Adaptability is their secret weapon. They don’t panic when faced with unexpected twists and turns; they thrive on them. They’re the daredevils of research, excited by the idea that every surprise brings a chance for a breakthrough.

They understand that research isn’t a linear path; it’s an expedition full of surprises. Good researchers approach each twist and turn as a new opportunity to learn, grow, and uncover the unknown.

9. Perseverance: Never Giving Up

Now, picture a good researcher as the indomitable hero of the research saga. The journey to groundbreaking discoveries is no walk in the park; it’s an epic adventure filled with obstacles and trials. What makes a good researcher extraordinary? Their unshakable perseverance.

In their world, setbacks are not dead ends; they are the very soil in which success takes root. They grasp that the path to pioneering research is not a sprint but a demanding marathon.

When confronted with challenges, they don’t retreat; they roll up their sleeves and forge ahead with unwavering resolve.

In their universe, perseverance is the North Star guiding them through the darkest nights of research. It’s the fire that keeps them warm when faced with the chilling winds of doubt.

They understand that every stumble is a lesson, every hurdle is an opportunity, and every fall is a chance to rise even higher.

In a realm where remarkable discoveries are born from sheer determination, good researchers are the embodiment of perseverance.

They don’t just weather the storms of research; they harness them to soar to new heights of understanding and innovation.

10. Problem-Solving Skills: Creative and Determined Issue Resolution

Think of a good researcher as a maverick in the world of problem-solving. They possess an innate ability to tackle research-related issues with a unique blend of creativity and unwavering determination. They’re not just issue-spotters; they’re issue-solvers.

In their realm, challenges aren’t roadblocks; they’re opportunities for innovation. Whether it’s deciphering a complex data conundrum, navigating unexpected research detours, or confronting formidable roadblocks, they approach each problem with a dash of unconventional thinking.

Their toolkit isn’t limited to traditional solutions; it includes a healthy dose of creativity. They know that sometimes the most extraordinary answers emerge from unconventional thinking.

When faced with adversity, they don’t back down; they dive headfirst into the challenge, armed with resourcefulness and an unyielding spirit.

In the world of research, where every obstacle conceals a chance for a groundbreaking discovery, these good researchers are the daring explorers.

They turn problems into springboards, propelling the journey of knowledge and unveiling new insights along the way.

What is the qualities of good researcher?

Exceptional researchers are a unique breed, possessing a blend of innate traits and developed skills that set them apart in the world of discovery. Here are the qualities that define an outstanding researcher:

Inherent Curiosity

Exceptional researchers are born with an insatiable curiosity about the world. They perpetually question, driven by an unrelenting thirst for knowledge. This curiosity fuels their exploration of new ideas and their deep dives into complex problems.

Independence and Initiative

They are fiercely independent, unafraid to challenge conventions and think outside the box. This independence empowers them to conduct research with rigor and objectivity, free from preconceived notions.

Critical Thinking

Exceptional researchers are expert critical thinkers. They scrutinize information, identifying biases and assumptions. This skill enables them to draw well-founded conclusions from their research, undeterred by misinformation.

Effective Communication

They are adept communicators, capable of presenting their findings clearly and concisely. Their ability to convey complex ideas is vital for sharing their discoveries with the broader scientific community.

Collaboration Prowess

Collaboration is second nature to them. Exceptional researchers seamlessly collaborate with others to achieve common research objectives. Their skill in teamwork is essential for handling large-scale research projects effectively.

Problem-Solving Expertise

Problem-solving is in their DNA. They spot issues, conceive and test solutions, and rigorously evaluate their effectiveness. This skill is the backbone of conducting thorough research.

In addition to these qualities, exceptional researchers boast an in-depth understanding of their chosen field. They stay abreast of the latest research findings and expertly apply this knowledge to their own work.

Furthermore, they adhere to ethical guidelines that govern research, conducting their inquiries responsibly and ethically.

Armed with these remarkable qualities, exceptional researchers not only expand our comprehension of the world but also contribute to solving critical problems and enhancing the quality of life for all.

What are the 7 major characteristics of research?

Research is a multifaceted endeavor, marked by seven pivotal characteristics that define its essence:

1, Empirical Foundation

At its core, research is grounded in empiricism. It shuns opinions, personal beliefs, and conjecture. Instead, it thrives on data and evidence drawn from real-world observations and experiments, bolstering its conclusions with solid support.

2. Systematic Approach

Research unfolds systematically, adhering to a meticulously designed process. It commences with defining the research question, identifying research methods, collecting data, rigorously analyzing it, and ultimately deriving well-founded conclusions. This systematic journey ensures both rigor and objectivity.

3. Logical Underpinning

Logic forms the backbone of research. It forges conclusions that harmonize seamlessly with the laws of logic, yielding findings that are not only profound but also reliable.

4. Cyclical Nature

Research possesses a cyclical essence. It commences with a question or problem, each exploration invariably begetting new inquiries. This continuous cycle propels researchers toward a deeper understanding of the ever-evolving world.

5. Analytical Rigor

Research demands meticulous data analysis. Researchers employ diverse analytical techniques to uncover patterns, trends, and relationships within the data. This scrutiny unveils the latent significance of the data, facilitating the derivation of meaningful conclusions.

6. Objective Stance

An unwavering objectivity characterizes research. Researchers diligently strive to avoid bias and partiality, ensuring that their personal beliefs or opinions exert no undue influence on their findings.

7. Replicability Standard

Research adheres to a replicability standard. Other researchers should be capable of replicating the study and achieving congruent results. This commitment to replicability bolsters the reliability and validity of research findings.

Incorporating these seven key characteristics, research emerges as a powerful tool for the exploration of the unknown, the validation of hypotheses, and the continuous advancement of knowledge.

What are the 3 important qualities of a good research?

When we delve into the world of outstanding research, we uncover the pillars that set it apart. Imagine these as the main characters in a compelling story:

1. Credibility

This is the unwavering foundation. Exceptional research is built on solid evidence and meticulous reasoning. It follows a rigorous and objective path, supported by thorough data and in-depth analysis.

2. Relevance

Consider this the heart of the matter. Exceptional research doesn’t shy away from addressing pressing questions and challenges.

It aims to contribute significantly to our understanding of the world and has the potential to solve crucial problems.

3. Originality

Think of this as the trailblazer, the innovator. Exceptional research ventures into uncharted territories, offering fresh and unique perspectives.

It doesn’t retrace well-worn paths; instead, it opens new doors to insights that haven’t been explored before.

These are the three pillars of remarkable research, igniting our quest to comprehend our world more deeply, confront significant challenges, and provide solutions that truly enhance our lives and the lives of those around us.

What are the 4 characteristics of a good research?

When we delve into the world of research, we discover the four cornerstones that define what makes research truly exceptional:

Imagine research as a sturdy ship navigating the vast sea of knowledge. What keeps it afloat? Credibility – the anchor of solid evidence and logical reasoning.

It’s about following a rigorous and objective methodology, with findings firmly supported by a wealth of data and meticulous analysis.

Good research is like a compass pointing to the critical questions and challenges that pique the curiosity of the research community and society.

It’s not just an exploration; it’s a journey with a purpose – to deepen our understanding of the world and unravel solutions to the most pressing problems.

Think of research as an explorer venturing into uncharted territory. It doesn’t follow the trodden paths; it forges its own.

Good research doesn’t echo what’s been said before; it blazes new trails, offering fresh insights and unique perspectives.

Effective research is a lighthouse, guiding others through the maze of complexity. Its findings are not buried in jargon or obscured by ambiguity.

They are presented with clarity and conciseness, ensuring that everyone can navigate the discoveries with ease.

These attributes, like the North Star, lead us in the pursuit of knowledge and understanding, casting light on the uncharted waters of research.

In the grand tapestry of knowledge, good researchers stand as the weavers of profound discovery. They embody a unique blend of qualities, shaping the course of understanding and change.

From the inquisitiveness that fuels their journey to the unwavering patience that carries them through the most intricate of labyrinths, these qualities are the compass, the guiding light.

The unquenchable curiosity of a good researcher keeps the embers of exploration burning bright. Patience, the steadfast companion, ensures that no detail remains in obscurity.

Their critical thinking propels them beyond the boundaries of convention, unraveling new layers of understanding.

In the chaos of data, they find serenity through organization, and in the midst of complexity, they wield the sword of effective communication.

Ethical integrity acts as their moral compass, while adaptability embraces the unpredictability of research’s twists.

But it’s perseverance, the indomitable spirit, that carries them through the darkest hours. They recognize that the path to groundbreaking research is often fraught with obstacles, but those obstacles serve as stepping stones to success.

These ten qualities, woven into the very fabric of their being, make good researchers the architects of transformation.

With every study they undertake, they draw closer to unraveling the mysteries of our world, bridging gaps in knowledge, and contributing to the betterment of humanity.

As we celebrate these qualities, we acknowledge the significance of their work. Through their endeavors, we glimpse the limitless potential of human exploration, and we are inspired to never cease questioning, exploring, and, above all, learning.

Frequently Asked Questions

Can anyone become a good researcher.

Yes, with dedication and a willingness to develop these qualities, anyone can become a good researcher.

Why is adaptability crucial for a researcher?

Research is unpredictable, and adaptability allows researchers to navigate unexpected challenges effectively.

What role does ethics play in research?

Ethical integrity is vital in research to ensure the well-being of participants and the integrity of the study.

How do researchers maintain their inquisitiveness?

Researchers stay curious by continually seeking new questions and exploring uncharted territories in their field.

Is critical thinking a natural talent, or can it be developed?

Critical thinking can be developed through practice and a commitment to questioning and evaluating information.

• australia (2)
• duolingo (13)
• Education (284)
• General (77)
• How To (18)
• IELTS (127)
• Latest Updates (162)
• Malta Visa (6)
• Permanent residency (1)
• Programming (31)
• Scholarship (1)
• Sponsored (4)
• Study Abroad (187)
• Technology (12)
• work permit (8)

Recent Posts

Literature Searching

Characteristics of a good research question

The first step in a literature search is to construct a well-defined question.  This helps in ensuring a comprehensive and efficient search of the available literature for relevant publications on your topic.  The well-constructed research question provides guidance for determining search terms and search strategy parameters.

A good or well-constructed research question is:

• Original and of interest to the researcher and the outside world
• It is clear and focused: it provides enough specifics that it is easy to understand its purpose and it is narrow enough that it can be answered. If the question is too broad it may not be possible to answer it thoroughly. If it is too narrow you may not find enough resources or information to develop a strong argument or research hypothesis.  
• The question concept is researchable in terms of time and access to a suitable amount of quality research resources.
• It is analytical rather than descriptive.  The research question should allow you to produce an analysis of an issue or problem rather than a simple description of it.  In other words, it is not answerable with a simple “yes” or “no” but requires a synthesis and analysis of ideas and sources.
• The results are potentially important and may change current ideas and/or practice
• And there is the potential to develop further projects with similar themes

The question you ask should be developed for the discipline you are studying. A question appropriate for Physical Therapy, for instance, is different from an appropriate one in Sociology, Political Science or Microbiology .

The well-constructed question provides guidance for determining search terms and search strategy parameters. The process of developing a good question to research involves taking your topic and breaking each aspect of it down into its component parts. 

One well-established way that can be used both for creating research questions and developing strategies is known as PICO(T). The PICO framework was designed primarily for questions that include clinical interventions and comparisons, however other types of questions may also be able to follow its principles.  If the PICO framework does not precisely fit your question, using its principles can help you to think about what you want to explore even if you do not end up with a true PICO question.

References/Additional Resources

Fandino W. (2019). Formulating a good research question: Pearls and pitfalls.   Indian journal of anaesthesia ,  63 (8), 611–616. 

Vandenbroucke, J. P., & Pearce, N. (2018). From ideas to studies: how to get ideas and sharpen them into research questions .  Clinical epidemiology ,  10 , 253–264.

Ratan, S. K., Anand, T., & Ratan, J. (2019). Formulation of Research Question - Stepwise Approach .  Journal of Indian Association of Pediatric Surgeons ,  24 (1), 15–20.

Lipowski, E.E. (2008). Developing great research questions. American Journal of Health-System Pharmacy, 65(17) , 1667–1670.

FINER Criteria

Another set of criteria for developing a research question was proposed by Hulley (2013) and is known as the FINER criteria. 

FINER stands for:

Feasible – Writing a feasible research question means that it CAN be answered under objective aspects like time, scope, resources, expertise, or funding. Good questions must be amenable to the formulation of clear hypotheses.

Interesting – The question or topic should be of interest to the researcher and the outside world. It should have a clinical and/or educational significance – the “so what?” factor. 

Novel – In scientific literature, novelty defines itself by being an answer to an existing gap in knowledge. Filling one of these gaps is highly rewarding for any researcher as it may represent a real difference in peoples’ lives.

Good research leads to new information. An investigation which simply reiterates what is previously proven is not worth the effort and cost. A question doesn’t have to be completely original. It may ask whether an earlier observation could be replicated, whether the results in one population also apply to others, or whether enhanced measurement methods can make clear the relationship between two variables.  

Ethical – In empirical research, ethics is an absolute MUST. Make sure that safety and confidentiality measures are addressed, and according to the necessary IRB protocols.

Relevant – An idea that is considered relevant in the healthcare community has better chances to be discussed upon by a larger number of researchers and recognized experts, leading to innovation and rapid information dissemination.

The results could potentially be important and may change current ideas and/or practice.

Cummings, S.R., Browner, W.S., & Hulley, S.B. (2013). Conceiving the research question and developing the study plan. In: Designing clinical research (Hulley, S. R. Cummings, W. S. Browner, D. Grady, & T. B. Newman, Eds.; Fourth edition.). Wolters Kluwer/Lippincott Williams & Wilkins. Pp. 14-22.    

• << Previous: Major Steps in a Literature Search
• Next: Types of Research Questions >>

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

The Top 5 Qualities of Every Good Researcher

• 3 minute read
• 216.2K views

Table of Contents

What makes a good researcher? Is it some undefinable, innate genius, or is it something that we can practice and build upon? If it was just the former, then there would be far fewer innovations in the history of humankind than there have been. A careful look at researchers through the ages reveals that they all have certain attributes in common that have helped contribute to their success.

The characteristics of a good researcher:

1. curiosity.

They ask questions. An endless thirst for knowledge is what sets the best of the best apart from the others. Good researchers constantly strive to learn more, not just about their own field, but about other fields as well. The world around us is fascinating, be it the physics behind the way light refracts, or the anthropological constructions of our society. A good researcher keeps exploring the world and keeps searching for answers.

2. Analytical ability and foresight

They look for connections. Information is useless without interpretation. What drives research forward is finding meaning in our observations and data. Good researchers evaluate data from every angle and search for patterns. They explore cause and effect and untangle the tricky web that interconnects everyday phenomena. And then take it one step further to ask, ‘What is the bigger picture? How will the research develop in the future?’

3. Determination

They try, try, and try again. Research can be a frustrating experience. Experiments may not pan out how we expect them to. Even worse, sometimes experiments may run smoothly until they are 95% complete before failing. What sets an average researcher apart from a truly good one? The truly good researcher perseveres. They accept this disappointment, learn from the failure, reevaluate their experiment, and keep moving forward.

4. Collaboration

Teamwork makes the dream work. Contrary to the common perception of the solitary genius in their lab, research is an extremely collaborative process. There is simply too much to do for just one person to do it all. Moreover, research is becoming increasingly multidisciplinary. It is impossible for just one person to have expertise in all these fields. In general, research is conducted in teams , with each researcher having their individual roles and responsibilities. Being able to coordinate, communicate, and get along with team members is a major factor that can contribute to one’s success as a researcher.

5. Communication

They get their message across. Communication skills are an essential asset for every researcher. Not only do they have to communicate with their team members, but they also have to communicate with co-authors, journals, publishers, and funders. Whether it is writing a crisp and effective abstract, presenting at a conference, or writing a persuasive grant proposal to secure research funding, communication appears everywhere in a researcher’s life. The message in the old adage, ‘If a tree falls in the forest, but no one is around to hear it, does it make a sound?’ applies to research too. A discovery could be groundbreaking, but what is the use if the researcher can’t communicate this discovery to the rest of the world?

These are just a few of the skills required by researchers to make it to the top of their field. Other attributes like creativity and time management are also worth mentioning. Nevertheless, having one or more of these top five characteristics will make the research process smoother for you and increase the chances of positive results. Set yourself up for success by building up these skills, focusing on excellence, and asking for help when you need it. Elsevier Author Services is here to aid you at every step of the research journey. From translation services by experts in the field, to preparing your manuscript for publication, to helping you submit the best possible grant proposal, you can trust us to guide you in your journey to doing great research.

How to Write Research Background: Key Points and Case Studies

Writing an Effective Cover Letter for Manuscript Resubmission

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

What Makes a Good Research Study?

Find out what separates a solid research study from a so-so one..

Posted March 31, 2018

• What Is ADHD?
• Take our ADHD Test
• Find a therapist to help with ADHD

One day you read online that drinking coffee reduces the chances of having age-related memory decline . You start drinking coffee. The next month you read that drinking coffee increases your chances of having age-related memory decline. What gives? In this article, you will learn how to "decipher" research studies to figure out what a research study is really saying - and what it doesn't say. You'll also discover how to tell if the reporting on a particular study was accurate or not. Ask yourself the following six questions when looking at a research study. Keep in mind these are just six of the many factors that make up a "clean" study.

1. Did the study use a placebo , and were the staff blinded to treatment?

The brain is very susceptible to placebos. There is evidence that even when you tell study subjects (participants) that they are getting a placebo, they improve (Carvalho, et al., 2016). In pharmaceutical studies, the US Food and Drug Administration (FDA) requires pharmaceutical companies to do double-blind placebo-controlled studies. This means that the study subjects, the physicians dispensing the drug, and the clinicians rating the subjects' behavior don't know what subjects are getting - drug or placebo. This eliminates a lot of bias , and it helps show whether the drug actually works.

2. Was there a bogus/sham treatment?

A bogus/sham treatment is one in which subjects are given a treatment that looks very much like the real treatment, except for one major difference. The bogus/sham treatment doesn't actually provide the therapeutic part of the treatment. For example, some acupuncture studies use a sham/bogus treatment, such as a 2017 study by Ugurlu, et al. regarding acupuncture treatment for fibromyalgia .

Bogus/sham treatments, when compared to active treatments, help researchers discover whether the active treatment is what works, or the fact that people think they are getting the active treatment.

3. How many people were there in the study (N)?

Logic says the more people you have in a study, or the study's "N", the better chance you have of your study representing of the general population (the "generalizability" of a study). Let's say you're studying the effects of apple juice on ADHD symptoms, and you have a total N of ten people. By chance, seven of those ten people have severe ADHD, two have moderate ADHD, and one has mild ADHD. You now could throw off the results of your study because you have so many people with severe ADHD in the study. When you have more subjects, or a larger N, in a study, there is more of a chance that you would have people that have mild, moderate, and severe ADHD.

4. Were the study groups randomized?

A good study randomizes their subjects into the active treatment and placebo groups. This means that the subjects are in those particular groups by chance. This provides extra "backup" that the effects from a treatment were actually from that treatment, not from study staff bias.

5. Who conducted the research, and who is paying for it?

If the people that created a treatment are also testing a treatment, this is a concern. When you have a horse in the race, so to speak, it is more difficult to be unbiased. Another concern is if an entity with a vested interest in a particular study outcome is paying for that study. For example, if there is a study on the effectiveness of widgets, and the sole source of funding is Widgets are Wonderful, Inc., and the researchers are employees of Widgets are Wonderful, that study better have some seriously good methodology to help eliminate bias. Even better, an independent research group is funded by an organization without ties to the study outcome.

6. Was the article published in a refereed (peer-reviewed/scholarly) journal?

In a refereed journal, a manuscript is reviewed by other experts in the field before it is published as an article. The authors of the manuscript are not disclosed to the reviewers, in order to reduce possible bias. When we review manuscripts for a journal, there are three main categories: reject, meaning the article goes no further; accept, with revisions, meaning the authors must edit their article before resubmitting it for publication; and accept as written, which is rare, but once in a while there is a manuscript with such good research methodology and writing that no additional editing is needed.

When a journal is not refereed, the standard of inclusion into that journal is not as high. This means the quality of the research may not be up to the same standards. Look up the journal online to find out if it is a peer-reviewed journal.

If you don't have university access, you can at least access the abstracts of journal articles at Google Scholar . The abstract lets you know the study's methodology, the number of study subjects, the outcomes, and the author's conclusions.

You may also see the term "open-access" used to describe a journal. An open-access journal is one that users can freely access, without a subscription or fees. Some open-access journals are peer-reviewed, some are not.

Copyright 2018 Sarkis Media

Carvalho, C., Caetano, J. M., Cunha, L., Rebouta, P., Kaptchuk, T. J., & Kirsch, I. (2016). Open-label Placebo Treatment in Chronic Low Back Pain: A Randomized Controlled Trial. Pain, 157(12), 2766–2772. http://doi.org/10.1097/j.pain.0000000000000700

Uğurlu, F. G., Sezer, N., Aktekin, L., Fidan, F., Tok, F., & Akkuş, S. (2017). The effects of acupuncture versus sham acupuncture in the treatment of fibromyalgia: a randomized controlled clinical trial. Acta reumatologica portuguesa, (1).

Stephanie Moulton Sarkis, Ph.D., N.C.C., D.C.M.H.S., L.M.H.C ., is the author of Gaslighting: Recognize Manipulative and Emotionally Abusive People — and Break Free .

• Find a Therapist
• Find a Treatment Center
• Find a Psychiatrist
• Find a Support Group
• Find Online Therapy
• United States
• Brooklyn, NY
• Chicago, IL
• Houston, TX
• Los Angeles, CA
• New York, NY
• Portland, OR
• San Diego, CA
• San Francisco, CA
• Seattle, WA
• Washington, DC
• Asperger's
• Bipolar Disorder
• Chronic Pain
• Eating Disorders
• Passive Aggression
• Personality
• Goal Setting
• Positive Psychology
• Stopping Smoking
• Low Sexual Desire
• Relationships
• Child Development
• Self Tests NEW
• Therapy Center
• Diagnosis Dictionary
• Types of Therapy

It’s increasingly common for someone to be diagnosed with a condition such as ADHD or autism as an adult. A diagnosis often brings relief, but it can also come with as many questions as answers.

• Emotional Intelligence
• Gaslighting
• Affective Forecasting
• Neuroscience

Characteristics of Good Research

Research is a systematic and scientific investigation of a specific topic or problem. It is a way of gathering and analyzing information in order to increase our understanding of a subject or issue. Research can be conducted in various fields, such as science, medicine, social science, education, and business. 

It can involve collecting and analyzing data, conducting experiments, studying literature and other sources, and using various research methods to draw conclusions. The ultimate goal of research is to generate new knowledge, solve problems, or answer questions that contribute to the advancement of a particular field of study.

What are the Features of A Good Research

Good research is characterized by several key features that ensure its reliability, validity, and usefulness. Here are some of the most important characteristics:

• Clear Objectives and Purpose : Good research starts with a clear and focused research question or hypothesis. It should have a specific aim or set of objectives that are feasible and sufficiently narrow to be thoroughly investigated.
• Well-structured Research Design : It involves a well-thought-out methodology that clearly defines the research process, including the selection of appropriate methods for data collection and analysis.
• Rigorous and Methodical : High-quality research is conducted rigorously and methodically, following established protocols and standards to ensure the accuracy and reliability of the data collected.
• Systematic and Orderly : It is conducted in a systematic, organized manner, with careful planning and execution of each step of the research process.
• Empirical Evidence and Reproducibility : Good research is based on empirical evidence that can be observed or experimented with. It should be replicable, meaning that other researchers can repeat the study and achieve similar results.
• Critical Analysis and Interpretation : It involves a critical analysis of the data, considering all possible variables and interpreting the findings in an unbiased, objective manner.
• Ethical Considerations : Ethical considerations are paramount in good research. This includes ensuring the privacy and confidentiality of participants, obtaining informed consent, and avoiding any form of bias or conflict of interest.
• Valid and Reliable Results : The findings of the research should be both valid (accurately representing what they are intended to measure) and reliable (consistent over time and across different observers or measures).
• Transparent and Replicable : Good research practices involve documenting the process thoroughly, so others can understand, evaluate, and replicate the study.
• Contribution to Knowledge and Practical Implications : Finally, good research contributes to the body of knowledge in its field and often has practical implications, offering solutions, informing policy, or advancing understanding in a particular area.

These characteristics ensure that research is trustworthy, contributes significantly to its field, and can be used as a reliable foundation for further inquiry.

What are the 7 elements of research?

Related Posts

What is Research?

Types of Research

Leave a comment cancel reply.

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Educational resources and simple solutions for your research journey

Top 10 Qualities and Characteristics of a Good Researcher

Year after year, people with different personalities and backgrounds step into the field of research eager to develop the key qualities of a good researcher , only to find themselves faced with anxiety and self-doubt. Becoming a good researcher is a challenging task that requires a combination of skills and attributes as well as time, dedication, and a lot of hard work.   

So what are the qualities of a good researcher and how does one build these must-have characteristics? This article answers this by sharing the top 10 qualities of a good researcher that you must work to develop, strengthen, and apply on your journey to research success.   

Table of Contents

Top 10 qualities of a good researcher  

• Curiosity:  A curious mind and an ability to look at things from different perspectives is what makes a good researcher better. Good researchers are observant about the world around them and open to new ideas and possibilities; they are always asking questions and looking for answers. This ability to see the bigger picture while being curious about the smaller details is what makes a good researcher explore new ideas, test hypotheses, and make new discoveries.
• Critical thinking:  Successful researchers can think critically about the information they gather while reading about new developments in their own and related fields. This is an essential characteristic of a good researcher . Instead of simply accepting existing knowledge as fact, you need to have the ability to analyze and evaluate the validity and reliability of sources, consider alternative explanations for results you observe, and find connections between seemingly unrelated concepts.

• Creativity:  The qualities of a good researcher do not just include curiosity and critical thinking, but also thinking creatively when it comes to problem solving. Nurturing the ability to think outside the box and come up with novel and often unconventional solutions to challenges you face is how to become a better researcher. This allows you to come up with more ground-breaking research studies and results addressing issues that others might easily miss.
• Objectivity:  Nurturing preconceived notions is detrimental to research. Avoid temptations to make unconclusive statements or introduce personal biases into research, which will impact your research and standing in the long run. Remember, building essential qualities of a good researcher means consciously keeping aside personal preferences and biases and applying sound judgement to your work even when under pressure.
• Collaborative spirit:  An important characteristic of a good researcher is being able to work well with others. With a shift toward more collaborative research, successful researchers often connect with and work with peers to come up with innovative approaches to research problems. While sharing ideas and partnering with other researchers can lead to breakthroughs and boost your researcher reputation, it also opens the door for your work to reach and potentially benefit a wider audience.
• Communication skills:  An added strength of a good researcher is being able to communicate your findings clearly and effectively, which is a key contributor to your success. This is applicable when writing your manuscripts, presenting at conferences, as well as when seeking funding for your work. Good researchers can explain their research to both specialists and non-specialists to ensure their work is understood and appreciated by a wider audience.
• Attention to detail:  One of the key qualities of a good researcher is being meticulous in your work. Researchers need to pay attention to every detail, from the design of an experiment to the analysis of data, and further in writing and submitting their manuscript for publication. This crucial characteristic can help you ensure your research is accurate, testable, and reliable, and also gives your manuscripts a better chance of acceptance.
• Time management:  To understand what are the characteristics of a good researcher , first ask yourself if you manage your time well. Most successful researchers organize, prioritize, and optimize their time efficiently, allowing them to not only keep up with their responsibilities but also make time for personal tasks. If you’re being pulled in different directions or overwhelmed with trying to manage your research, stay updated on your research reading, or meeting your writing deadlines, consider honing this skill as a prerequisite to becoming a good researcher.
• Persistence & flexibility:  Research can be a long, difficult process with several hurdles and changes along the way. One of the key requirements to becoming a good researcher is being able to adapt to new technologies and changing circumstances and persevere despite setbacks and challenges that inevitably arise. Developing the qualities of a good researcher means anticipating problems, adjusting plans to tackle challenges head-on, and being patient while moving forward toward achieving your goals.
• Focus on self-care:  Anxiety, stress, and mental health issues are common among academics. Successful researchers are better equipped to manage this by adopting a healthy balanced lifestyle. Understanding what works for you can also improve your efficiency and productivity. Being aware of your strengths and weaknesses and using this to your advantage is key to becoming a good researcher.

In conclusion, perfecting the characteristics of a good researcher is not quick or easy, but by working consistently toward developing or strengthening these essential qualities, you will be well on your way to finding success as a well-established researcher.  

R Discovery is a literature search and research reading platform that accelerates your research discovery journey by keeping you updated on the latest, most relevant scholarly content. With 250M+ research articles sourced from trusted aggregators like CrossRef, Unpaywall, PubMed, PubMed Central, Open Alex and top publishing houses like Springer Nature, JAMA, IOP, Taylor & Francis, NEJM, BMJ, Karger, SAGE, Emerald Publishing and more, R Discovery puts a world of research at your fingertips.  

Try R Discovery Prime FREE for 1 week or upgrade at just US$72 a year to access premium features that let you listen to research on the go, read in your language, collaborate with peers, auto sync with reference managers, and much more. Choose a simpler, smarter way to find and read research – Download the app and start your free 7-day trial today !  

Related Posts

How to get a PhD in Computer Science? 

How to get a PhD in Accounting? 

What Constitutes a Good Research?

The Declining Art of Good Research

We seem to be compromising our commitment to good research in favor of publishable research, and there are a combination of trends that are accountable for this.

The first is the continued pressure of “publish or perish” for young academics seeking to move forward on the track for fewer and fewer tenured positions (or increasingly draconian renewable contracts).

Secondly, the open access model of research publication has created a booming population of academic journals with pages to fill and new researchers willing to pay article publication fees (APFs).

Thirdly, budget-strapped institutions have been aggressively targeting doctoral research candidates and the higher fees they bring to the table.

When these three trends are combined, the resulting onslaught of quantity over quality leads us to question what “good” research looks like anymore.

Is it the institution from which the research originated, or the debatable rank of the journal that published it?

Good Research as a Methodological Question

When looking to learn how to recognize what “good” research looks like, it makes sense to start at the beginning with the basic scope of the project:

• Does the research have a solid hypothesis?
• Is there evidence of a comprehensive literature review from reputable sources that clearly defines a target area for valuable research?
• Is the research team allocating sufficient time/resources to do the job properly, or were compromises made in order to accommodate the available funding?
• Is there evidence of a willingness to refine the hypothesis and research strategy if needed?
• Are the expectations of the implications of the research realistic?

Characteristics of a Good Research

For conducting a systematic research, it is important understand the characteristics of a good research.

• Its relevance to existing research conducted by other researchers.
• A good research is doable and replicable in future.
• It must be based on a logical rationale and tied to theory.
• It must generate new questions or hypotheses for incremental work in future.
• It must directly or indirectly address some real world problem.
• It must clearly state the variables of the experiment.
• It must conclude with valid and verifiable findings.

Good Research as an Ethical Question

The question as to whether or not the research is worth conducting at all could generate an extended and heated debate. Researchers are expected to publish, and research budgets are there to be spent.

We can hope that there was some degree of discussion and oversight before the research project was given the green light by a Principal Investigator or Research Supervisor, but those decisions are often made in a context of simple obligation rather than perceived need.

Consider the example of a less than proactive doctoral student with limited time and resources to complete a dissertation topic. A suggestion is made by the departmental Research Supervisor to pick a dissertation from a decade ago and simply repeat it. The suggestion meets the need for expediency and simplicity, but raises as many questions as it answers:

• What is the validity of the study – just because it can be repeated, should it?
• What was the contribution of the original study to the general body of knowledge? Will this additional data be an improvement?
• Given the lack of interest among academic journals in replicated studies, is the suggestion denying the student the opportunity to get published?
• Is directing a student to replication in the interests of expediency meeting a broader academic goal of graduating proficient researchers?

The Building Blocks of “Good” Research

There is no shortage of reputable, peer-reviewed journals that publish first-rate research material for new researchers to model.

That doesn’t mean you should copy the research topic or the methodology, but it wouldn’t hurt to examine the protocol in detail and make note of the specific decisions made and criteria put in place when that protocol was developed and implemented.

The challenge lies in sticking to those tried-and-true methodologies when your research data doesn’t prove to be as rich and fruitful as you had hoped.

Have you ever been stuck while in the middle of conducting a research? How did you cope with that? Let us know your approach while conducting a good research in the comments section below!

You can also visit our  Q&A forum  for frequently asked questions related to different aspects of research writing and publishing answered by our team that comprises subject-matter experts, eminent researchers, and publication experts.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Trending Now
• Upcoming Webinars
• Webinar Mobile App

Mastering Research Funding: A step-by-step guide to finding and winning grants

Identifying relevant funding opportunities Importance of eligibility criteria Understanding the funder’s perspective Crafting a strong…

• Career Corner

Academic Webinars: Transforming knowledge dissemination in the digital age

Digitization has transformed several areas of our lives, including the teaching and learning process. During…

• Manuscripts & Grants
• Reporting Research

Mastering Research Grant Writing in 2024: Navigating new policies and funder demands

Entering the world of grants and government funding can leave you confused; especially when trying…

How to Create a Poster That Stands Out: Tips for a smooth poster presentation

It was the conference season. Judy was excited to present her first poster! She had…

Academic Essay Writing Made Simple: 4 types and tips

The pen is mightier than the sword, they say, and nowhere is this more evident…

Recognizing the Signs: A guide to overcoming academic burnout

Intersectionality in Academia: Dealing with diverse perspectives

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

• Industry News
• Publishing Research
• AI in Academia
• Promoting Research
• Diversity and Inclusion
• Infographics
• Expert Video Library
• Other Resources
• Enago Learn
• Upcoming & On-Demand Webinars
• Peer Review Week 2024
• Open Access Week 2023
• Conference Videos
• Enago Report
• Journal Finder
• Enago Plagiarism & AI Grammar Check
• Editing Services
• Publication Support Services
• Research Impact
• Translation Services
• Publication solutions
• AI-Based Solutions
• Thought Leadership
• Call for Articles
• Call for Speakers
• Author Training
• Edit Profile

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

Which among these features would you prefer the most in a peer review assistant?

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Pharmacol Pharmacother
• v.4(2); Apr-Jun 2013

The critical steps for successful research: The research proposal and scientific writing: (A report on the pre-conference workshop held in conjunction with the 64 th annual conference of the Indian Pharmaceutical Congress-2012)

Pitchai balakumar.

Pharmacology Unit, Faculty of Pharmacy, AIMST University, Semeling, 08100 Bedong. Kedah Darul Aman, Malaysia

Mohammed Naseeruddin Inamdar

1 Department of Pharmacology, Al-Ameen College of Pharmacy, Bengaluru, Karnataka, India

Gowraganahalli Jagadeesh

2 Division of Cardiovascular and Renal Products, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, USA

An interactive workshop on ‘The Critical Steps for Successful Research: The Research Proposal and Scientific Writing’ was conducted in conjunction with the 64 th Annual Conference of the Indian Pharmaceutical Congress-2012 at Chennai, India. In essence, research is performed to enlighten our understanding of a contemporary issue relevant to the needs of society. To accomplish this, a researcher begins search for a novel topic based on purpose, creativity, critical thinking, and logic. This leads to the fundamental pieces of the research endeavor: Question, objective, hypothesis, experimental tools to test the hypothesis, methodology, and data analysis. When correctly performed, research should produce new knowledge. The four cornerstones of good research are the well-formulated protocol or proposal that is well executed, analyzed, discussed and concluded. This recent workshop educated researchers in the critical steps involved in the development of a scientific idea to its successful execution and eventual publication.

INTRODUCTION

Creativity and critical thinking are of particular importance in scientific research. Basically, research is original investigation undertaken to gain knowledge and understand concepts in major subject areas of specialization, and includes the generation of ideas and information leading to new or substantially improved scientific insights with relevance to the needs of society. Hence, the primary objective of research is to produce new knowledge. Research is both theoretical and empirical. It is theoretical because the starting point of scientific research is the conceptualization of a research topic and development of a research question and hypothesis. Research is empirical (practical) because all of the planned studies involve a series of observations, measurements, and analyses of data that are all based on proper experimental design.[ 1 – 9 ]

The subject of this report is to inform readers of the proceedings from a recent workshop organized by the 64 th Annual conference of the ‘ Indian Pharmaceutical Congress ’ at SRM University, Chennai, India, from 05 to 06 December 2012. The objectives of the workshop titled ‘The Critical Steps for Successful Research: The Research Proposal and Scientific Writing,’ were to assist participants in developing a strong fundamental understanding of how best to develop a research or study protocol, and communicate those research findings in a conference setting or scientific journal. Completing any research project requires meticulous planning, experimental design and execution, and compilation and publication of findings in the form of a research paper. All of these are often unfamiliar to naïve researchers; thus, the purpose of this workshop was to teach participants to master the critical steps involved in the development of an idea to its execution and eventual publication of the results (See the last section for a list of learning objectives).

THE STRUCTURE OF THE WORKSHOP

The two-day workshop was formatted to include key lectures and interactive breakout sessions that focused on protocol development in six subject areas of the pharmaceutical sciences. This was followed by sessions on scientific writing. DAY 1 taught the basic concepts of scientific research, including: (1) how to formulate a topic for research and to describe the what, why , and how of the protocol, (2) biomedical literature search and review, (3) study designs, statistical concepts, and result analyses, and (4) publication ethics. DAY 2 educated the attendees on the basic elements and logistics of writing a scientific paper and thesis, and preparation of poster as well as oral presentations.

The final phase of the workshop was the ‘Panel Discussion,’ including ‘Feedback/Comments’ by participants. There were thirteen distinguished speakers from India and abroad. Approximately 120 post-graduate and pre-doctoral students, young faculty members, and scientists representing industries attended the workshop from different parts of the country. All participants received a printed copy of the workshop manual and supporting materials on statistical analyses of data.

THE BASIC CONCEPTS OF RESEARCH: THE KEY TO GETTING STARTED IN RESEARCH

A research project generally comprises four key components: (1) writing a protocol, (2) performing experiments, (3) tabulating and analyzing data, and (4) writing a thesis or manuscript for publication.

Fundamentals in the research process

A protocol, whether experimental or clinical, serves as a navigator that evolves from a basic outline of the study plan to become a qualified research or grant proposal. It provides the structural support for the research. Dr. G. Jagadeesh (US FDA), the first speaker of the session, spoke on ‘ Fundamentals in research process and cornerstones of a research project .’ He discussed at length the developmental and structural processes in preparing a research protocol. A systematic and step-by-step approach is necessary in planning a study. Without a well-designed protocol, there would be a little chance for successful completion of a research project or an experiment.

Research topic

The first and the foremost difficult task in research is to identify a topic for investigation. The research topic is the keystone of the entire scientific enterprise. It begins the project, drives the entire study, and is crucial for moving the project forward. It dictates the remaining elements of the study [ Table 1 ] and thus, it should not be too narrow or too broad or unfocused. Because of these potential pitfalls, it is essential that a good or novel scientific idea be based on a sound concept. Creativity, critical thinking, and logic are required to generate new concepts and ideas in solving a research problem. Creativity involves critical thinking and is associated with generating many ideas. Critical thinking is analytical, judgmental, and involves evaluating choices before making a decision.[ 4 ] Thus, critical thinking is convergent type thinking that narrows and refines those divergent ideas and finally settles to one idea for an in-depth study. The idea on which a research project is built should be novel, appropriate to achieve within the existing conditions, and useful to the society at large. Therefore, creativity and critical thinking assist biomedical scientists in research that results in funding support, novel discovery, and publication.[ 1 , 4 ]

Elements of a study protocol

Research question

The next most crucial aspect of a study protocol is identifying a research question. It should be a thought-provoking question. The question sets the framework. It emerges from the title, findings/results, and problems observed in previous studies. Thus, mastering the literature, attendance at conferences, and discussion in journal clubs/seminars are sources for developing research questions. Consider the following example in developing related research questions from the research topic.

Hepatoprotective activity of Terminalia arjuna and Apium graveolens on paracetamol-induced liver damage in albino rats.

How is paracetamol metabolized in the body? Does it involve P450 enzymes? How does paracetamol cause liver injury? What are the mechanisms by which drugs can alleviate liver damage? What biochemical parameters are indicative of liver injury? What major endogenous inflammatory molecules are involved in paracetamol-induced liver damage?

A research question is broken down into more precise objectives. The objectives lead to more precise methods and definition of key terms. The objectives should be SMART-Specific, Measurable, Achievable, Realistic, Time-framed,[ 10 ] and should cover the entire breadth of the project. The objectives are sometimes organized into hierarchies: Primary, secondary, and exploratory; or simply general and specific. Study the following example:

To evaluate the safety and tolerability of single oral doses of compound X in normal volunteers.

To assess the pharmacokinetic profile of compound X following single oral doses.

To evaluate the incidence of peripheral edema reported as an adverse event.

The objectives and research questions are then formulated into a workable or testable hypothesis. The latter forces us to think carefully about what comparisons will be needed to answer the research question, and establishes the format for applying statistical tests to interpret the results. The hypothesis should link a process to an existing or postulated biologic pathway. A hypothesis is written in a form that can yield measurable results. Studies that utilize statistics to compare groups of data should have a hypothesis. Consider the following example:

• The hepatoprotective activity of Terminalia arjuna is superior to that of Apium graveolens against paracetamol-induced liver damage in albino rats.

All biological research, including discovery science, is hypothesis-driven. However, not all studies need be conducted with a hypothesis. For example, descriptive studies (e.g., describing characteristics of a plant, or a chemical compound) do not need a hypothesis.[ 1 ]

Relevance of the study

Another important section to be included in the protocol is ‘significance of the study.’ Its purpose is to justify the need for the research that is being proposed (e.g., development of a vaccine for a disease). In summary, the proposed study should demonstrate that it represents an advancement in understanding and that the eventual results will be meaningful, contribute to the field, and possibly even impact society.

Biomedical literature

A literature search may be defined as the process of examining published sources of information on a research or review topic, thesis, grant application, chemical, drug, disease, or clinical trial, etc. The quantity of information available in print or electronically (e.g., the internet) is immense and growing with time. A researcher should be familiar with the right kinds of databases and search engines to extract the needed information.[ 3 , 6 ]

Dr. P. Balakumar (Institute of Pharmacy, Rajendra Institute of Technology and Sciences, Sirsa, Haryana; currently, Faculty of Pharmacy, AIMST University, Malaysia) spoke on ‘ Biomedical literature: Searching, reviewing and referencing .’ He schematically explained the basis of scientific literature, designing a literature review, and searching literature. After an introduction to the genesis and diverse sources of scientific literature searches, the use of PubMed, one of the premier databases used for biomedical literature searches world-wide, was illustrated with examples and screenshots. Several companion databases and search engines are also used for finding information related to health sciences, and they include Embase, Web of Science, SciFinder, The Cochrane Library, International Pharmaceutical Abstracts, Scopus, and Google Scholar.[ 3 ] Literature searches using alternative interfaces for PubMed such as GoPubMed, Quertle, PubFocus, Pubget, and BibliMed were discussed. The participants were additionally informed of databases on chemistry, drugs and drug targets, clinical trials, toxicology, and laboratory animals (reviewed in ref[ 3 ]).

Referencing and bibliography are essential in scientific writing and publication.[ 7 ] Referencing systems are broadly classified into two major types, such as Parenthetical and Notation systems. Parenthetical referencing is also known as Harvard style of referencing, while Vancouver referencing style and ‘Footnote’ or ‘Endnote’ are placed under Notation referencing systems. The participants were educated on each referencing system with examples.

Bibliography management

Dr. Raj Rajasekaran (University of California at San Diego, CA, USA) enlightened the audience on ‘ bibliography management ’ using reference management software programs such as Reference Manager ® , Endnote ® , and Zotero ® for creating and formatting bibliographies while writing a manuscript for publication. The discussion focused on the use of bibliography management software in avoiding common mistakes such as incomplete references. Important steps in bibliography management, such as creating reference libraries/databases, searching for references using PubMed/Google scholar, selecting and transferring selected references into a library, inserting citations into a research article and formatting bibliographies, were presented. A demonstration of Zotero®, a freely available reference management program, included the salient features of the software, adding references from PubMed using PubMed ID, inserting citations and formatting using different styles.

Writing experimental protocols

The workshop systematically instructed the participants in writing ‘ experimental protocols ’ in six disciplines of Pharmaceutical Sciences.: (1) Pharmaceutical Chemistry (presented by Dr. P. V. Bharatam, NIPER, Mohali, Punjab); (2) Pharmacology (presented by Dr. G. Jagadeesh and Dr. P. Balakumar); (3) Pharmaceutics (presented by Dr. Jayant Khandare, Piramal Life Sciences, Mumbai); (4) Pharmacy Practice (presented by Dr. Shobha Hiremath, Al-Ameen College of Pharmacy, Bengaluru); (5) Pharmacognosy and Phytochemistry (presented by Dr. Salma Khanam, Al-Ameen College of Pharmacy, Bengaluru); and (6) Pharmaceutical Analysis (presented by Dr. Saranjit Singh, NIPER, Mohali, Punjab). The purpose of the research plan is to describe the what (Specific Aims/Objectives), why (Background and Significance), and how (Design and Methods) of the proposal.

The research plan should answer the following questions: (a) what do you intend to do; (b) what has already been done in general, and what have other researchers done in the field; (c) why is this worth doing; (d) how is it innovative; (e) what will this new work add to existing knowledge; and (f) how will the research be accomplished?

In general, the format used by the faculty in all subjects is shown in Table 2 .

Elements of a research protocol

Biostatistics

Biostatistics is a key component of biomedical research. Highly reputed journals like The Lancet, BMJ, Journal of the American Medical Association, and many other biomedical journals include biostatisticians on their editorial board or reviewers list. This indicates that a great importance is given for learning and correctly employing appropriate statistical methods in biomedical research. The post-lunch session on day 1 of the workshop was largely committed to discussion on ‘ Basic biostatistics .’ Dr. R. Raveendran (JIPMER, Puducherry) and Dr. Avijit Hazra (PGIMER, Kolkata) reviewed, in parallel sessions, descriptive statistics, probability concepts, sample size calculation, choosing a statistical test, confidence intervals, hypothesis testing and ‘ P ’ values, parametric and non-parametric statistical tests, including analysis of variance (ANOVA), t tests, Chi-square test, type I and type II errors, correlation and regression, and summary statistics. This was followed by a practice and demonstration session. Statistics CD, compiled by Dr. Raveendran, was distributed to the participants before the session began and was demonstrated live. Both speakers worked on a variety of problems that involved both clinical and experimental data. They discussed through examples the experimental designs encountered in a variety of studies and statistical analyses performed for different types of data. For the benefit of readers, we have summarized statistical tests applied frequently for different experimental designs and post-hoc tests [ Figure 1 ].

Conceptual framework for statistical analyses of data. Of the two kinds of variables, qualitative (categorical) and quantitative (numerical), qualitative variables (nominal or ordinal) are not normally distributed. Numerical data that come from normal distributions are analyzed using parametric tests, if not; the data are analyzed using non-parametric tests. The most popularly used Student's t -test compares the means of two populations, data for this test could be paired or unpaired. One-way analysis of variance (ANOVA) is used to compare the means of three or more independent populations that are normally distributed. Applying t test repeatedly in pair (multiple comparison), to compare the means of more than two populations, will increase the probability of type I error (false positive). In this case, for proper interpretation, we need to adjust the P values. Repeated measures ANOVA is used to compare the population means if more than two observations coming from same subject over time. The null hypothesis is rejected with a ‘ P ’ value of less than 0.05, and the difference in population means is considered to be statistically significant. Subsequently, appropriate post-hoc tests are used for pairwise comparisons of population means. Two-way or three-way ANOVA are considered if two (diet, dose) or three (diet, dose, strain) independent factors, respectively, are analyzed in an experiment (not described in the Figure). Categorical nominal unmatched variables (counts or frequencies) are analyzed by Chi-square test (not shown in the Figure)

Research and publication ethics

The legitimate pursuit of scientific creativity is unfortunately being marred by a simultaneous increase in scientific misconduct. A disproportionate share of allegations involves scientists of many countries, and even from respected laboratories. Misconduct destroys faith in science and scientists and creates a hierarchy of fraudsters. Investigating misconduct also steals valuable time and resources. In spite of these facts, most researchers are not aware of publication ethics.

Day 1 of the workshop ended with a presentation on ‘ research and publication ethics ’ by Dr. M. K. Unnikrishnan (College of Pharmaceutical Sciences, Manipal University, Manipal). He spoke on the essentials of publication ethics that included plagiarism (attempting to take credit of the work of others), self-plagiarism (multiple publications by an author on the same content of work with slightly different wordings), falsification (manipulation of research data and processes and omitting critical data or results), gift authorship (guest authorship), ghostwriting (someone other than the named author (s) makes a major contribution), salami publishing (publishing many papers, with minor differences, from the same study), and sabotage (distracting the research works of others to halt their research completion). Additionally, Dr. Unnikrishnan pointed out the ‘ Ingelfinger rule ’ of stipulating that a scientist must not submit the same original research in two different journals. He also advised the audience that authorship is not just credit for the work but also responsibility for scientific contents of a paper. Although some Indian Universities are instituting preventive measures (e.g., use of plagiarism detecting software, Shodhganga digital archiving of doctoral theses), Dr. Unnikrishnan argued for a great need to sensitize young researchers on the nature and implications of scientific misconduct. Finally, he discussed methods on how editors and peer reviewers should ethically conduct themselves while managing a manuscript for publication.

SCIENTIFIC COMMUNICATION: THE KEY TO SUCCESSFUL SELLING OF FINDINGS

Research outcomes are measured through quality publications. Scientists must not only ‘do’ science but must ‘write’ science. The story of the project must be told in a clear, simple language weaving in previous work done in the field, answering the research question, and addressing the hypothesis set forth at the beginning of the study. Scientific publication is an organic process of planning, researching, drafting, revising, and updating the current knowledge for future perspectives. Writing a research paper is no easier than the research itself. The lectures of Day 2 of the workshop dealt with the basic elements and logistics of writing a scientific paper.

An overview of paper structure and thesis writing

Dr. Amitabh Prakash (Adis, Auckland, New Zealand) spoke on ‘ Learning how to write a good scientific paper .’ His presentation described the essential components of an original research paper and thesis (e.g., introduction, methods, results, and discussion [IMRaD]) and provided guidance on the correct order, in which data should appear within these sections. The characteristics of a good abstract and title and the creation of appropriate key words were discussed. Dr. Prakash suggested that the ‘title of a paper’ might perhaps have a chance to make a good impression, and the title might be either indicative (title that gives the purpose of the study) or declarative (title that gives the study conclusion). He also suggested that an abstract is a succinct summary of a research paper, and it should be specific, clear, and concise, and should have IMRaD structure in brief, followed by key words. Selection of appropriate papers to be cited in the reference list was also discussed. Various unethical authorships were enumerated, and ‘The International Committee of Medical Journal Editors (ICMJE) criteria for authorship’ was explained ( http://www.icmje.org/ethical_1author.html ; also see Table 1 in reference #9). The session highlighted the need for transparency in medical publication and provided a clear description of items that needed to be included in the ‘Disclosures’ section (e.g., sources of funding for the study and potential conflicts of interest of all authors, etc.) and ‘Acknowledgements’ section (e.g., writing assistance and input from all individuals who did not meet the authorship criteria). The final part of the presentation was devoted to thesis writing, and Dr. Prakash provided the audience with a list of common mistakes that are frequently encountered when writing a manuscript.

The backbone of a study is description of results through Text, Tables, and Figures. Dr. S. B. Deshpande (Institute of Medical Sciences, Banaras Hindu University, Varanasi, India) spoke on ‘ Effective Presentation of Results .’ The Results section deals with the observations made by the authors and thus, is not hypothetical. This section is subdivided into three segments, that is, descriptive form of the Text, providing numerical data in Tables, and visualizing the observations in Graphs or Figures. All these are arranged in a sequential order to address the question hypothesized in the Introduction. The description in Text provides clear content of the findings highlighting the observations. It should not be the repetition of facts in tables or graphs. Tables are used to summarize or emphasize descriptive content in the text or to present the numerical data that are unrelated. Illustrations should be used when the evidence bearing on the conclusions of a paper cannot be adequately presented in a written description or in a Table. Tables or Figures should relate to each other logically in sequence and should be clear by themselves. Furthermore, the discussion is based entirely on these observations. Additionally, how the results are applied to further research in the field to advance our understanding of research questions was discussed.

Dr. Peush Sahni (All-India Institute of Medical Sciences, New Delhi) spoke on effectively ‘ structuring the Discussion ’ for a research paper. The Discussion section deals with a systematic interpretation of study results within the available knowledge. He said the section should begin with the most important point relating to the subject studied, focusing on key issues, providing link sentences between paragraphs, and ensuring the flow of text. Points were made to avoid history, not repeat all the results, and provide limitations of the study. The strengths and novel findings of the study should be provided in the discussion, and it should open avenues for future research and new questions. The Discussion section should end with a conclusion stating the summary of key findings. Dr. Sahni gave an example from a published paper for writing a Discussion. In another presentation titled ‘ Writing an effective title and the abstract ,’ Dr. Sahni described the important components of a good title, such as, it should be simple, concise, informative, interesting and eye-catching, accurate and specific about the paper's content, and should state the subject in full indicating study design and animal species. Dr. Sahni explained structured (IMRaD) and unstructured abstracts and discussed a few selected examples with the audience.

Language and style in publication

The next lecture of Dr. Amitabh Prakash on ‘ Language and style in scientific writing: Importance of terseness, shortness and clarity in writing ’ focused on the actual sentence construction, language, grammar and punctuation in scientific manuscripts. His presentation emphasized the importance of brevity and clarity in the writing of manuscripts describing biomedical research. Starting with a guide to the appropriate construction of sentences and paragraphs, attendees were given a brief overview of the correct use of punctuation with interactive examples. Dr. Prakash discussed common errors in grammar and proactively sought audience participation in correcting some examples. Additional discussion was centered on discouraging the use of redundant and expendable words, jargon, and the use of adjectives with incomparable words. The session ended with a discussion of words and phrases that are commonly misused (e.g., data vs . datum, affect vs . effect, among vs . between, dose vs . dosage, and efficacy/efficacious vs . effective/effectiveness) in biomedical research manuscripts.

Working with journals

The appropriateness in selecting the journal for submission and acceptance of the manuscript should be determined by the experience of an author. The corresponding author must have a rationale in choosing the appropriate journal, and this depends upon the scope of the study and the quality of work performed. Dr. Amitabh Prakash spoke on ‘ Working with journals: Selecting a journal, cover letter, peer review process and impact factor ’ by instructing the audience in assessing the true value of a journal, understanding principles involved in the peer review processes, providing tips on making an initial approach to the editorial office, and drafting an appropriate cover letter to accompany the submission. His presentation defined the metrics that are most commonly used to measure journal quality (e.g., impact factor™, Eigenfactor™ score, Article Influence™ score, SCOPUS 2-year citation data, SCImago Journal Rank, h-Index, etc.) and guided attendees on the relative advantages and disadvantages of using each metric. Factors to consider when assessing journal quality were discussed, and the audience was educated on the ‘green’ and ‘gold’ open access publication models. Various peer review models (e.g., double-blind, single-blind, non-blind) were described together with the role of the journal editor in assessing manuscripts and selecting suitable reviewers. A typical checklist sent to referees was shared with the attendees, and clear guidance was provided on the best way to address referee feedback. The session concluded with a discussion of the potential drawbacks of the current peer review system.

Poster and oral presentations at conferences

Posters have become an increasingly popular mode of presentation at conferences, as it can accommodate more papers per meeting, has no time constraint, provides a better presenter-audience interaction, and allows one to select and attend papers of interest. In Figure 2 , we provide instructions, design, and layout in preparing a scientific poster. In the final presentation, Dr. Sahni provided the audience with step-by-step instructions on how to write and format posters for layout, content, font size, color, and graphics. Attendees were given specific guidance on the format of text on slides, the use of color, font type and size, and the use of illustrations and multimedia effects. Moreover, the importance of practical tips while delivering oral or poster presentation was provided to the audience, such as speak slowly and clearly, be informative, maintain eye contact, and listen to the questions from judges/audience carefully before coming up with an answer.

Guidelines and design to scientific poster presentation. The objective of scientific posters is to present laboratory work in scientific meetings. A poster is an excellent means of communicating scientific work, because it is a graphic representation of data. Posters should have focus points, and the intended message should be clearly conveyed through simple sections: Text, Tables, and Graphs. Posters should be clear, succinct, striking, and eye-catching. Colors should be used only where necessary. Use one font (Arial or Times New Roman) throughout. Fancy fonts should be avoided. All headings should have font size of 44, and be in bold capital letters. Size of Title may be a bit larger; subheading: Font size of 36, bold and caps. References and Acknowledgments, if any, should have font size of 24. Text should have font size between 24 and 30, in order to be legible from a distance of 3 to 6 feet. Do not use lengthy notes

PANEL DISCUSSION: FEEDBACK AND COMMENTS BY PARTICIPANTS

After all the presentations were made, Dr. Jagadeesh began a panel discussion that included all speakers. The discussion was aimed at what we do currently and could do in the future with respect to ‘developing a research question and then writing an effective thesis proposal/protocol followed by publication.’ Dr. Jagadeesh asked the following questions to the panelists, while receiving questions/suggestions from the participants and panelists.

• Does a Post-Graduate or Ph.D. student receive adequate training, either through an institutional course, a workshop of the present nature, or from the guide?
• Are these Post-Graduates self-taught (like most of us who learnt the hard way)?
• How are these guides trained? How do we train them to become more efficient mentors?
• Does a Post-Graduate or Ph.D. student struggle to find a method (s) to carry out studies? To what extent do seniors/guides help a post graduate overcome technical difficulties? How difficult is it for a student to find chemicals, reagents, instruments, and technical help in conducting studies?
• Analyses of data and interpretation: Most students struggle without adequate guidance.
• Thesis and publications frequently feature inadequate/incorrect statistical analyses and representation of data in tables/graphs. The student, their guide, and the reviewers all share equal responsibility.
• Who initiates and drafts the research paper? The Post-Graduate or their guide?
• What kind of assistance does a Post-Graduate get from the guide in finalizing a paper for publication?
• Does the guide insist that each Post-Graduate thesis yield at least one paper, and each Ph.D. thesis more than two papers, plus a review article?

The panelists and audience expressed a variety of views, but were unable to arrive at a decisive conclusion.

WHAT HAVE THE PARTICIPANTS LEARNED?

At the end of this fast-moving two-day workshop, the participants had opportunities in learning the following topics:

• Sequential steps in developing a study protocol, from choosing a research topic to developing research questions and a hypothesis.
• Study protocols on different topics in their subject of specialization
• Searching and reviewing the literature
• Appropriate statistical analyses in biomedical research
• Scientific ethics in publication
• Writing and understanding the components of a research paper (IMRaD)
• Recognizing the value of good title, running title, abstract, key words, etc
• Importance of Tables and Figures in the Results section, and their importance in describing findings
• Evidence-based Discussion in a research paper
• Language and style in writing a paper and expert tips on getting it published
• Presentation of research findings at a conference (oral and poster).

Overall, the workshop was deemed very helpful to participants. The participants rated the quality of workshop from “ satisfied ” to “ very satisfied .” A significant number of participants were of the opinion that the time allotted for each presentation was short and thus, be extended from the present two days to four days with adequate time to ask questions. In addition, a ‘hands-on’ session should be introduced for writing a proposal and manuscript. A large number of attendees expressed their desire to attend a similar workshop, if conducted, in the near future.

ACKNOWLEDGMENT

We gratefully express our gratitude to the Organizing Committee, especially Professors K. Chinnasamy, B. G. Shivananda, N. Udupa, Jerad Suresh, Padma Parekh, A. P. Basavarajappa, Mr. S. V. Veerramani, Mr. J. Jayaseelan, and all volunteers of the SRM University. We thank Dr. Thomas Papoian (US FDA) for helpful comments on the manuscript.

The opinions expressed herein are those of Gowraganahalli Jagadeesh and do not necessarily reflect those of the US Food and Drug Administration

Source of Support: Nil

Conflict of Interest: None declared.

What is quality research? A guide to identifying the key features and achieving success

Every researcher worth their salt strives for quality. But in research, what does quality mean?

Simply put, quality research is thorough, accurate, original and relevant. And to achieve this, you need to follow specific standards. You need to make sure your findings are reliable and valid. And when you know they're quality assured, you can share them with absolute confidence.

You’ll be able to draw accurate conclusions from your investigations and contribute to the wider body of knowledge in your field.

Importance of quality research

Quality research helps us better understand complex problems. It enables us to make decisions based on facts and evidence. And it empowers us to solve real-world issues. Without quality research, we can't advance knowledge or identify trends and patterns. We also can’t develop new theories and approaches to solving problems.

With rigorous and transparent research methods, you’ll produce reliable findings that other researchers can replicate. This leads to the development of new theories and interventions. On the other hand, low-quality research can hinder progress by producing unreliable findings that can’t be replicated, wasting resources and impeding advancements in the field.

In all cases, quality control is critical. It ensures that decisions are based on evidence rather than gut feeling or bias.

Standards for quality research

Over the years, researchers, scientists and authors have come to a consensus about the standards used to check the quality of research. Determined through empirical observation, theoretical underpinnings and philosophy of science, these include:

1. Having a well-defined research topic and a clear hypothesis

This is essential to verify that the research is focused and the results are relevant and meaningful. The research topic should be well-scoped and the hypothesis should be clearly stated and falsifiable .

For example, in a quantitative study about the effects of social media on behavior, a well-defined research topic could be, "Does the use of TikTok reduce attention span in American adolescents?"

This is good because:

• The research topic focuses on a particular platform of social media ( TikTok ). And it also focuses on a specific group of people (American adolescents).
• The research question is clear and straightforward, making it easier to design the study and collect relevant data.
• You can test the hypothesis and a research team can evaluate it easily. This can be done through the use of various research methods, such as survey research, experiments or observational studies.
• The hypothesis is focused on a specific outcome (the attention span). Then, this can be measured and compared to control groups or previous research studies.

2. Ensuring transparency

Transparency is crucial when conducting research. You need to be upfront about the methods you used, such as:

• Describing how you recruited the participants.
• How you communicated with them.
• How they were incentivized.

You also need to explain how you analyzed the data, so other researchers can replicate your results if necessary. re-registering your study is a great way to be as transparent in your research as possible. This  involves publicly documenting your study design, methods and analysis plan before conducting the research. This reduces the risk of selective reporting and increases the credibility of your findings.

3. Using appropriate research methods

Depending on the topic, some research methods are better suited than others for collecting data. To use our TikTok example, a quantitative research approach, such as a behavioral test that measures the participants' ability to focus on tasks, might be the most appropriate.

On the other hand, for topics that require a more in-depth understanding of individuals' experiences or perspectives, a qualitative research approach, such as interviews or focus groups, might be more suitable. These methods can provide rich and detailed information that you can’t capture through quantitative data alone.

4. Assessing limitations and the possible impact of systematic bias

When you present your research, it’s important to consider how the limitations of your study could affect the result. This could be systematic bias in the sampling procedure or data analysis, for instance. Let’s say you only study a small sample of participants from one school district. This would limit the generalizability and content validity of your findings.

5. Conducting accurate reporting

This is an essential aspect of any research project. You need to be able to clearly communicate the findings and implications of your study . Also, provide citations for any claims made in your report. When you present your work, it’s vital that you describe the variables involved in your study accurately and how you measured them.

Curious to learn more? Read our Data Quality eBook .

How to identify credible research findings

To determine whether a published study is trustworthy, consider the following:

• Peer review: If a study has been peer-reviewed by recognized experts, rest assured that it’s a reliable source of information. Peer review means that other scholars have read and verified the study before publication.
• Researcher's qualifications: If they're an expert in the field, that’s a good sign that you can trust their findings. However, if they aren't, it doesn’t necessarily mean that the study's information is unreliable. It simply means that you should be extra cautious about accepting its conclusions as fact.
• Study design: The design of a study can make or break its reliability. Consider factors like sample size and methodology.
• Funding source: Studies funded by organizations with a vested interest in a particular outcome may be less credible than those funded by independent sources.
• Statistical significance: You've heard the phrase "numbers don't lie," right? That's what statistical significance is all about. It refers to the likelihood that the results of a study occurred by chance. Results that are statistically significant are more credible.

Achieve quality research with Prolific

Want to ensure your research is high-quality? Prolific can help.

Our platform gives you access to a carefully vetted pool of participants. We make sure they're attentive, honest, and ready to provide rich and detailed answers where needed. This helps to ensure that the data you collect through Prolific is of the highest quality.

Streamline your research process and feel confident in the results you receive. Our minimum pay threshold and commitment to fair compensation motivate participants to provide valuable responses and give their best effort. This ensures the quality of your research and helps you get the results you need. 

You might also like

High-quality human data to deliver world-leading research and AIs.

Follow us on

All Rights Reserved Prolific 2024

What is Research Design? Characteristics, Types, Process, & Examples

Link Copied

Share on Facebook

Share on Twitter

Share on LinkedIn

Your search has come to an end!

Ever felt like a hamster on a research wheel fast, spinning with a million questions but going nowhere? You've got your topic; you're brimming with curiosity, but... what next? So, forget the research rut and get your papers! This ultimate guide to "what is research design?" will have you navigating your project like a pro, uncovering answers and avoiding dead ends. Know the features of good research design, what you mean by research design, elements of research design, and more.

What is Research Design?

Before starting with the topic, do you know what is research design? Research design is the structure of research methods and techniques selected to conduct a study. It refines the methods suited to the subject and ensures a successful setup. Defining a research topic clarifies the type of research (experimental, survey research, correlational, semi-experimental, review) and its sub-type (experimental design, research problem, descriptive case-study).

There are three main types of designs for research:

1. Data Collection

2. Measurement

3. Data Analysis

Elements of Research Design 

Now that you know what is research design, it is important to know the elements and components of research design. Impactful research minimises bias and enhances data accuracy. Designs with minimal error margins are ideal. Key elements include:

1. Accurate purpose statement

2. Techniques for data collection and analysis

3. Methods for data analysis

4. Type of research methodology

5. Probable objections to research

6. Research settings

7. Timeline

8. Measurement of analysis

Got a hang of research, now book yout student accommodation with one click!

Book through amber today!

Characteristics of Research Design

Research design has several key characteristics that contribute to the validity, reliability, and overall success of a research study. To know the answer for what is research design, it is important to know the characteristics. These are-

1. Reliability

A reliable research design ensures that each study’s results are accurate and can be replicated. This means that if the research is conducted again under the same conditions, it should yield similar results.

2. Validity

A valid research design uses appropriate measuring tools to gauge the results according to the research objective. This ensures that the data collected and the conclusions drawn are relevant and accurately reflect the phenomenon being studied.

3. Neutrality

A neutral research design ensures that the assumptions made at the beginning of the research are free from bias. This means that the data collected throughout the research is based on these unbiased assumptions.

4. Generalizability

A good research design draws an outcome that can be applied to a large set of people and is not limited to the sample size or the research group.

Research Design Process

What is research design? A good research helps you do a really good study that gives fair, trustworthy, and useful results. But it's also good to have a bit of wiggle room for changes. If you’re wondering how to conduct a research in just 5 mins , here's a breakdown and examples to work even better.

1. Consider Aims and Approaches

Define the research questions and objectives, and establish the theoretical framework and methodology.

2. Choose a Type of Research Design

Select the suitable research design, such as experimental, correlational, survey, case study, or ethnographic, according to the research questions and objectives.

3. Identify Population and Sampling Method

Determine the target population and sample size, and select the sampling method, like random, stratified random sampling, or convenience sampling.

4. Choose Data Collection Methods

Decide on the data collection methods, such as surveys, interviews, observations, or experiments, and choose the appropriate instruments for data collection.

5. Plan Data Collection Procedures

Create a plan for data collection, detailing the timeframe, location, and personnel involved, while ensuring ethical considerations are met.

6. Decide on Data Analysis Strategies

Select the appropriate data analysis techniques, like statistical analysis, content analysis, or discourse analysis, and plan the interpretation of the results.

What are the Types of Research Design?

A researcher must grasp various types to decide which model to use for a study. There are different research designs that can be broadly classified into quantitative and qualitative.

Qualitative Research

Qualitative research identifies relationships between collected data and observations through mathematical calculations. Statistical methods validate or refute theories about natural phenomena. This research method answers "why" a theory exists and explores respondents' perspectives.

Quantitative Research

Quantitative research is essential when statistical conclusions are needed to gather actionable insights. Numbers provide clarity for critical business decisions. This method is crucial for organizational growth, with insights from complex numerical data guiding future business decisions.

Qualitative Research vs Quantitative Research

While researching, it is important to know the difference between qualitative and quantitative research. Here's a quick difference between the two:

The research types can be further divided into 5 categories:

1. Descriptive Research

Descriptive research design focuses on detailing a situation or case. It's a theory-driven method that involves gathering, analysing, and presenting data. This approach offers insights into the reasons and mechanisms behind a research subject, enhancing understanding of the research's importance. When the problem statement is unclear, exploratory research can be conducted.

2. Experimental Research

Experimental research design investigates cause-and-effect relationships. It’s a causal design where the impact of an independent variable on a dependent variable is observed. For example, the effect of price on customer satisfaction. This method efficiently addresses problems by manipulating independent variables to see their effect on dependent variables. Often used in social sciences, it involves analysing human behaviour by studying changes in one group's actions and their impact on another group.

3. Correlational Research

Correlational research design is a non-experimental technique that identifies relationships between closely linked variables. It uses statistical analysis to determine these relationships without assumptions. This method requires two different groups. A correlation coefficient between -1 and +1 indicates the strength and direction of the relationship, with +1 showing a positive correlation and -1 a negative correlation.

4. Diagnostic Research

Diagnostic research design aims to identify the underlying causes of specific issues. This method delves into factors creating problematic situations and has three phases: 

• Issue inception
• Issue diagnosis
• Issue resolution

5. Explanatory Research

Explanatory research design builds on a researcher’s ideas to explore theories further. It seeks to explain the unexplored aspects of a subject, addressing the what, how, and why of research questions.

Benefits of Research Design

After learning about what is research design and the process, it is important to know the key benefits of a well-structured research design:

1. Minimises Risk of Errors: A good research design minimises the risk of errors and reduces inaccuracy. It ensures that the study is carried out in the right direction and that all the team members are on the same page.

2. Efficient Use of Resources: It facilitates a concrete research plan for the efficient use of time and resources. It helps the researcher better complete all the tasks, even with limited resources.

3. Provides Direction: The purpose of the research design is to enable the researcher to proceed in the right direction without deviating from the tasks. It helps to identify the major and minor tasks of the study.

4. Ensures Validity and Reliability: A well-designed research enhances the validity and reliability of the findings and allows for the replication of studies by other researchers. The main advantage of a good research design is that it provides accuracy, reliability, consistency, and legitimacy to the research.

5. Facilitates Problem-Solving: A researcher can easily frame the objectives of the research work based on the design of experiments (research design). A good research design helps the researcher find the best solution for the research problems.

6. Better Documentation: It helps in better documentation of the various activities while the project work is going on.

That's it! You've explored all the answers for what is research design in research? Remember, it's not just about picking a fancy method – it's about choosing the perfect tool to answer your burning questions. By carefully considering your goals and resources, you can design a research plan that gathers reliable information and helps you reach clear conclusions. 

Frequently Asked Questions

What are the key components of a research design, how can i choose the best research design for my study, what are some common pitfalls in research design, and how can they be avoided, how does research design impact the validity and reliability of a study, what ethical considerations should be taken into account in research design.

Your ideal student home & a flight ticket awaits

Follow us on :

Related Posts

A Guide to Make Your Job Application Stand Out

10 Tips On How To Get A First Class Degree

17 Reasons to Study Abroad

amber © 2024. All rights reserved.

4.8/5 on Trustpilot

Rated as "Excellent" • 4800+ Reviews by students

Rated as "Excellent" • 4800+ Reviews by Students

Qualities of a Good Researcher: Top 11 Traits

• Post author: Edeh Samuel Chukwuemeka ACMC
• Post published: June 27, 2024
• Post category: Scholarly Articles

Top 11 Qualities of A Good Researcher: Research entails the gathering of bits and pieces of information and compiling same for the purpose of increasing the stock in knowledge and solving problems. It is an art; a practice done to increase knowledge and render solutions to problems. This is because a research raises questions, which when answered, bridges the gap in knowledge in any field it was undertaken.

The task of researcing is no small work. This is due mainly to the fact that the information contained in a research work is usually for public consumption. There is no gainsaying the fact that anything made for the public must be above par. Hence, a research work which is geared towards adding value to knowledge as well as proffering solutions to problems must be rich in content and accurate.

Having known that a piece of research work is of great importance, the quality of such piece cannot be overlooked. Thus, for a research to be adjudged as being above standard, and also capable of serving its core purpose, the one undertaking the research must possess certain qualities. These qualities will enable him produce a work that will stand the test of time.

RECOMMENDED: Qualities of a good Enterpreneur

Traits/Qualities of a Good Researcher

These qualities which he must possess are outlined below:

1. Accuracy: A researcher must ensure that his research work is accurate. He should ensure that the facts and figures which he is presenting are true and verifiable. There should be no room for conjecture or guesses.

He should exhibit due diligence in presenting his work so as not to present a false and misleading research as the accuracy of the research determines the credibility to be attached to the researcher.

2. Open mindedness : To explain how powerful the mind is, Albert Einstein once said that the measure of intelligence is the ability to change the mind. It is one of the most important characteristics of a good researcher because researching has to do with finding new fact which may sometimes require that the researchers alter previously valid facts.

The crux is that, a researcher must not be someone who hardly gives up on his beliefs, custom or knowledge. He has to be someone who is able to see things in different lights. He must understand that nothing is actually static and things change over time.

Researching may sometimes lead you to find out that even some of the things you consider fundamental knowledge are not actually fundamental; or maybe there are more to it. Thus, a researcher who is not open minded during researching is limited to alot of knowledge.

Recommended: How to remember what you have read faster

3. Motivation: A researcher must have the ability to motivate himself to work. He should not be easily discouraged. In the course of his research, he might come across some hostile respondents. This should not deter him from carrying on his research work.

Again, he might be met with opposition from his colleagues. Such oppositions should not serve as a source of discouragement to him. A researcher therefore, must have the ability to encourage and motivate himself to push on until he actualizes his aim.

Also see: Ways to Improve Your Reading Speed

4. Patience: One of the sterling attributes of a good researcher is patience. This quality is a follow-up attribute to being motivated. A researcher must exhibit a high degree of patience, both with his respondents and in the course of his research when the much needed result is not forthcoming, in addition to his being consistent with the effort he puts in.

5. Prudence: The quality of being prudent has to do with his ability to manage the resources at his disposal. Research is a capital intensive project and there are possibilities that one might not be well funded to carry out such project. The managerial skill and ability of the researcher becomes handy for a successful execution of the project.

Also, even if the resources are in abundance, he must be able to manage same so as to avoid waste and extravagant spendings. Therefore, a researcher must be able to effectively manage the resources at his disposal for optimum output.

Also see: Characteristics of fundamental human rights

6. Expertise: Though the aim of research is to add knowledge to already existing knowledge (as one ventures into new areas not yet explored), the researcher must have reasonable amount of knowledge in the field he intends carrying out his research.

It will amount to a ruse if an archaeologist embarks on a research in medicine. A researcher must therefore be an expert in the field of his research or must possess at least, reasonable amount of knowledge in the field he professes to carry out his research.

7. Unbiased in his Research: At the beginning of his research, a researcher must not have a preconceived notion or idea about the subject of his research. This is because the research may likely not be accurate as his leanings will mostly be towards his bias. If there exists such bias, the researcher should be liberal enough to accept findings that are against his held beliefs. This will aid him to present an accurate and unbiased research work.

Also see: Highest paying law firms in Nigeria

8. Interest: A researcher must show sufficient interest in the work he is doing. He should not be distracted. His interest should be focused on the work. This is quite different from being motivated. This is because it is the interest one has in a task that keeps him motivated.

If one is not interested in a research work, he cannot be motivated, even if all necessary things are in place to drive the project. A researcher has to build interest from within, in the field of his research to see him through the research work.

9. Amiable Personality: A researcher must have a friendly disposition. He should be easily approachable and should also have the ability to communicate with people in a friendly and coherent manner. A researcher that is unfriendly, gloomy and unapproachable may find it difficult extracting information from his respondents.

A researcher should also exhibit friendliness to whoever works with him (if he is in a team with other researchers). This act of friendliness reduces friction within the team and the team is most likely to finish up their task in record time.

Recommended: Advantages and Disadvantages of an unwritten constitution

10. The Ability to Work Under Pressure : A researcher must be able to work under pressure and unfavorable situations. The ability to carry out a task within little time frame and also work in conditions that are less favorable (for example, under a hostile boss) is a quality which the researcher should have to enable him carry out his task.

Limited time may be allocated to accomplish a research project such that if the researcher is unable to manage his time, the work will not be done. It is therefore a needed quality of the researcher to be able to persevere and work in unfriendly situations if he must accomplish his task on time.

11. Analytical in his research: A researcher should be analytical and should also be able to exhibit sound judgment. Proper analysis of issues is key to having a good research work. His ability to reason rationally and give sound judgment affects the quality of the research work.

Where the ability to give sound judgment is missing, there may be a misapplication of principles and rules. This might be fatal if such misapplication happens in a research work of great importance to health. Therefore, a researcher must have the natural ability to discern what is right between two factors brought before him.

Also see: Best side hustles for students to make money while in school

Conclusively, whether a research work is an academic or organizational work, the researcher must be able to consistently gather information that are needed for the task. Research has to do with the discovery of new information and only a well planned method of information gathering can aid in the actualizing of this aim. The quality of a research work, its success and its usefulness is dependent on the level of diligence exhibited by the researcher. It is imperative that the researcher is one with good attributes to enable him scale the litmus test of acceptability of his work in the society.

Edeh Samuel Chukwuemeka, ACMC, is a lawyer and a certified mediator/conciliator in Nigeria. He is also a developer with knowledge in various programming languages. Samuel is determined to leverage his skills in technology, SEO, and legal practice to revolutionize the legal profession worldwide by creating web and mobile applications that simplify legal research. Sam is also passionate about educating and providing valuable information to people.

This Post Has One Comment

Easy to understand, simple and to the point thankyou

Comments are closed.

COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK

Research Assistant

• Columbia University Medical Center
• Opening on: Sep 24 2024
• Technical Grade 5
• Job Type: Support Staff - Union
• Bargaining Unit: SSA
• Regular/Temporary: Regular
• End Date if Temporary:
• Hours Per Week: 35
• Standard Work Schedule:
• Salary Range: $59,845.49-$59,845.49

Position Summary

 The Area Psychotic Disorders is seeking a full-time Research Assistant to work within the Horga Lab at Columbia Psychiatry (Guillermo Horga, MD PhD, Principal Investigator). The Research Assistant will support NIH/NIMH- and foundation-funded projects that aim to identify behavioral and neuroimaging studies in humans, with or without psychiatric conditions. The Research Assistant will have opportunities to contribute to research, grants (e.g., R-awards), and interdisciplinary collaborations across Columbia University.

Responsibilities

• (10%)-Developing and performing online experiments (e.g., Amazon M-Turk).
• (20%)-Performing fMRI experiments with human participants.
• (5%)-Maintaining research computing platforms and data servers, running QC and MRI/behavior analysis pipelines.
• (30%)-Recruiting, scheduling, interviewing, and administering tasks to participants.
• (30%)-Assist in the preparation of grant submissions, IRB, and compliance.
• (5%)-Perform other related duties and responsibilities as assigned/requested.

Minimum Qualifications

• Bachelor's degree and at least one and one-half years of related experience or equivalent in education, training and experience.

Preferred Qualifications

• Minimum of one (1) year of experience as a Research Assistant or performing equivalent duties in research labs (part or full time).
• Significant experience and training with regulatory procedures associated with human subjects research.
• Excellent programming and data analysis skills with experience in python, Matlab, and shell.
• High degree of self-organization and independent work.
• Strong decision-making skills, attention to detail, and the ability to solve problems independently.
• Team spirit, reliability, and sense of responsibility.

Other Requirements 

• Ability to perform work in a neat and orderly manner in accordance with safety regulations.
• Ability to present experimental data to the Principal Investigator.

Equal Opportunity Employer / Disability / Veteran

Columbia University is committed to the hiring of qualified local residents.

Commitment to Diversity 

Columbia university is dedicated to increasing diversity in its workforce, its student body, and its educational programs. achieving continued academic excellence and creating a vibrant university community require nothing less. in fulfilling its mission to advance diversity at the university, columbia seeks to hire, retain, and promote exceptionally talented individuals from diverse backgrounds.  , share this job.

Thank you - we'll send an email shortly.

Other Recently Posted Jobs

Quantitative Analyst, Columbia Investment Management Company

Refer someone to this job

• ©2022 Columbia University
• Accessibility
• Administrator Log in

Wait! Before you go, are you interested in a career at Columbia University? Sign up here! 

Thank you, for sharing your information. A member of our team will reach out to you soon!

This website uses cookies as well as similar tools and technologies to understand visitors' experiences. By continuing to use this website, you consent to Columbia University's usage of cookies and similar technologies, in accordance with the Columbia University Website Cookie Notice .

Advertisement

Prototype-based learning for real estate valuation: a machine learning model that explains prices

• Original Research
• Open access
• Published: 23 September 2024

Cite this article

You have full access to this open access article

• Jose A. Rodriguez-Serrano   ORCID: orcid.org/0009-0005-0239-8117 1  

44 Accesses

Explore all metrics

The systematic prediction of real estate prices is a foundational block in the operations of many firms and has individual, societal and policy implications. In the past, a vast amount of works have used common statistical models such as ordinary least squares or machine learning approaches. While these approaches yield good predictive accuracy, most models work very differently from the human intuition in understanding real estate prices. Usually, humans apply a criterion known as “direct comparison”, whereby the property to be valued is explicitly compared with similar properties. This trait is frequently ignored when applying machine learning to real estate valuation. In this article, we propose a model based on a methodology called prototype-based learning , that to our knowledge has never been applied to real estate valuation. The model has four crucial characteristics: (a) it is able to capture non-linear relations between price and the input variables, (b) it is a parametric model able to optimize any loss function of interest, (c) it has some degree of explainability, and, more importantly, (d) it encodes the notion of direct comparison. None of the past approaches for real estate prediction comply with these four characteristics simultaneously. The experimental validation indicates that, in terms of predictive accuracy, the proposed model is better or on par to other machine learning based approaches. An interesting advantage of this method is the ability to summarize a dataset of real estate prices into a few “prototypes”, a set of the most representative properties.

Explore related subjects

• Artificial Intelligence

Avoid common mistakes on your manuscript.

1 Introduction

Real estate valuation (Pagourtzi et al., 2003 ; d’Amato and Kauko, 2017 ) refers to estimating the price of a property with accuracy on the basis of the current market, and impacts a wide range of stakeholders across industry and society, such as individuals and organizations involved in real estate transactions, or policymakers seeking to manage urban development and housing policies effectively.

From an operations research standpoint, the ability of firms to predict real estate prices with precision is crucial for revenue management (Tchuente and Nyawa, 2022 ), portfolio optimization (Amédée-Manesme and Barthélémy, 2018 ; Kok et al., 2017 ), or risk management (Doumpos et al., 2021 ) and serves as a foundational step which impacts the operations of specific sectors, such as insurance or market research.

The progress in digitization and increased data accessibility has enabled the systematic and large-scale estimation of property prices, known as mass valuation  (McCluskey et al., 2013 ; Kok et al., 2017 ), often involving the use of statistical or machine learning techniques. According to Alexandridis et al. ( 2019 ), “automatic mass valuation can reduce operational costs since it is inexpensive and can be performed in a regular basis”.

A common approach to mass valuation of real estate properties is hedonic pricing (Malpezzi, 2003 ). Hedonic models decompose the utility of a good in the sum of utilities of its characteristics independently. An iconic example of a hedonic price model is OLS regression (James et al., 2013 ), wherein the unit value assigned to each characteristic corresponds to the regression coefficient. OLS has been widely applied to real estate valuation (Ottensmann et al., 2008 ) and possesses numerous benefits. First, it can be considered a supervised learning model: it can be adjusted with training data to optimize any given loss function, and later be used to predict the price of new properties. Second, it is easily interpretable: the price directly decomposes as a linear combination of the input variables. Third, OLS is a well-established, accessible and easily reproducible methodology.

Nevertheless, the OLS methodology for price prediction is not without common criticisms. These include its limited ability to handle intricate, non-linear associations or the risk for misleading coefficient interpretations when OLS assumptions are violated. Another drawback of OLS, which is crucial for real estate modeling, is that incorporating spatial coordinates is not straightforward. While technically feasible, it is unlikely that variables such as longitude and latitude have linear relationship with price. This is the reason why some hedonic models tend to incorporate manually constructed variables describing location (Tchuente and Nyawa, 2022 ) or other factors related to location, such as quality of the views (Potrawa and Tetereva, 2022 ) or employment potential (Ottensmann et al., 2008 ).

One step further in sophistication, state-of-the-art machine learning (ML) techniques, such as random forests (Breiman, 2001 ) or neural networks (Goodfellow et al., 2016 ) have attracted the attention of researchers in real estate valuation (Doumpos et al., 2021 ; Peterson and Flanagan, 2009 ). Their advantages include their capability to deal with non-linear relationships, which normally translates to superior performance. Importantly for real estate research, some models like decision trees, random forests or neural networks can accept location variables such as longitude and latitude without any further processing (Tchuente and Nyawa, 2022 ).

However, there is a prevalent consensus that while these systems excel in prediction capabilities, they often fall short in providing valuable insights due to their inherent lack of interpretability. Notably, the main criticism is the difficulty to assign a meaning to the model parameters. To counterbalance these limitations, some works in real estate valuation have successfully combined machine learning models with explainable AI (XAI) techniques (Lorenz et al., 2022 ; Potrawa and Tetereva, 2022 ). These techniques complement machine learning models by extracting metrics such as relevance scores for each variable. Still, an issue is that these techniques are applied ex post and remain detached from the model itself.

Furthermore, one additional limitation of both OLS and ML models is that they ignore a fundamental cue in valuation theory: direct comparison—perhaps the most common and intuitive way in which humans appraise real estate. In direct comparison, “the value of the property being appraised [...] is assumed to relate closely to the the selling prices of selling properties within the same market area”, as put by  Pagourtzi et al. ( 2003 ), who name this concept the comparables method . The same idea has also been dubbed direct capital comparison (DCC) by  Lins and L.F.d.L. Novaes, and L.F.L. Legey. ( 2005 ). In order to unify the terminology, in the following we refer to this concept as direct comparison .

If the models used for real estate valuation incorporated the notion of direct comparison, they would be better aligned with human expectations and market domain knowledge. Yet it can be clearly argued that OLS and, more broadly, machine learning approaches, do not implement explicitly the idea of direct comparison, because these models establish direct mappings from input variables to prices. In other words, most of these predictive models do not involve similarity computations between the input property and the nearby or similar properties.

There are notable exceptions to the previous statement, such as K-Nearest Neighbors (KNN) (Kramer and Kramer, 2013 ) and its variants. These are designed to offer predictions grounded in such comparative analyses and have exhibited good performance in the domain of real estate price prediction (Choy and Ho, 2023 ; Isakson, 1988 ). One technical drawback of KNN is that it is a non-parametric density estimation method; as such it cannot be adjusted to minimize a loss metric. The only mechanism to increase the quality of the estimation is by increasing K , which renders the method inefficient.

Based on this analysis, it is clear that none of the discussed approaches simultaneously complies with one of these characteristics: (i) capability to model complex and non-linear relationships, (ii) capability to optimize a predictive objective function, (iii) exhibit some degree of explainability, and (iv) implement the notion of direct comparison explicitly.

In this article, we propose a method that complies with these four characteristics simultaneously. The method is based on a methodology known as “prototype-based learning” (Biehl et al., 2016 ) which, to the author’s knowledge, has not been previously applied for real estate valuation. Prototype-based learning is a predictive model that:

can learn non-linear relations between the dependent and input variables,

is a supervised learning method that can be trained to minimize any given loss function (unlike KNN),

includes explainability as part of the model, and

implements explicitly the notion of direct comparison: the model predicts the price of a property by contrasting the property of interest to a set of (automatically inferred) reference properties.

Table 1 summarizes these four characteristics and compares the aforementioned models and approaches in terms of which of them they fulfill.

The model works by minimizing a prediction error, based on comparing the variables of a property with the distributions of a set of references, and then optimizing these references. Therefore, as a by-product, the model produces a segmentation of the data into groups (or areas) with differentiated characteristics, similar to sub-market models (Goodman and Thibodeau, 2007 ; Rey-Blanco et al., 2024 ).

We study prototype-based models using a public dataset of real estate prices in Seattle. Our experiments conclude that a) prototype-based learning successfully distills the prototypes as theoretically expected, b) the method outperforms other strategies for finding prototypes, c) the prediction error observed is lower compared to other machine learning models, and d) the model can incorporate location-based information in an effective way.

The rest of the article describes the model (Sect. 3 ) in detail, reports the different experiments (Sect. 4 ), provides a detailed discussion (Sect. 5 ), reviews the practical and research implications (Sect. 6 ) and comments on conclusions and future perspectives (Sect. 7 ).

2 Related works

2.1 hedonic models and ols.

The origins of the hedonic pricing theory have been a subject of debate between scholars (Goodman, 1998 ; Colwell and Dilmore, 1999 ). It is often acknowledged that the first use of “hedonic” is by Court ( 1939 ). However, Haas ( 1922 ) already proposed to predict land price per acre based on a linear model more than one century ago.

As of today, OLS is still a common method to model, and predict, land or real estate prices. We refer the readers to  Hong et al. ( 2020 ) and  Potrawa and Tetereva ( 2022 ) for comprehensive reviews of prior research that adhere to this methodological approach and delve into various categories of employed variables. A well-established consensus, substantiated by empirical evidence, underscores the positive correlation between property prices and factors such as floor area, the number of bedrooms, and bathrooms. In contrast, certain variables, such as property age, exhibit a negative association with property prices (Hong et al., 2020 ). It is important to consider factors other from those describing the physical property, such as “locational” or “neighborhood”-based factors (Potrawa and Tetereva, 2022 ) which tend to exhibit a strong influence, too. For instance, Ottensmann et al. ( 2008 ) and Hussain et al. ( 2019 ) study the impact of urban factors, such as proximity to employment locations, or impoverished neighborhoods, respectively, in real estate prices. Some works even go as far as using variables encode the quality of the view, as extracted by image analysis techniques (Potrawa and Tetereva, 2022 ) or quality scores extracted from user reviews (Gibbs et al., 2018 ).

OLS finds advantages in its simplicity and ability to interpret the coefficients as the average increase in price after a one-unit increase of the corresponding variable. However, some drawbacks include the limitation of the functional form, incapable to model non-linear relations between variables, interactions or colinearities (Zurada et al., 2011 ). Another common drawback is the usual assumption of linear distribution, rarely observed in practice (Potrawa and Tetereva, 2022 ). Despite these (well-known) technical limitations, OLS still finds many applications, as some practitioners find practical productivity increases in predictive applications.

2.2 ML models

As an alternative to OLS, previous studies have resorted to more advanced machine learning techniques in order to obtain models with more complex functional form.

This includes the use of neural networks (Peterson and Flanagan, 2009 ; McCluskey et al., 2013 ), which produce non-linear predictions by building a chain of linear operations and non-linear activation functions, and are considered universal approximators (Goodfellow et al., 2016 ). Other common approaches to produce robust non-linear characteristics are the random forests (Hong et al., 2020 ) or other forms of tree-based ensembles (McCluskey and W., D. Zulkarnain Daud, and N. Kamarudin., 2014 ; Kok et al., 2017 ). These models are based on regression trees, which produce predictions based on a sequence of variable tests, capturing non-linear relations and interactions between variables. Then they average many regression trees using different criteria to increase robustness.

Neural networks and tree-based ensembles are probably the two most common state-of-the-practice approaches in machine learning. We refer the reader to Doumpos et al. ( 2021 ) or Tchuente and Nyawa ( 2022 ) for an exhaustive review and comparison between different machine learning methods in real estate modeling.

Because of their added complexity, machine learning models tend to be, on the one hand, more accurate than OLS. On the other hand, an oft-cited issue of most ML approaches is their lack of interpretability. Interpretability denotes the capability to easily understand and describe how an output is produced given an input, and is an important feature for market regulators, or in general to augment the trust in decision-support systems. While some ML models are considered white-box, such as decision trees, most state-of-the-art ML models, such as those based on neural networks or tree ensembles are considered black-box (Tchuente et al., 2024 ). To overcome that limitation, there has been a recent emergence of explainable AI (XAI) techniques (Lundberg et al., 2020 ). In real estate prediction, Lorenz et al. ( 2022 ) have applied XAI methods such as partial dependence plots and feature importance to complement ML-based models, the latter also being used by Potrawa and Tetereva ( 2022 ). XAI methods are employed to diagnose black-box models, often are applied ex post, and are not part of the model. In contrast, as acknowledged by Li et al. ( 2018 ) (and further explained in Sect. 3 ), prototype-based models provide explanations naturally as part of the model.

2.3 Towards direct comparison: KNN and clustering

Among specific machine learning models used in real estate research, those that take into account direct comparisons by design include those based on KNN (Choy and Ho, 2023 ; Isakson, 1988 ; Tchuente and Nyawa, 2022 ). In KNN, the regression is taken as a weighted average of the outputs of the nearest data points of the training set, based on some distance (usually, Euclidean) between the vectors of input variables.

As such, KNN is capable of modeling complex, non-linear effects between price and the input variables. Its advantages also includes its methodological simplicity and interpretability. However, one drawback of standard KNN on the predictive performance side is that it is a non-parametric model and cannot be adjusted based on a target loss function.

In order to combine the benefits of KNN and other machine learning methods, some real estate studies perform spatial weighting schemes (Bitter et al., 2007 ; McCluskey et al., 2013 ), of which a well-known type is geographically weighted regression  (Fotheringham et al., 2003 ). Spatial weighting assigns more importance during model training to those samples which are proximal to the example whose price is to be predicted.

The model proposed in this paper addresses both characteristics: first, it uses comparative-based measures, akin to KNN, while offering the flexibility to be fine-tuned based on a user-defined loss function. Experiments demonstrate that this crucially reduces the prediction error. Second, it incorporates spatial weighting as an intrinsic feature.

2.4 Prototype-based learning

In summary, the advantages of prototype-based models in terms of modeling non-linear relations, explainability, availability of comparison-based valuation, and ability to be optimized given a loss function make those models very promising for predicting real estate prices.

The term prototype-based models  (Biehl et al., 2016 ) encompasses a family of approaches rather than a singular methodology, and within this family we focus on the subset of supervised learning approaches. These make predictions by leveraging latent “prototypes”, which refer to a reduced set of data samples which summarize most of the patterns occurring the whole dataset. Prototypes can correspond to actual data points, or can consist of calculated variables and not strictly correspond to any real observed datum. Additionally, when the space is segmented based on proximity to prototypes, these models facilitate the identification of distinct regions or clusters within the data space, effectively acting as “cluster-aware supervised learning algorithms” (Chen and Xie, 2022 ).

The next section is devoted to describing the specific method employed in the paper and will introduce many technical aspects of prototype-based methods. This methodology finds applications in diverse areas such as computer vision (Li et al., 2018 ), natural sciences (Owomugisha et al., 2021 ), or healthcare (van Veen et al., 2022 ), including COVID detection (Kaden et al., 2022 ). To the best of the author’s knowledge, however, this method has never been applied to real estate prediction, in spite of its appealing benefits.

3.1 Notation

The proposed model requires a dataset of real estate properties with N samples, each with D independent or (in the following) input variables \(x_{nd}\) (thus \(n\in \{1,\ldots ,N\}\) and \(D\in \{1,\ldots ,D\}\) ) describing characteristics of the real estate properties, and a dependent or output variable \(y_n\) indicating the observed price. By price we mean either the direct price, or any transformation of the price, such as the price normalized by a constant or non-constant factor (e.g. price per square foot), or the logarithm of the price, just to mention some commonly used options. Our model is compatible with all these definitions.

We will use the boldface form \(\textbf{x}_n\) , common in the ML literature, when we have to refer jointly to the set of all input variables \(x_{n1}, \ldots , x_{nD}\) of sample n . We will omit the index n when it is not necessary for the discussion. In a slight abuse of language, occasionally the text may refer to \(\textbf{x}\) shortly as a “property” (instead of “the values of the input variables describing a property”).

3.2 Basic model structure

The main design principle of our model is that it should implement the notion of direct comparison: if we observe a new property with input variables \(\textbf{x}\) , we should be able to infer its price by comparing it to a number of “references”, denoted as prototypes, and interpolating from the price of similar references.

Next, we discuss how the model proposed in this article directly implements this principle.

We define a set of M prototypes , which represent a set of properties with similar input variables and with an attributed price. Formally, each reference consists of:

A prototype distribution, which models the probability distribution \(f_m(\textbf{x})\) of properties within group m . In the simplest case, we take a multivariate Normal distribution

where \(\textbf{p}_m = (p_{m1},\ldots ,p_{mD})\) specifies a vector of means and \({\varvec{\sigma }}_m\) refers to the vector of standard deviations (assuming diagonal covariance). Footnote 1 Because \(\textbf{p}_m\) represents the mode of \(f_m\) , it therefore indicates the typical values of the input variables for each prototype.

A prototype value \(v_m\) , which models an price attributed to that group of properties.

To compare a property \(\textbf{x}\) to each of the M references, a principled method (Bishop, 2006 ) is determining the posterior probability that \(f_m\) has generated \(\textbf{x}\) , i.e.

This comparison should yield, ideally, a small set of properties with non-negligible values of \(\gamma _m(\textbf{x})\) which represent the most similar references. Then the predicted value is a a weighted average Footnote 2 of the references, where the weighing coefficient is precisely the value of \(\gamma \) :

As we will verify empirically later, it is common that most values of \(\gamma _m(\textbf{x})\) are negligible and only a few terms contribute to the sum, and therefore Eq. 3 can be understood as an “interpolation” of the attributed values of the most similar references.

Note that the proposed model is completely determined by \(p_{md}\) and \(v_m\) (and also \(\sigma _{md}\) in case these are optimized). We jointly denote these parameters as \(\theta \) . Its optimal values will be learned from the data using the following loss function:

The first term is as mean squared error (MSE), which measures the fitness error of the model with respect to the data, and R is a regularization term, common in ML tasks to penalize model complexity or to impose reasonable restrictions. In our case, we use

This term averages the distance between each prototype and its closest data point, similar to Li et al. ( 2018 ), therefore adding a penalty if prototypes \(\textbf{p}_m\) are far from real properties \(\textbf{x}\) (here, \(\Vert \cdot \Vert \) stands for the Euclidean norm).

It is straightforward to identify that Eq. 4 is differentiable with respect to \(\theta \) . Therefore, we can use standard solvers that involve automatic differentiation (Baydin et al., 2018 ) such as | keras| to implement and this model and find its optimal parameters.

3.3 Model interpretation and simple example

The parameters \(p_{md}\) , \(v_m\) of the model have a natural interpretation. Specifically, as \(p_{md}\) is the mean of the distribution \(f_m\) , it can be interpreted as the variables of a representative property, and \(v_m\) as its representative price. Therefore, not only will the model be able to predict property prices, it also produces a list of M prototypical properties and suggests values for their expected variables and price.

To illustrate the model capabilities, for now we select a simple example on a controlled setting (experiments on real data are reported in the next section). Figure  1 shows a scatterplot corresponding to a small dataset with two input variables. The locations of the circles on the horizontal and vertical axes represent, respectively, the input variables \(x_{n1}\) and \(x_{n2}\) , and the color indicates the value of the output variable y (yellow for \(y=0.0\) and blue for \(y=1.0\) ).

In this case, we train the prototype-based model with \(M=20\) , and using a loss function with \(\eta =0\) . In this setting, the parameters can be directly translated to the plot: \(p_{m1}\) and \(p_{m2}\) represent the location of the prototype means on the horizontal and vertical axes. The color indicates the prototype values \(v_m\) (black for \(v_m=0\) and white for \(v_m=1\) ). The color scale is continuous, although perhaps that is not perceivable as most prototype values are close to \(v_m=0\) or \(v_m=1\) .

(Best viewed in color) Illustrative example of the prototype-based method. (Color figure online)

By visual inspection, it is clear in this example that the locations \(p_{md}\) of the prototypes span the different “clusters” in the data and the values \(v_m\) match the values of \(y_n\) in each area. As a conclusion, we have quantitative evidence in a controlled example that the obtained prototypes can be interpreted as representative property characteristics and their price.

To shed light on how the model implements the intuition of direct comparison , consider the prediction for a given sample \(\textbf{x}\) . To apply Eq. 3 , first we need to determine the values of \(\gamma \) in Eq. 2 . Continuing with our example, in the case of two variables, using a bivariate Normal distribution for \(f_m\) , we have:

If we define \(\lambda _m\equiv \log f_m(\textbf{x})\) , then Eq. 2 becomes

sometimes known as the softmax function.

As we can verify from Eq. 6 , as the Euclidean distance from \((x_1, x_2)\) to \((p_{m1}, p_{m2})\) decreases, the value of \(\gamma _m\) increases and the contribution of the value \(v_m\) becomes higher. Crucially, this is precisely the expected behavior in a comparison-based model.

3.4 Model extension as neural network

The prototype distribution \(f_m\) in Eq. 1 takes as input a property \(\textbf{x}\) . However, we note that our model is compatible with approaches that first compute a so-called embedding \(\textbf{h}= g(\textbf{x})\) , where g denotes the result of neural network layers applying some transformation to the input variables. In this section, we demonstrate how our model can be interpreted as a neural network and integrated into standard neural network frameworks. We utilize common neural network terminology throughout. Readers who are not familiar with neural network models can skip this section without impacting their understanding of the rest of the article. Alternatively, they may refer to foundational texts such as Goodfellow et al. ( 2016 ) for further information.

First, note the sequential computation of Eqs. 2 and 3 can be expressed in neural network vocabulary:

Eq. 2 is equivalent to passing the inputs \(\textbf{x}\) through a probabilistic custom layer with M units, where the m th unit outputs \(\log f_{m}(\textbf{x})\) , and then applying a standard “softmax” activation function.

Eq. 3 is simply a dense layer, which takes as input the previous outputted \(\gamma \) ’s, multiplies them with some weights and outputs \(\hat{y}\) . Since there is only 1 output, there are m weights which are precisely the prototype values \(v_m\) .

A schematic representation of the prototype-based model interpreted as a neural network is depicted in Fig. 2 a.

a Prototype-based model interpreted as a neural network. b Example of extending the prototype-based model neural network with an additional hidden (embedding) layer

This interpretation as neural network has two main benefits. First, it facilitates the code implementation of our model, as the previously described network can be implemented using commonly available libraries such as | tensorflow| or | keras| (see Sect. 4.2 for implementation details).

A second benefit is that we can add further hidden layers between the inputs and the custom probabilistic layer. This adds a (potentially non-linear) transformation of the input variables and thus increases the complexity of the predictive function. An example where a hidden layer (embedding) is added to our model is illustrated in Fig. 2 b. With this architecture, now the input to the prototype-based model is the output of the hidden layer. Footnote 3

In such a case, the parameters of the neural network g can be pre-trained, or jointly optimized during the training of the prototype-based model. Technically, one needs to replace \(f_m(\textbf{x})\) by \(f_m(\textbf{h})\) and this turns the loss function in Eq. 4 an implicit function of g and its weights. This change is straightforward to apply in the aforementioned automatic differentiation tools.

One drawback of this extension is that the prototype-based model becomes non-interpretable if additional layers are added. However, in some cases, prioritizing predictive accuracy over interpretability may justify this approach. Nonetheless, this extension is optional, and we simply emphasize that our model is compatible with it.

4 Experimental validation

We use data from a record of properties in King County (Seattle), Footnote 4 which includes property prices and other variables collected from May 2014 to May 2015.

The dataset contains information about 21,613 properties. Table 2 indicates a summary of the variables available in the dataset that will be relevant for the analyses below, and some typical statistics. Fig. 3 displays a map of the location of the properties , where the color indicates the price per square foot.

(Best viewed in color) Locations of properties in out dataset on a real map. The color intensity indicates the quantile of the price per square foot (darker indicates higher). (Color figure online)

4.2 Model implementation

The model is implemented in the Python programming language, using | keras| as the automatic differentiation engine. The code is publicly available. Footnote 5

4.3 Model using location variables only

The first experiment will evaluate the model considering exclusively location and price per square foot. There are several reasons for this choice. First, it is widely acknowledged that “location is the primary factor in determining market prices”  (Gabrielli et al., 2023 ) and that this primacy arises from its multifaceted influence on factors such as proximity to economic opportunities, appeal of certain leaving areas or transportation accessibility. Ultimately “virtually every attribute of a piece of land beyond its physical dimensions including any improvements can be reduced to location” (Atack and Margo, 1998 ). Therefore, we simulate a model that predicts real estate price solely based on location. Second, when the input variables are location, the prototypes \(\textbf{p}_m\) of our model can be directly interpreted as a list of typical locations, and the values as typical prices in these locations. While deliberately simplistic, this is still very relevant information for practitioners in real estate or seeking to obtain a geographical segmentation of the market and sub-market analyses (Goodman and Thibodeau, 2007 ).

Experimental protocol

To perform model training and evaluation, an initial step involves a linear partition of the dataset into training and testing subsets (with a training/test distribution of 60%/40%), structured in accordance with chronological order (given by the date column). This approach emulates real-world conditions where available information is time-bound, and the goal is to predict property values beyond a specific temporal point. This method effectively mitigates the potential leakage of temporal information into the predictive models. Notably, within the training subset, 20% of the data is reserved for parameter validation, ensuring that model parameters are optimized effectively.

For the analysis, we take latitude and longitude as the input variables, and construct an output variable price per square foot as the quotient of the available columns price and square feet .

In the data pre-processing stage, each input variable is standardized (to zero mean and unit variance). The output variable y (price per square foot) is scaled by a factor of 1/1000, with the objective of rescaling the values of y to fall within the approximate range of (0.0, 1.0), which makes numerical optimization more favorable.

To build our model, we use \(M=50\) prototypes. The model is implemented and optimized using the | keras| library, with an Adam optimizer with learning rate \(l_r\) , batch size b and number of epochs E . To these hyperparameters, we add the regularization coefficient \(\eta \) of Eq. 4 and also use a constant preset \(\sigma \) for n Eq.  1 . All these hyperparameters are systematically determined using a standard grid search strategy on the validation set. We highlight that we use a constant value of \(\sigma \) for both coordinates and for all prototypes, so that the computation of importances \(\gamma \) of Eq. 2 is explicitly a function of the Euclidean distance between each property and the prototype mean.

Our model is subjected to comparative evaluation against two alternatives. First, we conduct a comparison with a nearest neighbor algorithm using randomly selected neighbors and M=50 (in order to match the number of prototypes). This comparison offers insights into the efficacy of the prototypes in guiding the predictive process. Second, we compare with an alternative way of discovering prototypes, wherein we use a clustering algorithm. Clustering stands as an alternative for identifying groups of properties with similar characteristics and valuations. Therefore, we apply a K-Means clustering algorithm (Bishop, 2006 ) with \(K=50\) and take its means as the prototypes \(\textbf{p}_{m}\) , and the average price per square foot in each cluster as the \(v_m\) . For all models, we measure the quality of fit with the root mean squared error (RMSE), in the original y -scale (before normalization).

To mitigate the impact of sources of variability such as random seeds used in parameter initialization, we repeat the experiment a total of 10 times, and report the RMSE averaged over the 10 runs.

Quantitative results

We first evaluate the predictive accuracy of the model. The RMSE values of the different models tested in experiments described above are shown in Table  3 . We observe the prototype-based model yields significantly lower RSME than the two baseline methods (the standard deviation of the RMSE over the 10 runs is smaller than 1 $/ft \(^2\) ), therefore indicating that it has accurate prediction capabilities.

Results: Interpretation of the prototypes

Now we turn to discussing the interpretation of the prototypes \(\textbf{p}_m\) obtained by the model. Because our input variables are longitude and latitude, the model parameters \(\textbf{p}_m\) and \(v_m\) can be interpreted directly as the prototype locations and prototype values. Figure 4 displays the locations \(\textbf{p}_m\) overlaid on the original data.

(Best viewed in color) Locations of the prototypes \(\textbf{p}_m\) overlaid in the original data, and showing the corresponding \(v_m\) for a specific example. (Color figure online)

Through our analysis, it becomes evident that the prototypes \(p_m\) exhibit a wide distribution that effectively spans the different regions of the entire dataset. Furthermore, as depicted in Fig.  5 , a closer examination is directed towards a specific prototype, shedding light on properties in close proximity to it or, in other words, all the properties that can be succinctly encapsulated or “summarized” by the respective prototype.

Zoom into one of the prototypes. (Color figure online)

We verify by visual inspection that the values \(v_m\) and the nearby prices are consistent. Therefore, we qualitatively verify that the behavior of the model mimics the process of valuation-based comparison that is similar to the human intuition behind pricing.

Also, we quantitatively investigate how many prototypes typically contribute to each prediction when evaluating Eq. 3 . To that end, we compute the prototype importances \(\gamma _m(\textbf{x})\) for all properties \(\textbf{x}\) of the test set, and count the non-negligible terms as those with \(\gamma _m(\textbf{x}) > 0.01 \) . Table 4 displays the resulting frequencies. Notably, in no case did we observe more than 5 prototypes with \(\gamma _m(\textbf{x}) > 0.01\) . Furthermore, for over 94% of the test samples, only 1 or 2 prototypes significantly contributed to the prediction.

Interpretation as zonal segmentation

The model’s output can alternatively be interpreted as a zonal segmentation: defining the m th “zone” as the area where all properties are closest to the m th prototype results in a Voronoi segmentation, as illustrated in Fig. 6 . Each zone represents a geographical area with similar property prices, where the reference price is specified by the prototype value \(v_m\) . In Fig. 6 , two examples of \(v_m\) are depicted to illustrate this concept.

Intepretation of the model as a zonal segmentation: each zone represents an area with approximately constant prices, where that constant is given by the obtained values \(v_m\) . Two examples are shown: a \(v_m \approx 424\) , b \(v_m \approx 165\)

This interpretation is connected to research in sub-market segmentation. According to Thibodeau ( 2003 ), “a housing [sub-]market defines a geographic area where the price of housing per unit of housing service [...] is constant”. Therefore, our model could be interpreted also as a means to perform sub-market segmentation, since each prototype m roughly defines an area where, in its vicinity, the price is approximately a constant \(v_m\) .

It is noteworthy that many traditional approaches to sub-market segmentation rely on grouping predefined market-based or policy-based elemental areas, such as postcodes or school districts (see, for example, Table 1 in Borst and McCluskey ( 2008 )). In contrast, our approach infers sub-markets automatically from raw data using a machine learning algorithm. This rather aligns us with recent works such as Rey-Blanco et al. ( 2024 ).

In any case, we emphasize this capability to produce market segments automatically as a practical by-product of the proposed model. An in-depth analysis of prototype-based models to the specific task of submarket segmentation would require a separate study, which we defer it to future work.

Results: On the effect of the number of prototypes

We now study the effect of the number of prototypes. We repeat the previous protocol using different number of prototypes and compare the three different methods with \(M=10,20,50,100\) . Results are reported graphically in Fig. 7 .

As expected, the error decreases for all methods with more prototypes but the proposed approach gives the smallest errors across all values of M . Also, if we observe how the error increases by decreasing M , our model degrades with a smaller slope compared to the other models.

RMSE of the different compared methods for different number of prototypes

4.4 Model with more variables

Following the protocol established in the last section, we now incorporate a more comprehensive set of variables. Apart from latitude and longitude, we consider number of bathrooms , square feet of living room , square feet of lot , number of floors , waterfront , view , square feet above , square feet of basement , square feet of living room in 2015 , square feet of lot in 2015 . Thus, the new model uses 12 input variables in total.

The experimental details are the same as in the last section, with one exception: because now we use several variables of different types, it is reasonable to expect that different variables need to be weighted differently for the importance computation of Eq. 2 . Our model also accepts this design choice, which translates to fitting the values of \(\sigma _m\) of Eq.  1 . For completeness, we will report the results using a constant hyperparameter \(\sigma \) for all prototypes (equivalently to the last section), and also by explicitly learning the \(\sigma _{md}\) ’s as part of the optimization process.

The results are summarized in Table 5 . Again, the table displays the values of RMSE for the same methods of the previous section. In this case, we found the random neighbor initialization of the KNN method yields a higher variance between the 10 runs. Therefore, we also report the minimum and maximum RMSE across the 10 runs to make clear there is no overlap in the error metrics.

There are some noteworthy observations when comparing to the results of the previous section, which leveraged location variables only. First, the RMSE of the prototype based method (using constant \(\sigma \) ) has improved (down to 68.1 in Table 5 vs. 76.6 of Table 3 ). This underscores the positive impact of adding more variables in the prototype-based model. This impact is even larger when one considers the improvement brought by learning the parameters \(\sigma _m\) (down to 62.0).

We can still interpret the means \(\textbf{p}_m\) as the prototype modes—with the clarification that now these do not only specify the typical locations, but also the typical values of each variable considered. As an illustration, Fig. 8 displays six scatterplots, covering the 12 input variables, and also the projections of the prototypes in the corresponding subspace.

Scatterplots for the 12-variable case, showing pairs of variables and the obtained prototypes

Second, when inspecting the results for the other 2 methods, we observe an opposite trend: in both cases the RMSE has not improved but increased as more variables are integrated (for K-Means, RMSE increases to 90.9 using the 12 input variables, up from a value of 81.7 with the model of the previous section; for KNN the increase is from 110 to 115).

A possible explanation for this phenomenon is as follows. When we used latitude and longitude as the only input variables, the proximity to a prototype (both KNN and K-Means are based on Euclidean distance) is a good proxy for the similarity in price, as argued previously. However, when additional variables are introduced, the distance calculation between a property and the prototype becomes a composite measure, encompassing multiple components, some of which may not have a direct correlation with price differences. In essence, this expanded feature space introduces dimensions that are not necessarily indicative of variations in property prices. Consequently, similarity between variables does not translate to an effective similarity in price. Footnote 6 The reason why our prototype-based model exhibits the opposite behavior can be attributed to the model’s ability to adjust the optimal locations of the prototypes (and, especially, the scales of each dimension). By doing so, it effectively hones in on the specific variables and combinations thereof that are most relevant for accurate price estimation.

Approximating the target distribution

Another research question we could ask is do the model predictions \(\hat{y}\) also approximate well the distribution of observed test prices y ? To that end, Fig. 9 displays the histogram of both test and estimated prices. For the case of \(M=50\) (the setting used in this section), indicated on the left, we see the distribution of estimated prices substantially overlaps that of real prices, albeit putting more mass in the center of the distribution and less in the extremes. However, this phenomenon is not surprising and, to some extent, expected, since prediction models can be interpreted as “smoothing the data” (and consequently reduce extreme observations). To verify this, if we increase M , as we use more prototypes we should expect less smoothing, which is clearly visible in the plot on the right for \(M=100\) .

Comparing the distribution of test prices with the distribution of estimated prices, for \(M=50\) (left)and \(M=100\) (right) prototypes

4.5 Comparison with other predictive models

So far, the prototype-based model has shown some interesting advantages in terms of predictive power and also in terms of explainability. And the previous sections have demonstrated that it outperforms other alternative ways of obtaining prototypes, both in the location-only case and when using multiple variables.

Now we compare the prototype-based model with three models discussed in Sect. 2 : (i) regression trees, (ii) random forests and (iii) a standard neural network based on a multi-layer perceptron. Footnote 7

First, we discuss the comparison settings. For all methods, we still use identical train/validation/test splits as in the case of the prototype-based model. We select the best hyperparameters by evaluating the RMSE of the trained models in the validation set. Then, we report the RMSE of the best hyperparameters on the test set. Again, we repeat this process 10 times and report the average RMSE, in order to mitigate factors such as randomness in parameter initialization.

For the regression tree, the only hyperparameter is the maximum number of leave nodes. The random forest also considers this parameter and also the number of trees. Finally, for the neural network we tune the following hyperparameters: the learning rate, the batch size, the number of epochs, and specific combinations of number of units and layers. Note that each setting of the hyperparameters determines the size of the model. For example, in a neural network, each unit has a number of coefficients equal to the number of units in the previous layer plus one. In the case of trees, each intermediate node has two parameters (the index of the variable and the threshold value), and the leaf nodes have one parameter (the value of the regression). We want to make sure the models are comparable to the prototype-based model in terms of the number of parameters. Therefore, when doing hyperparameter optimization, we restrict them to the ones that yield models (roughly) comparable in size to our prototype-based model (the model with \(M=50\) for the 12 input variables and learned scales \(\sigma _m\) has 1250 parameters).

Table 6 shows the RMSE of the different methods on the test set, the number of parameters of the models, and a column reminding whether the model is explainable.

In the case of the decision tree, by setting the maximum number of leaf nodes to 250, we get a tree with 1748 parameters, and the test RMSE is 67.6 $/ft \(^2\) . In the case of the random forest, setting max leaf nodes to 50 and number of estimators to 5 yields 1740 parameters and a test RMSE of 65.8 $/ft \(^2\) (in the case of random forest, the error can be further reduced to 64.6 $/ft \(^2\) , but using 50 estimators, which yields 17,400 parameters, many orders of magnitude higher than for our model). The best performing neural network has 3 hidden layers, with 22 units in each layer (amounting to 1321 parameters) and yields a test RMSE of 61.9 $/ft \(^2\) .

When we compare these values with the RMSE reported by the prototype-based model in the previous section (included in the table as a reminder), we verify that its error is significantly lower than for random forests and regression trees, and on par with the neural network. We conclude our model is comparable or better to the other alternatives, but with the added advantages, such as explainability.

4.6 Results with the neural network extension

Finally, we qualitatively evaluate the extension of the model as a neural network introduced in Sect. 3.4 . Specifically, we illustrate the capability to apply the prototype based model to a so-called an embedding computed from a pre-existing neural network, instead from the input variables. As discussed in Sect. 3.4 , this increases the complexity of the model but hinders explainability.

For the pre-existing neural network, we use the network trained in the last section, remove the output layer and add a 2-unit layer. This produces an intermediate representation which, consistently with Sect. 3.4 , we denote as \(\textbf{h}\) . Then, we feed \(\textbf{h}\) to the prototype-based model.

Implementation-wise, we proceed in two separate steps: first, we transform the properties \(\textbf{x}_n\) to embeddings \(\textbf{h}_n\) by computing a forward pass, and use these embeddings \(\textbf{h}_n\) to train the prototype-based model.

Because \(\textbf{h}_n\) has two dimensions, we can visualize the prototype distribution in this space. The result is in Fig. 10 .

Prototypes (red) obtained for the neural network embeddings of properties (blue). (Color figure online)

We visualize the prototype modes (red) on top of a sample of properties in the projected space \(\textbf{h}\) . While it may not be possible to assign an interpretable meaning to each projection component, we identify some structure, e.g. top left is lower prices, bottom right higher prices. Also, by inspecting prototypes we can still assign a meaning to some of them, as indicated in the figure, based on the value (such as “expensive apartments with no basement and good views”).

This experiment is to demonstrate qualitatively the potential extensions, but it suggests interesting directions for application and research.

5 Discussion

The main desideratum of the prototype-based model is to predict real estate prices mimicking the process of comparison-based valuation. The results in the previous section have validated that the model has the correct behaviour, in two aspects. First, the interpretation of the prototype distributions is as expected: qualitative analysis using data visualizations show that the prototypes are well-spread across all the space of input variables, as summarized in Fig. 8 . As a matter of fact, when using spatial variables only (latitude and longitude) the prototypes spread well across all the different physical regions, as depicted in Figs. 4 and 5 . Inspection of the prototype values also indicates a reasonable segmentation of the space and seemingly well-approximated price estimates.

Second, on top of this qualitative examination, the quantitative evaluation confirms that the prototypes and values obtained by the model, in different settings, are appropriate. The strongest argument for this is that the predictive errors obtained by the prototype-based model are smaller than (i) alternative ways of constructing the prototypes, as summarized in Table 3 , and (ii) alternative machine learning methods, as summarized in Table 6 .

The experiments have focused on predicting the price per square foot, as this is a well-understood measure of unit of constructed land. We conducted experiments in two settings: predicting the price based only on location, and predicting based on location and other variables describing the property. The first setting is compatible well-acknowledged assumption in the real estate sector that location is the crucial factor of land price (Gabrielli et al., 2023 ; Atack and Margo, 1998 ). Even though this setting could be regarded as simplistic, a result like the one in Fig. 4 could be already interesting and actionable by firms with high stakes in the real estate market: it can be interpreted as a zonal segmentation of the real estate market, sometimes also referred to as a sub-market analysis (Goodman and Thibodeau, 2007 ) (as analyzed in Sect. 4.3 ). Effectively, this information alone already produces accurate predictions: by inspecting Tables 3 and 6 , we actually note that the error of the prototype-based model using location-only variables is on par with those of machine learning models using all the variables. We conclude that location information alone, when coupled with a model that exploits this information adequately, is a very powerful signal.

Quantifying the location cliché

We could actually quantify the “location, location, location” cliche according to our data and model, which can be evaluated by comparing the \(R^2 = 1 - RSS/TSS\) values of our regression models, where RSS denotes the residual sum-of-squares of the model under consideration, and TSS denotes the sum-of-squares with respect to the mean of the test data.

For the model using the location variables only, \(R^2=0.56\) , for the model using all the variables but constant \(\sigma \) , \(R^2=0.64\) , and for the model using all the variables, \(R^2=0.73\) . By inspecting the RSS values, the second model reduces the variance by about 20% compared to the first, and the third model reduces the variance by about 34% compared to the first. We see a big part of the variance is captured by the location variables, but adding more variables produces a significant reduction. According to this analysis, location is far from being the only factor that is significant.

However, these ratios also show an apparent limitation of the model, as the fraction of variance explained by the best model is \(R^2=0.73\) , indicating that there is a substantial portion of the variance not accounted for by the model, which can be attributed to unobserved variables or inherent irreducible noise within the dataset. This is not surprising given the complexity in the real estate market and is comparable to other studies (Tchuente and Nyawa, 2022 ). Some studies have indeed tried to cover this gap by capturing non-conventional variables such as the quality of the view from the window (Potrawa and Tetereva, 2022 ) using computer vision techniques.

In any case, this limitation points the need for caution when utilizing the model for individual sample predictions, as it may not adequately capture the intricacies of a specific data point. However, it is essential to recognize the utility of the model when used for group predictions or to summarize a real estate dataset with a reduced set of prototypes.

Modeling location in a principled way

Another aspect of the model is that it treats location in a general purpose way. Usually, location is difficult to model. Some previous studies added variables to the model such as the distance to different employment centers (Ottensmann et al., 2008 ). Those studies were focused on inference and precisely studied the effects of proximity to those key locations in the price. However, when the goal is price prediction with a general-purpose methodology that is valid across different markets, it will be cumbersome to specify the key locations for each new study. The prototype-based model will capture the price changes in different areas regardless of the reason, without having to define specific variables.

Prototype distributions

The proposed model employs Normal distributions for the prototypes, for several reasons. First, we note previous prototype-based models, such as Li et al. ( 2018 ), work directly with the L2-norm between prototypes and samples. Therefore, using a distribution such as the Normal distribution represents a principled generalization. This has several advantages. First, as demonstrated in Table 5 , using a distribution allows fitting additional parameters ( \(\sigma \) ’s) and this has a beneficial impact in reducing the prediction error. Still, as discussed in Sect. 3.3 , the significance of a prototype \(\textbf{p}\) in predicting a property \(\textbf{x}\) reduces to the Euclidean (or scaled Euclidean) distance between \(\textbf{p}\) and \(\textbf{x}\) when the covariance is diagonal. A second reason is that our keras implementation leverages the tensorflow_probability Python module, which supports Normal distributions. All in all, the choice of distribution aligns with common practice. However, it is important to note that our model is not limited to Normal distributions and can accommodate other types of unimodal distributions for the prototypes.

Explainability

Finally, it is also worth pointing this method could be considered a XAI method, because the prototypes and values can be thought of as a summary of the dataset and guide the explanation of predictions. This was already noted in a computer vision task (Li et al., 2018 ). While in real estate studies XAI techniques have also been used, we are not aware of previous prototype-based models—which are especially attractive for this scenario.

6 Practical and research implications

While the text has made reference to various practical implications of the model, they can be explicitly summarized as follows. First, the proposed model allows firms to perform a more accurate prediction of real estate properties. Consequently, firms can make better-informed investment decisions, optimize pricing strategies, and improve customer satisfaction by providing more accurate and understandable property valuations. However, where the model excels is in providing explanations in the forms of direct comparisons. Users of such a model can easily understand the rationale for a predicted price, on the basis of comparing it with a nearby “prototype”, which could drastically change the relation in which real estate agents, and other stakeholders, interact with ML systems. Moreover, the prototypes can be used as “new data” to enrich operations of certain firms.

Beyond industry players, the model can also benefit governments and social actors by providing insights into the factors driving price dynamics. This understanding can aid in the development of social impact initiatives and policies aimed at addressing housing affordability and other real estate-related social issues. Thus, the model not only promises significant commercial benefits but also holds potential for broader societal impact.

From a research standpoint, this work introduces novel perspectives on interpreting machine learning models for real estate predictions. Traditionally, explainable artificial intelligence (XAI) approaches have concentrated on determining the relevance of each variable, as seen in studies by  Lorenz et al. ( 2022 ) and  Potrawa and Tetereva ( 2022 ). Even recent works, such as those by  Bauer et al. ( 2023 ) and  Holstein et al. ( 2023 ), have explored how real estate professionals utilize model information, still with a primary focus on feature importance but lacking other cues such as proximity or similarity, used traditionally by real estate agents. The present work unveils this gap and demands for more intuitive and human-centric approaches in the real estate price prediction, with the aim of contributing to the transformation of this research field.

7 Conclusions and future perspectives

This article has introduced the application of prototype-based models in real estate valuation. Our research started with four desiderata that a model should ideally exhibit for real estate estimation: (a) non-linearity, (b) parametric learning using a loss function, (c) explainability, and (d) ability to implement the “direct comparison” criterion. Experimental results support that the proposed model has the four characteristics simultaneously, which was not the case of other machine learning models. Moreover, this is attained without sacrificing predictive accuracy: experimental results indicate that the prediction error obtained by the model is significantly lower than decision trees, random forests or KNN, and on par with neural networks (remarking, again, that the latter does not comply with the four characteristics discussed above). We also tried to highlight the usefulness of some of these characteristics, such as the capability to summarize a dataset with a reduced set of prototypes and the possibility to draw these prototypes on a map. We hope the method will be useful to operations research academics and practitioners which have shown previous interest in real estate valuation, as well as for the specific operations of some firms.

Future lines of research include using more complex form of the prototypes and values. In the present study, the prototypes are Normal distributions and the values are constant and we could explore other distributions or non-constant values. Perhaps if we could perform variable selection within the model we would obtain an even higher degree of explainability. Finally, we also acknowledge that there could be some use cases where researchers or practitioners are limited to the use of linear models (e.g. when they perform inference or work in an industry constrained by heavy regulation, such as banking or insurance). In such a situation, we could investigate if we could use the prototype-based model as a “feature engineering” method, where for example the obtained values of \(\gamma \) (Eq. 2 ) are used as additional location-related variables in the linear models.

A simplification used in some cases is to assume the same value of the standard deviation for all variables (isotropic covariance) and all prototypes, in which we will refer to the standard deviations as the scalar \(\sigma \) . Moreover, in that simplified case, \(\sigma \) can be fixed a priori and treated as hyperparameter (which sometimes improves the numerical stability of the model).

Because, by definition, \(\sum _m \gamma _m(\textbf{x})=1\) , Eq. 3 can be interpreted as a weighted average.

According to our previous definition \(\textbf{h} = g(\textbf{x})\) , in this simple case of 1 hidden layer, g is simply \(g(\textbf{x}) = \alpha (\textbf{W}\textbf{x}\) ), where W denotes a matrix of weights and \(\alpha \) an activation function.

Publicly available at https://geodacenter.github.io/data-and-lab/KingCounty-HouseSales2015/

The code is structured as follows: model and examples are available in https://github.com/prof-jose/prototype_based_learning and the experimental benchmark in https://github.com/prof-jose/prototype_based_learning .

For instance, consider the K-Means method and property #4607 from the test set, which shows the largest error difference between the location-only model and the model with all features. This property has normalized coordinates (−1.62, −1.05) and a normalized price of 0.118. The nearest prototype in the location-only model is at location (−1.81, −1.05) with a value of \(v_9\) =0.149. In the model with all features, the closest prototype is at (−0.55, −0.83), but the proximity has been driven by very similar values of the “waterfront”, “floors” and “bathrooms” variables (and clearly not location). However, this prototype’s normalized value is \(v'_5=0.44\) , which deviates significantly from the actual price. This is an example where similarity in specific variables does not necessarily imply a price similarity. The K-Means algorithm creates clusters (prototypes) without considering variable importance or property prices, leading to errors for properties like #4607 in the all-features model, which we verify is not an isolated case.

Note this refers to a standard implementation of a neural network, not to the custom neural network discussed in 3.4 .

Alexandridis, A. K., Karlis, D., Papastamos, D., & Andritsos, D. (2019). Real estate valuation and forecasting in non-homogeneous markets: A case study in Greece during the financial crisis. Journal of the Operational Research Society, 70 (10), 1769–1783.

Article   Google Scholar  

Amédée-Manesme, C. O., & Barthélémy, F. (2018). Ex-ante real estate value at risk calculation method. Annals of Operations Research, 262 , 257–285.

Atack, J., & Margo, R. A. (1998). Location, location, location! the price gradient for vacant urban land: New York, 1835 to 1900. The Journal of Real Estate Finance and Economics, 16 (2), 151–172.

Bauer, K., von Zahn, M., & Hinz, O. (2023). Expl (AI) ned: The impact of explainable artificial intelligence on users’ information processing. Information systems research, 34 (4), 1582–1602.

Baydin, A. G., Pearlmutter, B. A., Radul, A. A., & Siskind, J. M. (2018). Automatic differentiation in machine learning: A survey. Journal of Machine Learning Research, 18 , 1–43.

Google Scholar  

Biehl, M., Hammer, B., & Villmann, T. (2016). Prototype-based models in machine learning. Wiley Interdisciplinary Reviews: Cognitive Science, 7 (2), 92–111.

Bishop, C. M. (2006). Pattern recognition and machine learning . Berlin: Springer.

Bitter, C., Mulligan, G. F., & Dall’erba, S. (2007). Incorporating spatial variation in housing attribute prices: A comparison of geographically weighted regression and the spatial expansion method. Journal of Geographical Systems , 9 , 7–27.

Borst, R. A., & McCluskey, W. J. (2008). Using geographically weighted regression to detect housing submarkets: Modeling large-scale spatial variations in value. Journal of Property Tax Assessment & Administration, 5 (1), 21–54.

Breiman, L. (2001). Random forests. Machine Learning, 45 , 5–32.

Chen, S., & Xie, W. (2022). On cluster-aware supervised learning: Frameworks, convergent algorithms, and applications. INFORMS Journal on Computing, 34 (1), 481–502.

Choy, L. H., & Ho, W. K. (2023). The use of machine learning in real estate research. Land, 12 (4), 740.

Colwell, P. F., & Dilmore, G. (1999). Who was first? An examination of an early hedonic study. Land Economics , 620–626.

Court, A. (1939). The dynamics of automobile demand . General Motors: Chapter Hedonic price indexes with automotive examples.

d’Amato, M., & Kauko, T. (2017). Advances in automated valuation modeling. Springer International Publishing AG . 10: 978-3.

Doumpos, M., Papastamos, D., Andritsos, D., & Zopounidis, C. (2021). Developing automated valuation models for estimating property values: A comparison of global and locally weighted approaches. Annals of Operations Research, 306 , 415–433.

Fotheringham, A. S., Brunsdon, C., & Charlton, M. (2003). Geographically weighted regression: The analysis of spatially varying relationships . Wiley.

Gabrielli, L., Ruggeri, A. G., & Scarpa, M. (2023). Location, location, location: Fluctuations in real estate market values after covid-19 and the war in Ukraine based on econometric and spatial analysis, random forest, and multivariate regression. Land, 12 (6), 1248.

Gibbs, C., Guttentag, D., Gretzel, U., Morton, J., & Goodwill, A. (2018). Pricing in the sharing economy: A hedonic pricing model applied to airbnb listings. Journal of Travel & Tourism Marketing, 35 (1), 46–56.

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning . MIT Press.

Goodman, A. C. (1998). Andrew court and the invention of hedonic price analysis. Journal of Urban Economics, 44 (2), 291–298.

Goodman, A. C., & Thibodeau, T. G. (2007). The spatial proximity of metropolitan area housing submarkets. Real Estate Economics, 35 (2), 209–232.

Haas, G. C. (1922). A statistical analysis of farm sales in blue earth county, Minnesota, as a basis for farm land appraisal. Technical report.

Holstein, K., De-Arteaga, M., Tumati, L., & Cheng, Y. (2023). Toward supporting perceptual complementarity in human-ai collaboration via reflection on unobservables. Proceedings of the ACM on Human-Computer Interaction, 7 (CSCW1), 1–20.

Hong, J., Choi, H., & Kim, W. S. (2020). A house price valuation based on the random forest approach: The mass appraisal of residential property in South Korea. International Journal of Strategic Property Management , 24 (3), 140–152.

Hussain, T., Abbas, J., Wei, Z., & Nurunnabi, M. (2019). The effect of sustainable urban planning and slum disamenity on the value of neighboring residential property: Application of the hedonic pricing model in rent price appraisal. Sustainability   11 (4). https://doi.org/10.3390/su11041144 .

Isakson, H. R. (1988). Valuation analysis of commercial real estate using the nearest neighbors appraisal technique. Growth and Change, 19 (2), 11–24.

James, G., Witten, D., Hastie, T., Tibshirani, R., et al. (2013). An introduction to statistical learning , (vol. 112). Springer.

Kaden, M., Bohnsack, K. S., Weber, M., Kudła, M., Gutowska, K., Blazewicz, J., & Villmann, T. (2022). Learning vector quantization as an interpretable classifier for the detection of sars-cov-2 types based on their rna sequences. Neural Computing and Applications, 34 (1), 67–78.

Kok, N., Koponen, E. L., & Martínez-Barbosa, C. A. (2017). Big data in real estate? From manual appraisal to automated valuation. The Journal of Portfolio Management, 43 (6), 202–211.

Kramer, O., & Kramer, O. (2013). K-nearest neighbors. Dimensionality reduction with unsupervised nearest neighbors , pp. 13–23 .

Li, O., Liu, H., Chen, C., & Rudin, C. (2018). Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. In Proceedings of the AAAI conference on artificial intelligence , vol. 32.

Lins, M. P. E., Novaes, L. F. D. L., Legey, L. F. L. (2005). Real estate appraisal: A double perspective data envelopment analysis approach. Annals of Operations Research , 138 , 79–96.

Lorenz, F., Willwersch, J., Cajias, M., & Fuerst, F. (2022). Interpretable machine learning for real estate market analysis. Real Estate Economics .

Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S. I. (2020). From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2 (1), 56–67.

Malpezzi, S., et al. (2003). Hedonic pricing models: A selective and applied review. Housing Economics and Public Policy, 1 , 67–89.

McCluskey, W. J., Zulkarnain Daud, D., & Kamarudin, N. (2014). Boosted regression trees: An application for the mass appraisal of residential property in Malaysia. Journal of Financial Management of Property and Construction , 19 (2), 152–167.

McCluskey, W. J., McCord, M., Davis, P. T., Haran, M., & McIlhatton, D. (2013). Prediction accuracy in mass appraisal: A comparison of modern approaches. Journal of Property Research, 30 (4), 239–265.

Ottensmann, J. R., Payton, S., & Man, J. (2008). Urban location and housing prices within a hedonic model. Journal of Regional Analysis and Policy ,  38 (1) .

Owomugisha, G., Melchert, F., Mwebaze, E., Quinn, J. A., & Biehl, M. (2021). Matrix relevance learning from spectral data for diagnosing cassava diseases. IEEE Access, 9 , 83355–83363.

Pagourtzi, E., Assimakopoulos, V., Hatzichristos, T., & French, N. (2003). Real estate appraisal: A review of valuation methods. Journal of Property Investment & Finance, 21 (4), 383–401.

Peterson, S., & Flanagan, A. (2009). Neural network hedonic pricing models in mass real estate appraisal. Journal of Real Estate Research, 31 (2), 147–164.

Potrawa, T., & Tetereva, A. (2022). How much is the view from the window worth? machine learning-driven hedonic pricing model of the real estate market. Journal of Business Research, 144 , 50–65.

Rey-Blanco, D., Arbués, P., López, F. A., & Páez, A. (2024). Using machine learning to identify spatial market segments. A reproducible study of major Spanish markets. Environment and Planning B: Urban Analytics and City Science ,  51 (1): 89–108 .

Tchuente, D., & Nyawa, S. (2022). Real estate price estimation in French cities using geocoding and machine learning. Annals of Operations Research , 1–38 .

Tchuente, D., Lonlac, J., & Kamsu-Foguem, B. (2024). A methodological and theoretical framework for implementing explainable artificial intelligence (xai) in business applications. Computers in Industry, 155 , 104044.

Thibodeau, T. G. (2003). Marking single-family property values to market. Real Estate Economics, 31 (1), 1–22.

van Veen, R., Meles, S. K., Renken, R. J., Reesink, F. E., Oertel, W. H., Janzen, A., De Vries, G. J., Leenders, K. L., & Biehl, M. (2022). Fdg-pet combined with learning vector quantization allows classification of neurodegenerative diseases and reveals the trajectory of idiopathic rem sleep behavior disorder. Computer Methods and Programs in Biomedicine, 225 , 107042.

Zurada, J., Levitan, A., & Guan, J. (2011). A comparison of regression and artificial intelligence methods in a mass appraisal context. Journal of Real Estate Research, 33 (3), 349–388.

Download references

Author information

Authors and affiliations.

Department of Operations, Innovation and Data Sciences, Universitat Ramon Llull Esade, Barcelona, Spain

Jose A. Rodriguez-Serrano

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jose A. Rodriguez-Serrano .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Rodriguez-Serrano, J.A. Prototype-based learning for real estate valuation: a machine learning model that explains prices. Ann Oper Res (2024). https://doi.org/10.1007/s10479-024-06273-1

Download citation

Received : 22 December 2023

Accepted : 03 September 2024

Published : 23 September 2024

DOI : https://doi.org/10.1007/s10479-024-06273-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Real estate valuation
• Non-linear regression
• Machine learning
• Prototype-based models
• Find a journal
• Publish with us
• Track your research