The Learning Scientists

Mar 8 Different Research Methods: Strengths and Weaknesses

By Megan Sumeracki

Image from Pixabay

Image from Pixabay

There are a lot of different methods of conducting research, and each comes with its own set of strengths and weaknesses. I've been thinking a lot about the various research approaches because I'm teaching a senior-level research methods class with a lab this spring. This has led me to think a lot about how these different research methodologies might work together. While most researchers are exposed to a variety of methodologies throughout graduate training, we tend to become engrossed with our own specialty. This makes sense, at least to me, as there are so many nuances that it can take years to become truly proficient in conducting research in our own areas. Specialization seems necessary; however, this is exactly why effective communication and collaboration is key. Given the strengths and weaknesses of different methodologies, a mixed method approach can be used to balance these strengths and weaknesses.

We have said many times before that "it takes a village" and open communication to solve large problems. When it comes to student learning, I feel strongly that it takes a diverse group of experts from different research backgrounds and various experiences teaching in schools. With the amount of time and dedication that it takes to become an expert researcher and an expert teacher, it would be hard for one person to become both! The same is true for research methodologies. There are pros and cons to each, and science is best served when we combine our efforts and tackle our questions from many different directions.

Image from Pixabay

In this spirit, in today's blog I am writing about the general research methodologies that might be used to help us understand student learning. For each methodology, I describe what it is and how it might be used, as well as the strengths and weaknesses of the approach. This blog is a bit longer than our typical blogs because I'm tackling some big topics, but hopefully you'll find the discussion of various research methodologies, together in one place, as important as I do!

Descriptive Research

The main purpose of descriptive research is exactly what it sounds like it should be: to describe what is going on. There are a lot of individual approaches that fall under the descriptive research umbrella. Here are a few:

Case studies are a very in-depth analysis of an individual person, small group of people, or even an event. A researcher might conduct a case study on an individual who has a specific learning disability, or on a classroom that is engaging in a particular mode of instruction. 

Observation research involves sitting back (so to speak) and watching how individuals interact in natural environments. A researcher might (with permission from the school and parents of the children, of course) watch a group of preschoolers through a 2-way mirror to see how the children interact with one another. There is also a special type of observation research called participatory observation . This method is used when it would be difficult or impossible to simply watch from a distance. You can think of this as going under cover, where the researcher joins a group to learn about the group. A classic example involves a researcher, Leon Festinger, who joined a cult who believed the world was going to be destroyed by a flood in the 1950s. From this work, Festinger proposed Cognitive Dissonance Theory (to read more, check out this page ).

Survey research is considered descriptive research. In this work, the researcher compiles a set of questions and asks people to answer these questions. The types of questions can vary. Some surveys might people to rate their feelings or beliefs on a scale from 1-7 (also known as a "Likert" scale) or answer yes-no questions. Some surveys might ask more open-ended questions, and there are many that utilize a mix of these types of questions. If the researcher is asking a lot of open-ended questions, then we might call the research an interview , or a focus group if there are a few people discussing a topic and answering questions in a group. In this research, the participants may actually be guiding the direction of the research.

There is another important distinction to be made under the descriptive research umbrella: quantitative research vs. qualitative research . In quantitative research, data is collected in the forms of numbers. If a researcher asks a student to indicate on a scale from 1-10 how much they think they will remember from a lesson, then we are quantifying the student's perception of their own learning. In qualitative research, words are collected, and sometimes those words might be quantified in some way to use for statistical analysis. If a researcher asks a student to describe their learning process, or conducts in-depth interviews with teachers about classroom learning, then we are dealing with qualitative research.

Descriptive research can provide an in-depth view of any topic we might want to study, and the level of detail that we can find in descriptive research is extremely valuable. This is particularly true of descriptive research that is collected qualitatively. In this form of research, we may find information that we never even knew to look for! This type of research can be used to create new research questions, or form hypotheses about cause and effect relationships (though we cannot determine cause and effect from this research alone). Observation research has an added benefit of allowing us to see how things work in their natural environments. 


We cannot determine a cause and effect relationship from descriptive research. For example, if a student talks about engaging with a particular learning strategy, and then provides an in-depth account of why they think it helped them learn, we cannot conclude that this strategy actually did help the student learn.

We also have to be very careful of reactivity in this type of research. Sometimes, people (and animals too) change their behavior if they know they're being observed. Similarly, in surveys we have to worry about participants providing responses that are considered desirable or in line with social norms. (For example, if a parent is asked, "did you ever smoke while pregnant with your child?" we have to worry about parents saying "no, never" because that is the more desirable answer, or the one that aligns with social norms.)

Correlational Research

Correlational studies involve measuring two or more variables. For that reason, this research is inherently quantitative. The researchers can then look at how related to variables are to one another. If two variables are related, or correlated, then we can use one variable to predict the value of another variable. The greater the correlation, the greater accuracy our prediction will have. For example, correlational research might be able to tell us what factors at home are related to greater student learning in the classroom. These factors might include things like eating a healthy breakfast, getting enough sleep, having access to a lot of books, feeling safe, etc. 

I often have my students think about car insurance to explain correlational research. Car insurance companies measure a lot of different variables, and then try to do their best to predict which customers are likely to cost them the most money (e.g., cause a car accident, have their car damaged, etc.). They know that on average younger males are more likely to cost them money, and that drivers who have received speeding tickets are more likely to cost them money. They also know that people living in certain areas are more likely to get into car accidents due to dense populations, or to have their car damaged while parked. Does this mean that a 16-year old boy who got a speeding ticket and lives in the city is definitely going to cause a car accident? No, of course not. Does this mean that getting a speeding ticket specifically causes later car accidents? No. It just means that the car insurance company knows that this type of person is more likely to cause the car accident, for any number of reasons, and uses this information to determine premiums. 

Correlational research can help us understand the complex relationships between a lot of different variables. If we measure these variables in realistic settings, then we can learn more about how the world really works. This type of research allows us to make predictions, and can tell us if two variables are not related, and thus searching for a cause-effect relationship between the two is a huge waste of time.

Correlation is not the same as causation! Even if two variables are related to one another, that does not mean we can say for certain how the cause and effect relationship works. Take caffeine average consumption and average test. Lets say we find that the two are correlated, where increased caffeine is related to higher test performance. We cannot say that caffeine caused greater test performance, or that greater test performance caused greater caffeine consumption. In reality, either of those could work! For example, students may drink more caffeine and this might lead them to perform better on tests. Or, the students who perform better on tests are then more likely to drink more caffeine. A third variable could be related to both of these as well! It could be that students who are more concerned about their grades might study more and achieve better test performance, and might also drink more caffeine to help them stay awake to study! We just don't know from the correlation alone, but knowing that the two variables are in some way related can be very useful information.

True Experiments

True experiments involve manipulating (or changing) one variable and then measuring another. There are a few things that are required in order for research to be considered a true experiment. First, we need to randomly assign students to different groups. This random assignment helps create equivalent groups from the beginning. Second, we need to change something (for example, the type of learning strategy) across the two groups, holding everything else as constant as possible. The key here is to make sure to isolate the thing we are changing, so that it is the only difference between the groups. We also need to make sure at least one of the groups serves as a control group, or a group that serves as a comparison. We need to make sure that the only thing being systematically changed is our manipulation. (Note, sometimes we can systematically manipulate multiple things at once, but these are more complicated designs.) Finally, we then measure learning across the different groups. If we find that our manipulation led to greater learning compared to the control group, and we made sure to conduct the experiment properly with random assignment and appropriate controls, then we can say that our manipulation caused learning. Taking the example from the correlational section, if we want to know whether drinking coffee increases test performance, then we need to randomly assign some students drink coffee and other students to drink a non-caffeinated beverage (the control) and then measure test performance. And then, we repeat to be more confident in our conclusions! Usually, we're repeating experiments with little changes to continue obtaining new information.

Experiments can also be conducted in a “within-subjects” design. This means that each individual participating in the experiment is serving as their own control. In these experiments, each person participates in all of the conditions. To make sure that the order of conditions or materials are not affecting the results, the researcher randomizes the order of conditions and materials in a process called counterbalancing. The researcher then randomly assigns different participants to different versions of the experiment, with the conditions coming up in different orders. There are a number of ways to implement counterbalancing to maintain control in an experiment so that researchers can identify cause and effect relationships. The specifics of how to do this are not important for our purposes here. The important thing to note is that, even when participants are in within-subjects experiments and are participating in multiple learning conditions, in order to determine cause and effect we still need to maintain control and rule out alternate explanations for any findings (e.g., order or material effects).

This type of experiment allows us to determine cause and effect relationships! True experiments are often be designed based on descriptive research or correlational research to determine underlying causes. If we really want to know how to promote student learning in the classroom or at home, then we need to know what causes learning.

Of course, true experiments are not without weaknesses. True experiments require a lot of control so that we can isolate the variables that are causing changes to occur. The more control we have, the better measurement we have. However, at the same time, the more control we have, the more artificial the experiment becomes. Just because we that a learning strategy causes learning in one specific experiment, doesn't mean that it will work the same way with different types of students, or in live classroom settings. In other words, the effect might not be generalizable. The solution to this problem is to approach the question with a number of different experiments, and to include the other research approaches to get a better picture of what is going on. One way we've tried to do this in research about learning is to utilize the lab to classroom model.

Lab to Classroom Model:

What is one solution to the big weakness associated with true experiments? Do a bunch more experiments! Not just any experiments, of course, but experiments that, together, help combat the weaknesses described above. When we talk about the lab to classroom model*, we are talking primarily about true experiments. In the lab to classroom model, we start out with basic, highly controlled experiments in very artificial settings. This allows us to best determine cause and effect relationships. We then slowly work our way up to the more realistic setting. We lose control when we do this, and it is more difficult to determine cause and effect, but when we take all of the experimental evidence together we can be much more confident in our conclusions!

The Lab to Classroom Model

The Lab to Classroom Model

* If you're not familiar with the lab to classroom model,  see this blog for a brief description, or listen to this podcast to hear Yana and I talk about the model.

BONUS -- A Design to Avoid: Pre-test Post-test designs

Finally, there is one design that you might see pop up here and there, and it has so many problems that it's worth mentioning explicitly. Pre-test post-test designs are exactly like what they sound like: you measure something before an intervention and after the intervention, and compare. This is not a true experiment, and does not allow us to determine cause and effect relationships.

For example, you might (unfortunately) see someone provide students with a pre-test to assess prior knowledge, then implement some sort of learning strategy, and then provide students with a post-test to see how much they have learned (compared to the pre-test). This design is extremely problematic! We truly have no idea whether the learning intervention caused any learning in this case. It could be that the students just got better over time, or that they learned from taking the pre-test, or that because they knew they were being tested before and after they were more likely to study at home! We cannot say the learning strategy did anything for certain. Using pre-tests and post-tests in research is acceptable, but only if there is a control group for comparison!

While not all methodologies discussed in this blog allow us to determine cause and effect, but they have other strengths that go along with them. This design does not allow us to infer causality, nor does it give us the in-depth, detailed information we get from descriptive research, nor does it tell us the relationships among many different variables!

Weekly Digest #100: Teachers' Implementations Of Learning Strategies

Mar 11 Weekly Digest #100: Teachers' Implementations Of Learning Strategies

Weekly Digest #99: In Defense of Inquiry Learning

Mar 4 Weekly Digest #99: In Defense of Inquiry Learning

Source strengths & weaknesses

Contributor: Logan Miller

With the Internet’s ever-expanding repository of information, appraising potential sources has become an increasingly important skill. Evaluating strengths and weaknesses can be a useful tactic to assess potential sources.

What is the research question?

The research question defines the population to be investigated, interventions or exposures, and outcomes.

The population is too broad, too narrow, or specific characteristics are too vague such that possible findings will be inconclusive.

Was the study design appropriate for the research question?

The researchers chose to study the data over several years to ensure accuracy of the results for the selected population.

The researchers collected observational data when experimental data could have produced stronger support for their hypothesis.

Did the study address potential sources of bias?

The study asserts that the members of the sample were chosen randomly, ensuring they are representative of the population.

The study design contains a potential for the above selection bias, but the researchers didn’t address this potential in the study.

Was the study performed according to the original design?

The study followed the design and reported this how they followed the design. 

The researchers were unable to recruit the desired number of participants, reducing their ability to produce significant findings or answer the research question.

Do the data justify the conclusions?

The study accurately reports the statistical significance of the data without suggesting a cause and effect relationship between exposures and outcomes.

The researchers misinterpret a statistically significant correlation between the exposures and outcomes as a cause and effect relationship.

Are there any conflicts of interest?

While a party that might want to see specific findings funded the study, the researchers disclose that the party had no input into the study.

The researchers offer no disclosure on what input the funding party did or didn’t have into the study.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

52 11.2 Strengths and weaknesses of survey research

Learning objectives.

Survey research, as with all methods of data collection, comes with both strengths and weaknesses. We’ll examine both in this section.

Strengths of survey methods

  Researchers employing survey methods to collect data enjoy a number of benefits. First, surveys are an excellent way to gather lots of information from many people. In a study of older people’s experiences in the workplace, researchers were able to mail a written questionnaire to around 500 people who lived throughout the state of Maine at a cost of just over $1,000. This cost included printing copies of a seven-page survey, printing a cover letter, addressing and stuffing envelopes, mailing the survey, and buying return postage for the survey. I realize that $1,000 is nothing to sneeze at, but just imagine what it might have cost to visit each of those people individually to interview them in person. You would have to dedicate a few weeks of your life at least, drive around the state, and pay for meals and lodging to interview each person individually. We could double, triple, or even quadruple our costs pretty quickly by opting for an in-person method of data collection over a mailed survey. Thus, surveys are relatively cost-effective.

a cartoon of a person choosing between a happy, neutral, and sad face

Related to the benefit of cost-effectiveness is a survey’s potential for generalizability. Surveys allow researchers to collect data from very large samples for a relatively low cost, therefore survey methods lend themselves to the probability sampling techniques discussed in Chapter 10. Of all the data collection methods described in this textbook, survey research is probably best to use when the researcher wishes to gain a representative picture of the attitudes and characteristics of a large group.

Survey research also tends to be a reliable method of inquiry. This is because surveys are standardized in that the same questions, phrased in exactly the same way, are posed to participants. Other methods like qualitative interviewing, which we’ll learn about in Chapter 13, do not offer the same consistency that a quantitative survey offers. This is not to say that all surveys are always reliable. A poorly phrased question can cause respondents to interpret its meaning differently, which can reduce that question’s reliability. Assuming well-constructed questions and survey design, one strength of this methodology is its potential to produce reliable results.

The versatility of survey research is also an asset. Surveys are used by all kinds of people in all kinds of professions, which means that understanding how to construct and administer surveys is a useful skill to have. Lawyers might use surveys in their efforts to select juries. Social services and other organizations (e.g., churches, clubs, fundraising groups, activist groups) use them to evaluate the effectiveness of their efforts. Businesses utilize surveys to inform marketing strategies for their products. Governments use surveys to understand community opinions and needs. Politicians and media outlets use surveys to understand their constituencies.

In sum, the following are benefits of survey research:

Weaknesses of survey methods

As with all methods of data collection, survey research comes with a few drawbacks. While some may argue that surveys are flexible because researchers can ask many different questions on a plethora of topics, survey researchers are generally confined to a single instrument for collecting data, the questionnaire. Surveys are in many ways rather inflexible . Let’s say you mail a survey out to 1,000 people and then discover, as responses start coming in, that your phrasing on a particular question seems to be confusing a number of respondents. At this stage, it’s too late for a do-over or to change the question for the respondents who haven’t yet returned their surveys. When conducting in-depth interviews, on the other hand, a researcher can provide respondents further explanation if they’re confused by a question and can tweak their questions as they learn more about how respondents seem to understand them.

Depth can also be a problem with surveys. Survey questions are standardized; thus, it can be difficult to ask anything other than very general questions that a broad range of people will understand. Due to the general nature of questions, survey results may not be as valid as results obtained using other methods of data collection that allow a researcher to comprehensively examine the topic being studied. For example, let’s think back to the opening example of this chapter and say that you want to learn something about voters’ willingness to elect an African American president. General Social Survey respondents were asked, “If your party nominated an African American for president, would you vote for him if he were qualified for the job?” Respondents were then asked to respond either yes or no to the question. What if someone’s opinion was more complex than a simple yes or no? What if, for example, a person was willing to vote for an African American woman but not an African American man? [1]

In sum, potential drawbacks to survey research include the following:

Key Takeaways

Image attributions

experience by mohamed_hassan CC-0

Scientific Inquiry in Social Work by Matthew DeCarlo is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book


  1. Images Of Strengths

    strengths and weaknesses of a research article

  2. Strengths and Weaknesses Essay

    strengths and weaknesses of a research article

  3. Strengths and weaknesses of field and laboratory experiments.

    strengths and weaknesses of a research article

  4. Selected Possible Generic Strengths and Weaknesses of PPAs and Surveys

    strengths and weaknesses of a research article

  5. Strengths and Weaknesses

    strengths and weaknesses of a research article

  6. 😀 What are your strengths and weaknesses in english. Essay about My Strengths and Weaknesses as

    strengths and weaknesses of a research article


  1. Practical Research 2,Chapter 1,Lesson 1(NATURE OF INQUIRY AND RESEARCH)

  2. Choosing a Research Topic

  3. Reputational Security: The Imperative to Reinvest in U.S. Strategic Communications Capabilities

  4. Integrated Method of Research Design

  5. Selection of Research Topic

  6. How to Choose a Research Topic


  1. Different Research Methods: Strengths and Weaknesses

    A researcher might conduct a case study on an individual who has a specific learning disability, or on a classroom that is engaging in a

  2. Guidelines for Critiquing a Quantitative Research Study

    Below are guidelines for critiquing research articles. Critiques include an analysis of both strengths and weaknesses or limitations of an article.

  3. strengthsweaknessesofresearch.doc

    ... to help you pinpoint strengths and weaknesses of a researcher's argument: ... If not, would changing the assumptions change the outcome of the study?


    Discuss the strength and weakness of the articles assumptions. ▫ Identify the article's audience ... the degree of further research required, and the.

  5. Source Strengths & Weaknesses

    What is the research question? · Was the study design appropriate for the research question? · Did the study address potential sources of bias?

  6. 52. 11.2 Strengths and weaknesses of survey research

    Key Takeaways · Strengths of survey research include its cost effectiveness, generalizability, reliability, and versatility. · Weaknesses of survey research

  7. Research papers and review considerations

    Key strengths of the paper. Again, where you think the value of the paper lies. • Key weaknesses of the paper. What would you do differently? •

  8. The Strengths and Weaknesses of Research Methodology

    Abstract: The purpose of this study is compared strengths and weaknesses of qualitative and quantitative research methodologies in social science fields.

  9. Critical Reviews of Journal Articles

    A critical review of a journal article is an evaluation of an article's strengths, weaknesses and validity. It is used to inform readers of an article's

  10. Weaknesses of Research

    Weaknesses of Research. The purpose of this page is to describe weaknesses of Research. "Weaknesses related to the Research Problem:.