←
Chapter page 6 / 382 Study design
→
English

2  Study design

Before digging into the details of working with data, we stop to think about how data come to be. That is, if the data are to be used to make broad and complete conclusions, then it is important to understand who or what the data represent. One important aspect of data provenance is sampling. Knowing how the observational units were selected from a larger entity will allow for generalizations back to the population from which the data were randomly selected. Additionally, by understanding the structure of the study, causal relationships can be separated from those relationships which are only associated. A good question to ask oneself before working with the data at all is, “How were these observations collected?”. You will learn a lot about the data by understanding its source.

2.1 Sampling principles and strategies

The first step in conducting research is to identify topics or questions that are to be investigated. A clearly laid out research question is helpful in identifying what subjects or cases should be studied and what variables are important. It is also important to consider how data are collected so that the data are reliable and help achieve the research goals.

2.1.1 Populations and samples

Consider the following three research questions:

  1. What is the average mercury content in swordfish in the Atlantic Ocean?
  2. Over the last five years, what is the average time to complete a degree for Duke undergrads?
  3. Does a new drug reduce the number of deaths in patients with severe heart disease?

Each research question refers to a target population. In the first question, the target population is all swordfish in the Atlantic Ocean, and each fish represents a case. Oftentimes, it is not feasible to collect data for every case in a population. Collecting data for an entire population is called a census. A census is difficult because it is too expensive to collect data for the entire population, but it might also be because it is difficult or impossible to identify the entire population of interest! Instead, a sample is taken. A sample is the data you have. Ideally, a sample is a small fraction of the population. For instance, 60 swordfish (or some other number) in the population might be selected, and this sample data may be used to provide an estimate of the population average and to answer the research question.

For the second and third questions above, identify the target population and what represents an individual case.1

2.1.2 Parameters and statistics

In most statistical analysis procedures, the research question at hand boils down to understanding a numerical summary. The number (or set of numbers) may be a quantity you are already familiar with (like the average) or it may be something you learn through this text (like the slope and intercept from a least squares model, provided in Section 7.2).

A numerical summary can be calculated on either the sample of observations or the entire population. However, measuring every unit in the population is usually prohibitive. So, a “typical” numerical summary is calculated from a sample. Yet, we can still conceptualize calculating the average income of all adults in Argentina.

We use specific terms in order to differentiate when a number is being calculated on a sample of data (sample statistic) and when it is being calculated or considered for calculation on the entire population (population parameter). The terms statistic and parameter are useful for communicating claims and models and will be used extensively in later chapters which delve into making inference on populations.

2.1.3 Anecdotal evidence

Consider the following possible responses to the three research questions:

  1. A man on the news got mercury poisoning from eating swordfish, so the average mercury concentration in swordfish must be dangerously high.
  2. I met two students who took more than 7 years to graduate from Duke, so it must take longer to graduate at Duke than at many other colleges.
  3. My friend’s dad had a heart attack and died after they gave him a new heart disease drug, so the drug must not work.

Each conclusion is based on data. However, there are two problems. First, the data only represent one or two cases. Second, and more importantly, it is unclear whether these cases are actually representative of the population. Data collected in this haphazard fashion are called anecdotal evidence.

Anecdotal evidence.

Be careful of data collected in a haphazard fashion. Such evidence may be true and verifiable, but it may only represent extraordinary cases and therefore not be a good representation of the population.

Anecdotal evidence typically is composed of unusual cases that we recall based on their striking characteristics. For instance, we are more likely to remember the two people we met who took 7 years to graduate than the six others who graduated in four years. Instead, of looking at the most unusual cases, we should examine a sample of many cases that better represent the population.

2.1.4 Sampling from a population

We might try to estimate the time to graduation for Duke undergraduates in the last five years by collecting a sample of graduates. All graduates in the last five years represent the population, and graduates who are selected for review are collectively called the sample. In general, we always seek to randomly select a sample from a population. The most basic type of random selection is equivalent to how raffles are conducted. For example, in selecting graduates, we could write each graduate’s name on a raffle ticket and draw 10 tickets. The selected names would represent a random sample of 10 graduates.

A large circle contains many dots which indicate all the graduates. A smaller circle contains a few of the dots (i.e., graduates) which have been randomly selected from the larger circle.
Figure 2.1: 10 graduates are randomly selected from the population to be included in the sample.

Suppose we ask a student who happens to be majoring in nutrition to select several graduates for the study. Which students do you think they might pick? Do you think their sample would be representative of all graduates?

They might pick a disproportionate number of graduates from health-related fields, as shown in Figure 2.2. When selecting samples by hand, we run the risk of picking a biased sample, even if our bias is unintended.

A large circle contains many dots which indicate all the graduates, but some of the dots have been greyed out where others are dark dots from which the sample is taken. A smaller circle contains a few of the dots (i.e., graduates) which have been selected from the biased group of dark dots in the large circle.
Figure 2.2: Asked to pick a sample of graduates, a nutrition major might inadvertently pick a disproportionate number of graduates from health-related majors.

If someone was permitted to pick and choose exactly which graduates were included in the sample, it is entirely possible that the sample would overrepresent that person’s interests, which may be entirely unintentional. This introduces bias into a sample. Sampling randomly helps address this problem. The most basic random sample is called a simple random sample and is equivalent to drawing names out of a hat to select cases. This means that each case in the population has an equal chance of being included and the cases in the sample are not related to each other.

The act of taking a simple random sample helps minimize bias. However, bias can crop up in other ways. Even when people are picked at random, e.g., for surveys, caution must be exercised if the non-response rate is high. For instance, if only 30% of the people randomly sampled for a survey actually respond, then it is unclear whether the results are representative of the entire population. This non-response bias can skew results.

A large circle contains many dots which indicate the population of interest, but some of the dots have been greyed out where others are dark dots from which the sample is taken (where the grey dots are potentially due to non-response bias). A smaller circle contains a few of the dots which have been selected from the group of dark dots in the large circle who were individuals willing to respond to the survey.
Figure 2.3: Due to the possibility of non-response, survey studies may only reach a certain group within the population. It is difficult, and oftentimes impossible, to completely fix this problem.

Another common downfall is a convenience sample, where individuals who are easily accessible are more likely to be included in the sample. For instance, if a political survey is done by stopping people walking in the Bronx, this will not represent all of New York City. It is often difficult to discern what sub-population a convenience sample represents.

We can easily access ratings for products, sellers, and companies through websites. These ratings are based only on those people who go out of their way to provide a rating. If 50% of online reviews for a product are negative, do you think this means that 50% of buyers are dissatisfied with the product? Why?2

2.1.5 Four sampling methods

Almost all statistical methods are based on the notion of implied randomness. If data are not collected in a random framework from a population, these statistical methods – the estimates and errors associated with the estimates – are not reliable. Here we consider four random sampling techniques: simple, stratified, cluster, and multistage sampling. Figure 2.4 and Figure 2.5 provide graphical representations of these techniques.

Simple random sampling is probably the most intuitive form of random sampling. Consider the salaries of Major League Baseball (MLB) players, where each player is a member of one of the league’s 30 teams. To take a simple random sample of 120 baseball players and their salaries, we could write the names of that season’s several hundreds of players onto slips of paper, drop the slips into a bucket, shake the bucket around until we are sure the names are all mixed up, then draw out slips until we have the sample of 120 players. In general, a sample is referred to as “simple random” if each case in the population has an equal chance of being included in the final sample and knowing that a case is included in a sample does not provide useful information about which other cases are included.

The top box shows a population of dots (i.e., individuals) where a handful of the dots have been sampled randomly. The bottom box shows the same population of dots but grouped in such a way that there are six strata. From each stratum three dots (i.e., individuals) are randomly selected.
Figure 2.4: Examples of simple random and stratified sampling. In the top panel, simple random sampling was used to randomly select the 18 cases (denoted in red). In the bottom panel, stratified sampling was used: cases were first grouped into strata, then simple random sampling was employed to randomly select 3 cases within each stratum.’

Stratified sampling is a divide-and-conquer sampling strategy. The population is divided into groups called strata. The strata are chosen so that similar cases are grouped together, then a second sampling method, usually simple random sampling, is employed within each stratum. In the baseball salary example, each of the 30 teams could represent a stratum, since some teams have a lot more money (up to 4 times as much!). Then we might randomly sample 4 players from each team for our sample of 120 players.

Stratified sampling is especially useful when the cases in each stratum are very similar with respect to the outcome of interest. The downside is that analyzing data from a stratified sample is a more complex task than analyzing data from a simple random sample. The analysis methods introduced in this book would need to be extended to analyze data collected using stratified sampling.

Why would it be good for cases within each stratum to be very similar?

We might get a more stable estimate for the subpopulation in a stratum if the cases are very similar, leading to more precise estimates within each group. When we combine these estimates into a single estimate for the full population, that population estimate will tend to be more precise since each individual group estimate is itself more precise.

In a cluster sample, we break up the population into many groups, called clusters. Then we sample a fixed number of clusters and include all observations from each of those clusters in the sample. A multistage sample is like a cluster sample, but rather than keeping all observations in each cluster, we would collect a random sample within each selected cluster.

In the top figure, dots are grouped into clusters, three clusters are selected, and every dot (i.e., all individuals) from each of the three clusters are sampled. In the bottom figure, dots are again grouped into clusters and three clusters are selected. However, random sampling is applied so that a random sample from each of the three selected clusters is taken.
Figure 2.5: Examples of cluster and multistage sampling. In the top panel, cluster sampling was used: data were binned into nine clusters, three of these clusters were sampled, and all observations within these three clusters were included in the sample. In the bottom panel, multistage sampling was used, which differs from cluster sampling only in that we randomly select a subset of each cluster to be included in the sample rather than measuring every case in each sampled cluster.’

Sometimes cluster or multistage sampling can be more economical than the alternative sampling techniques. Also, unlike stratified sampling, these approaches are most helpful when there is a lot of case-to-case variability within a cluster but the clusters themselves do not look very different from one another. For example, if neighborhoods represented clusters, then cluster or multistage sampling work best when the populations inside each neighborhood are very diverse. A downside of these methods is that more advanced techniques are typically required to analyze the data, though the methods in this book can be extended to handle such data.

Suppose we are interested in estimating the malaria rate in a densely tropical portion of rural Indonesia. We learn that there are 30 villages in that part of the Indonesian jungle, each more or less like the next, but the distances between the villages are substantial. We want to test 150 individuals for malaria. What sampling method should we use?

A simple random sample would likely draw individuals from all 30 villages, which could make data collection expensive. Stratified sampling would be a challenge since it is unclear how we would build strata of similar individuals. However, cluster sampling or multistage sampling seem like very good ideas. With multistage sampling, we could randomly select half of the villages, then randomly select 10 people from each. This could reduce data collection costs substantially in comparison to a simple random sample, and the cluster sample would still yield reliable information, even if we would need to analyze the data with more advanced methods than those introduced in this book.

2.2 Experiments

Studies where the researchers assign treatments to cases are called experiments. When this assignment includes randomization, e.g., using a coin flip to decide which treatment a patient receives, it is called a randomized experiment. Randomized experiments are fundamentally important when trying to show a causal connection between two variables.

2.2.1 Principles of experimental design

  1. Controlling. Researchers assign treatments to cases, and they do their best to control any other differences in the groups3. For example, when patients take a drug in pill form, some patients take the pill with only a sip of water while others may have it with an entire glass of water. To control for the effect of water consumption, a doctor may instruct every patient to drink a 12-ounce glass of water with the pill.
  1. Randomization. Researchers randomize patients into treatment groups to account for variables that cannot be controlled. For example, some patients may be more susceptible to a disease than others due to their dietary habits. In this example dietary habit is a confounding variable4, which is defined as a variable that is associated with both the explanatory and response variables. Randomizing patients into the treatment or control group helps even out such differences.

Confounding variable.

A confounding variable is one that is associated with both the explanatory and response variables. Because it is associated with both variables, it prevents the study from concluding that the explanatory variable caused the response variable. Consider a silly example with total ice-cream sales as the explanatory variable and number of boating accidents as the response variable (which may seem highly correlated). Outside temperature is associated with both variables, and therefore we cannot conclude that high ice-cream sales is a cause of more boating accidents.

Confounding variables may or may not be measured as part of the study. Regardless, drawing cause-and-effect conclusions is difficult in an observational study because of the ever-present possibility of confounding variables.

  1. Replication. The more cases researchers observe, the more accurately they can estimate the effect of the explanatory variable on the response. In a single study, we replicate by collecting a sufficiently large sample. What is considered sufficiently large varies from experiment to experiment, but at a minimum we want to have multiple subjects (experimental units) per treatment group. Another way of achieving replication is replicating an entire study to verify an earlier finding. The term replication crisis refers to the ongoing methodological crisis in which past findings from scientific studies in several disciplines have failed to be replicated. Pseudoreplication occurs when individual observations under different treatments are heavily dependent on each other. For example, suppose you have 50 subjects in an experiment where you’re taking blood pressure measurements at 10 time points throughout the course of the study. By the end, you will have 50 \(\times\) 10 = 500 measurements. Reporting that you have 500 observations would be considered pseudoreplication, as the blood pressure measurements of a given individual are not independent of each other. Pseudoreplication often happens when the wrong entity is replicated, and the reported sample sizes are exaggerated.

  1. Blocking. Researchers sometimes know or suspect that variables, other than the treatment, influence the response. Under these circumstances, they may first group individuals based on this variable into blocks and then randomize cases within each block to the treatment groups. This strategy is often referred to as blocking. For instance, if we are looking at the effect of a drug on heart attacks, we might first split patients in the study into low-risk and high-risk blocks, then randomly assign half the patients from each block to the control group and the other half to the treatment group, as shown in Figure 2.6. This strategy ensures that each treatment group has the same number of low-risk patients and the same number of high-risk patients.
Before randomly allocating, the red low risk patients and blue high risk patients are split into two separate groups. Subsequently, half of the red low risk patients are randomly chosen to receive the treatment, and half of the blue high risk patients are randomly chosen to receive the treatment.
Figure 2.6: Blocking for patient risk. Patients are first divided into low-risk and high-risk blocks, then patients in each block are evenly randomized into the treatment groups. This strategy ensures equal representation of patients in each treatment group from both risk categories.

It is important to incorporate the first three experimental design principles into any study, and this book describes applicable methods for analyzing data from such experiments. Blocking is a slightly more advanced technique, and statistical methods in this book may be extended to analyze data collected using blocking.

2.2.2 Reducing bias in human experiments

Randomized experiments have long been considered to be the gold standard for data collection, but they do not ensure an unbiased perspective into the cause-and-effect relationship in all cases. Human studies are perfect examples where bias can unintentionally arise. Here we reconsider a study where a new drug was used to treat heart attack patients. In particular, researchers wanted to know if the drug reduced deaths in patients.

These researchers designed a randomized experiment because they wanted to draw causal conclusions about the drug’s effect. Study volunteers5 were randomly placed into two study groups. One group, the treatment group, received the drug. The other group, called the control group, did not receive any drug treatment.

Put yourself in the place of a person in the study. If you are in the treatment group, you are given a fancy new drug that you anticipate will help you. On the other hand, a person in the other group does not receive the drug and sits idly, hoping her participation does not increase her risk of death. These perspectives suggest there are actually two effects in this study: the one of interest is the effectiveness of the drug, and the second is an emotional effect of (not) taking the drug, which is difficult to quantify.

Researchers aren’t usually interested in the emotional effect, which might bias the study. To circumvent this problem, researchers do not want patients to know which group they are in. When researchers keep the patients uninformed about their treatment, the study is said to be blind. But there is one problem: if a patient does not receive a treatment, they will know they’re in the control group. A solution to this problem is to give a fake treatment to patients in the control group. This is called a placebo, and an effective placebo is the key to making a study truly blind. A classic example of a placebo is a sugar pill that is made to look like the actual treatment pill. However, offering such a fake treatment may not be ethical in certain experiments. For example, in medical experiments, typically the control group must get the current standard of care. Oftentimes, a placebo results in a slight but real improvement in patients. This effect has been dubbed the placebo effect.

The patients are not the only ones who should be blinded: doctors and researchers can unintentionally bias a study. When a doctor knows a patient has been given the real treatment, they might inadvertently give that patient more attention or care than a patient that they know is on the placebo. To guard against this bias, which again has been found to have a measurable effect in some instances, most modern studies employ a double-blind setup where doctors or researchers who interact with patients are, just like the patients, unaware of who is or is not receiving the treatment.6

Look back to the study in Section 1.1 where researchers were testing whether stents were effective at reducing strokes in at-risk patients. Is this an experiment? Was the study blinded? Was it double-blinded?7

For the study in Section 1.1, could the researchers have employed a placebo? If so, what would that placebo have looked like?8

You may have many questions about the ethics of sham surgeries to create a placebo. These questions may have even arisen in your mind when in the general experiment context, where a possibly helpful treatment was withheld from individuals in the control group; the main difference is that a sham surgery tends to create additional risk, while withholding a treatment only maintains a person’s risk.

There are always multiple viewpoints of experiments and placebos, and rarely is it obvious which is ethically “correct”. For instance, is it ethical to use a sham surgery when it creates a risk to the patient? However, if we do not use sham surgeries, we may promote the use of a costly treatment that has no real effect; if this happens, money and other resources will be diverted away from other treatments that are known to be helpful. Ultimately, this is a difficult situation where we cannot perfectly protect both the patients who have volunteered for the study and the patients who may benefit (or not) from the treatment in the future.

2.3 Observational studies

Studies where no treatment has been explicitly applied (or explicitly withheld) are called observational studies. For instance, studies on the loan data and county data described in Section 1.2 are would both be considered observational, as they rely on observational data.

Making causal conclusions based on experiments is often reasonable, since we can randomly assign the explanatory variable(s), i.e., the treatments. However, making the same causal conclusions based on observational data can be treacherous and is not recommended. Thus, observational studies are generally only sufficient to show associations or form hypotheses that can be later checked with experiments.

Suppose an observational study tracked sunscreen use and skin cancer, and it was found that the more sunscreen someone used, the more likely the person was to have skin cancer. Does this mean sunscreen causes skin cancer?

No! Some previous research tells us that using sunscreen actually reduces skin cancer risk, so maybe there is another variable that can explain this hypothetical association between sunscreen usage and skin cancer, as shown in Figure 2.7. One important piece of information that is absent is sun exposure. If someone is out in the sun all day, they are more likely to use sunscreen and more likely to get skin cancer. Exposure to the sun is unaccounted for in the simple observational investigation.

Three boxes are shown in a triangle arrangement representing: sun exposure, using sunscreen, and skin cancer. A solid arrow connects sun exposure as a causal mechanism to using sunscreen; a solid arrow also connects sun exposure as a causal mechanism to skin cancer. A questioning arrow indicates that the causal effect of using sunscreen on skin cancer is unknown.
Figure 2.7: Sun exposure may be the root cause of both sunscreen use and skin cancer.

In this example, sun exposure is a confounding variable. The presence of confounding variables is what inhibits the ability for observational studies to make causal claims. While one method to justify making causal conclusions from observational studies is to exhaust the search for confounding variables, there is no guarantee that all confounding variables can be examined or measured.

Figure 1.2 shows a negative association between the homeownership rate and the percentage of housing units that are in multi-unit structures in a county. However, it is unreasonable to conclude that there is a causal relationship between the two variables. Suggest a variable that might explain the negative relationship.9

Observational studies come in two forms: prospective and retrospective studies. A prospective study identifies individuals and collects information as events unfold. For instance, medical researchers may identify and follow a group of patients over many years to assess the possible influences of behavior on cancer risk. One example of such a study is The Nurses’ Health Study. Started in 1976 and expanded in 1989, the Nurses’ Health Study has collected data on over 275,000 nurses and is still enrolling participants. This prospective study recruits registered nurses and then collects data from them using questionnaires. Retrospective studies collect data after events have taken place, e.g., researchers may review past events in medical records. Some datasets may contain both prospectively- and retrospectively collected variables, such as medical studies which gather information on participants’ lives before they enter the study and subsequently collect data on participants throughout the study.

2.4 Chapter review

2.4.1 Summary

A proficient analyst will have a good sense of the types of data they are working with and how to visualize the data in order to gain a complete understanding of the variables. Equally important, however, is the data source. In this chapter, we have discussed randomized experiments and taking good, random, representative samples from a population. When we discuss inferential methods (starting in Chapter 11), the conclusions that can be drawn will be dependent on how the data were collected. Figure 2.8 summarizes how sampling and assignment methods relate to the scope of inference.10 Regularly revisiting Figure 2.8 will be important when making conclusions from a given data analysis.

A two by two table describing the scenarios of random sample or not and random allocation or not. Selecting randomly from a population allows for generalization back to the population. Randomly allocating in an experiment allows for establishing causation.
Figure 2.8: Analysis conclusions should be made carefully according to how the data were collected. Very few datasets come from the top left box because usually ethics require that random assignment of treatments can only be given to volunteers. Both representative (ideally random) sampling and experiments (random assignment of treatments) are important for how statistical conclusions can be made on populations.

2.4.2 Terms

The terms introduced in this chapter are presented in Table 2.1. If you’re not sure what some of these terms mean, we recommend you go back in the text and review their definitions. You should be able to easily spot them as bolded text.

Table 2.1: Terms introduced in this chapter.
anecdotal evidence experiment replication
bias multistage sample replication crisis
blind non-response bias representative
blocking non-response rate retrospective study
census observational study sample
cluster placebo sample bias
cluster sampling placebo effect sample statistic
confounding variable population simple random sample
control population parameter simple random sampling
control group prospective study strata
convenience sample pseudoreplication stratified sampling
double-blind randomized experiment treatment group

2.5 Exercises

Answers to odd-numbered exercises can be found in Appendix A.2.

  1. Parameters and statistics. Identify which value represents the sample mean and which value represents the claimed population mean.

    1. American households spent an average of about $52 in 2007 on Halloween merchandise such as costumes, decorations and candy. To see if this number had changed, researchers conducted a new survey in 2008 before industry numbers were reported. The survey included 1,500 households and found that average Halloween spending was $58 per household.

    2. The average GPA of students in 2001 at a private university was 3.37. A survey on a sample of 203 students from this university yielded an average GPA of 3.59 a decade later.

  1. Sleeping in college. A recent article in a college newspaper stated that college students get an average of 5.5 hrs of sleep each night. A student who was skeptical about this value decided to conduct a survey by randomly sampling 25 students. On average, the sampled students slept 6.25 hours per night. Identify which value represents the sample mean and which value represents the claimed population mean.

  1. Air pollution and birth outcomes, scope of inference. Researchers collected data to examine the relationship between air pollutants and preterm births in Southern California. During the study air pollution levels were measured by air quality monitoring stations. Length of gestation data were collected on 143,196 births between the years 1989 and 1993, and air pollution exposure during gestation was calculated for each birth. (Ritz et al. 2000)

    1. Identify the population of interest and the sample in this study.

    2. Comment on whether the results of the study can be generalized to the population, and if the findings of the study can be used to establish causal relationships.

  1. Cheaters, scope of inference. Researchers studying the relationship between honesty, age and self-control conducted an experiment on 160 children between the ages of 5 and 15. The researchers asked each child to toss a fair coin in private and to record the outcome (white or black) on a paper sheet, and said they would only reward children who report white. Half the students were explicitly told not to cheat and the others were not given any explicit instructions. Differences were observed in the cheating rates in the instruction and no instruction groups, as well as some differences across children’s characteristics within each group. (Bucciol and Piovesan 2011)

    1. Identify the population of interest and the sample in this study.

    2. Comment on whether the results of the study can be generalized to the population, and if the findings of the study can be used to establish causal relationships.

  1. Gamification and statistics, scope of inference. Researchers investigating the effects of gamification (application of game-design elements and game principles in non-game contexts) on learning statistics randomly assigned 365 college students in a statistics course to one of four groups; one of these groups had no reading exercises and no gamification, one group had reading but no gamification, one group had gamification but no reading, and a final group had gamification and reading. Students in all groups also attended lectures. The study found that gamification had a positive impact on student learning compared to traditional teaching methods involving reading exercises. (Legaki et al. 2020)

    1. Identify the population of interest and the sample in this study.

    2. Comment on whether the results of the study can be generalized to the population, and if the findings of the study can be used to establish causal relationships.

  1. Stealers, scope of inference. In a study of the relationship between socio-economic class and unethical behavior, 129 University of California undergraduates at Berkeley were asked to identify themselves as having low or high social-class by comparing themselves to others with the most (least) money, most (least) education, and most (least) respected jobs. They were also presented with a jar of individually wrapped candies and informed that the candies were for children in a nearby laboratory, but that they could take some if they wanted. After completing some unrelated tasks, participants reported the number of candies they had taken. It was found that those who were identified as upper-class took more candy than others. (Piff et al. 2012)

    1. Identify the population of interest and the sample in this study.

    2. Comment on whether the results of the study can be generalized to the population, and if the findings of the study can be used to establish causal relationships.

  1. Relaxing after work. The General Social Survey asked the question, “After an average work day, about how many hours do you have to relax or pursue activities that you enjoy?” to a random sample of 1,155 Americans. The average relaxing time was found to be 1.65 hours. Determine which of the following is an observation, a variable, a sample statistic, or a population parameter.11

    1. An American in the sample.

    2. Number of hours spent relaxing after an average work day.

    3. 1.65.

    4. Average number of hours all Americans spend relaxing after an average work day.

  1. Cats on YouTube. Suppose you want to estimate the percentage of videos on YouTube that are cat videos. It is impossible for you to watch all videos on YouTube so you use a random video picker to select 1000 videos for you. You find that 2% of these videos are cat videos. Determine which of the following is an observation, a variable, a sample statistic, or a population parameter.

    1. Percentage of all videos on YouTube that are cat videos.

    2. 2%.

    3. A video in your sample.

    4. whether a video is a cat video.

  1. Course satisfaction across sections. A large college class has 160 students. All 160 students attend the lectures together, but the students are divided into 4 groups, each of 40 students, for lab sections administered by different teaching assistants. The professor wants to conduct a survey about how satisfied the students are with the course, and he believes that the lab section a student is in might affect the student’s overall satisfaction with the course.

    1. What type of study is this?

    2. Suggest a sampling strategy for carrying out this study.

  1. Housing proposal across dorms. On a large college campus first-year students and sophomores live in dorms located on the eastern part of the campus and juniors and seniors live in dorms located on the western part of the campus. Suppose you want to collect student opinions on a new housing structure the college administration is proposing and you want to make sure your survey equally represents opinions from students from all years.

    1. What type of study is this?

    2. Suggest a sampling strategy for carrying out this study.

  1. Internet use and life expectancy. The following scatterplot was created as part of a study evaluating the relationship between estimated life expectancy at birth (as of 2014) and percentage of internet users (as of 2009) in 208 countries for which such data were available.12

    1. Describe the relationship between life expectancy and percentage of internet users.

    2. What type of study is this?

    3. State a possible confounding variable that might explain this relationship and describe its potential effect.

  1. Stressed out. A study that surveyed a random sample of otherwise healthy high school students found that they are more likely to get muscle cramps when they are stressed. The study also noted that students drink more coffee and sleep less when they are stressed.

    1. What type of study is this?

    2. Can this study be used to conclude a causal relationship between increased stress and muscle cramps?

    3. State possible confounding variables that might explain the observed relationship between increased stress and muscle cramps.

  1. Evaluate sampling methods. A university wants to determine what fraction of its undergraduate student body support a new $25 annual fee to improve the student union. For each proposed method below, indicate whether the method is reasonable or not.

    1. Survey a simple random sample of 500 students.

    2. Stratify students by their field of study, then sample 10% of students from each stratum.

    3. Cluster students by their ages (e.g., 18 years old in one cluster, 19 years old in one cluster, etc.), then randomly sample three clusters and survey all students in those clusters.

  1. Random digit dialing. The Gallup Poll uses a procedure called random digit dialing, which creates phone numbers based on a list of all area codes in America in conjunction with the associated number of residential households in each area code. Give a possible reason the Gallup Poll chooses to use random digit dialing instead of picking phone numbers from the phone book.

  1. Haters are gonna hate, study confirms. A study published in the Journal of Personality and Social Psychology asked a group of 200 randomly sampled participants recruited online using Amazon’s Mechanical Turk to evaluate how they felt about various subjects, such as camping, health care, architecture, taxidermy, crossword puzzles, and Japan in order to measure their attitude towards mostly independent stimuli. Then, they presented the participants with information about a new product: a microwave oven. This microwave oven does not exist, but the participants didn’t know this, and were given three positive and three negative fake reviews. People who reacted positively to the subjects on the dispositional attitude measurement also tended to react positively to the microwave oven, and those who reacted negatively tended to react negatively to it. Researchers concluded that “some people tend to like things, whereas others tend to dislike things, and a more thorough understanding of this tendency will lead to a more thorough understanding of the psychology of attitudes.” (Hepler and Albarracı́n 2013)

    1. What are the cases?

    2. What is (are) the response variable(s) in this study?

    3. What is (are) the explanatory variable(s) in this study?

    4. Does the study employ random sampling? Explain your reasoning.

    5. Is this an observational study or an experiment? Explain your reasoning.

    6. Can we establish a causal link between the explanatory and response variables?

    7. Can the results of the study be generalized to the population at large?

  1. Reading the paper. Below are excerpts from two articles published in the NY Times:

    1. An excerpt from an article titled Risks: Smokers Found More Prone to Dementia is below. Based on this study, can we conclude that smoking causes dementia later in life? Explain your reasoning. (Rabin 2010)

    “Researchers analyzed data from 23,123 health plan members who participated in a voluntary exam and health behavior survey from 1978 to 1985, when they were 50-60 years old. 23 years later, about 25% of the group had dementia, including 1,136 with Alzheimer’s disease and 416 with vascular dementia. After adjusting for other factors, the researchers concluded that pack-a-day smokers were 37% more likely than nonsmokers to develop dementia, and the risks went up with increased smoking; 44% for one to two packs a day; and twice the risk for more than two packs.”

    1. An excerpt from an article titled The School Bully Is Sleepy is below. A friend of yours who read the article says, “The study shows that sleep disorders lead to bullying in school children.” Is this statement justified? If not, how best can you describe the conclusion that can be drawn from this study? (Parker-Pope 2011)

    “The University of Michigan study, collected survey data from parents on each child’s sleep habits and asked both parents and teachers to assess behavioral concerns. About a third of the students studied were identified by parents or teachers as having problems with disruptive behavior or bullying. The researchers found that children who had behavioral issues and those who were identified as bullies were twice as likely to have shown symptoms of sleep disorders.”

  1. Sampling strategies. A statistics student who is curious about the relationship between the amount of time students spend on social networking sites and their performance at school decides to conduct a survey. Various research strategies for collecting data are described below. In each, name the sampling method proposed and any bias you might expect.

    1. They randomly sample 40 students from the study’s population, give them the survey, ask them to fill it out, and bring it back the next day.

    2. They give out the survey only to their friends, making sure each one of them fills it out.

    3. They post a link to an online survey on Facebook and ask their friends to fill it out.

    4. They randomly sample 5 classes and asks a random sample of students from those classes to fill out the survey.

  1. Family size. Suppose we want to estimate household size, where a “household” is defined as people living together in the same dwelling, and sharing living accommodations. If we select students at random at an elementary school and ask them what their family size is, will this be a good measure of household size? Or will our average be biased? If so, will it overestimate or underestimate the true value?

  1. Light and exam performance. A study is designed to test the effect of light level on exam performance of students. The researcher believes that light levels might have different effects on people who wear glasses and people who don’t, so they want to make sure both groups of people are equally represented in each treatment. The treatments are fluorescent overhead lighting, yellow overhead lighting, no overhead lighting (only desk lamps).

    1. What is the response variable?

    2. What is the explanatory variable? What are its levels?

    3. What is the blocking variable? What are its levels?

  1. Vitamin supplements. To assess the effectiveness of taking large doses of vitamin C in reducing the duration of the common cold, researchers recruited 400 healthy volunteers from staff and students at a university. A quarter of the patients were assigned a placebo, and the rest were evenly divided between 1g Vitamin C, 3g Vitamin C, or 3g Vitamin C plus additives to be taken at onset of a cold for the following two days. All tablets had identical appearance and packaging. The nurses who handed the prescribed pills to the patients knew which patient received which treatment, but the researchers assessing the patients when they were sick did not. No statistically discernible differences were observed in any measure of cold duration or severity between the four groups, and the placebo group had the shortest duration of symptoms. (Audera et al. 2001)

    1. Was this an experiment or an observational study? Why?

    2. What are the explanatory and response variables in this study?

    3. Were the patients blinded to their treatment?

    4. Was this study double-blind?

    5. Participants are ultimately able to choose whether to use the pills prescribed to them. We might expect that not all of them will adhere and take their pills. Does this introduce a confounding variable to the study? Explain your reasoning.

  1. Light, noise, and exam performance. A study is designed to test the effect of light level and noise level on exam performance of students. The researcher believes that light and noise levels might have different effects on people who wear glasses and people who don’t, so they want to make sure both groups of people are equally represented in each treatment. The light treatments considered are fluorescent overhead lighting, yellow overhead lighting, no overhead lighting (only desk lamps). The noise treatments considered are no noise, construction noise, and human chatter noise.

    1. What type of study is this?

    2. How many factors are considered in this study? Identify them, and describe their levels.

    3. What is the role of the wearing glasses variable in this study?

  1. Music and learning. You would like to conduct an experiment in class to see if students learn better if they study without any music, with music that has no lyrics (instrumental), or with music that has lyrics. Briefly outline a design for this study.
  1. Soda preference. You would like to conduct an experiment in class to see if your classmates prefer the taste of regular Coke or Diet Coke. Briefly outline a design for this study.

  1. Exercise and mental health. A researcher is interested in the effects of exercise on mental health and they propose the following study: use stratified random sampling to ensure representative proportions of 18-30, 31-40 and 41- 55 year-olds from the population. Next, randomly assign half the subjects from each age group to exercise twice a week, and instruct the rest not to exercise. Conduct a mental health exam at the beginning and at the end of the study, and compare the results.

    1. What type of study is this?

    2. What are the treatment and control groups in this study?

    3. Does this study make use of blocking? If so, what is the blocking variable?

    4. Does this study make use of blinding?

    5. Comment on whether the results of the study can be used to establish a causal relationship between exercise and mental health, and indicate whether the conclusions can be generalized to the population at large.

    6. Suppose you are given the task of determining if this proposed study should get funding. Would you have any reservations about the study proposal?

  1. Chia seeds and weight loss. Chia Pets – those terra-cotta figurines that sprout fuzzy green hair – made the chia plant a household name. But chia has since gained a reputation as a diet supplement. In one 2009 study, 38 men and 38 women were recruited and and divided each randomly into two groups: treatment or control. One group was given 25 grams of chia seeds twice a day, and the other was given a placebo. The subjects volunteered to be a part of the study. After 12 weeks, the scientists found no statistically discernible difference between the groups in appetite or weight loss. (Nieman et al. 2009)

    1. What type of study is this?

    2. What are the experimental and control treatments in this study?

    3. Has blocking been used in this study? If so, what is the blocking variable?

    4. Has blinding been used in this study?

    5. Comment on whether we can make a causal statement, and indicate whether we can generalize the conclusion to the population at large.

  1. City council survey. A city council has requested a household survey be conducted in a suburban area of their city. The area is broken into many distinct and unique neighborhoods, some including large homes, some with only apartments, and others a diverse mixture of housing structures. For each part below, identify the sampling methods described, and describe the statistical pros and cons of the method in the city’s context.

    1. Randomly sample 200 households from the city.

    2. Divide the city into 20 neighborhoods, and then sample 10 households from each neighborhood.

    3. Divide the city into 20 neighborhoods, randomly sample 3 neighborhoods, and then sample all households from those 3 neighborhoods.

    4. Divide the city into 20 neighborhoods, randomly sample 8 neighborhoods, and then randomly sample 50 households from those neighborhoods.

    5. Sample the 200 households closest to the city council offices.

  1. Flawed reasoning. Identify the flaw(s) in reasoning in the following scenarios. Explain what the individuals in the study should have done differently if they wanted to make such strong conclusions.

    1. Students at an elementary school are given a questionnaire that they are asked to return after their parents have completed it. One of the questions asked is, “Do you find that your work schedule makes it difficult for you to spend time with your kids after school?” Of the parents who replied, 85% said “no”. Based on these results, the school officials conclude that a great majority of the parents have no difficulty spending time with their kids after school.

    2. A survey is conducted on a simple random sample of 1,000 women who recently gave birth, asking them about whether they smoked during pregnancy. A follow-up survey asking if the children have respiratory problems is conducted 3 years later. However, only 567 of these women are reached at the same address. The researcher reports that these 567 women are representative of all mothers.

    3. An orthopedist administers a questionnaire to 30 of his patients who do not have any joint problems and finds that 20 of them regularly go running. He concludes that running decreases the risk of joint problems.

  1. Income and education in US counties. The scatterplot below shows the relationship between per capita income (in thousands of dollars) and percent of population with a bachelor’s degree in 3,142 counties in the US in 2019.13

    1. What are the explanatory and response variables?

    2. Describe the relationship between the two variables. Make sure to discuss unusual observations, if any.

    3. Can we conclude that having a bachelor’s degree increases one’s income?

  1. Eat well, feel better. In a public health study on the effects of consumption of fruits and vegetables on psychological well-being in young adults, participants were randomly assigned to three groups: (1) diet-as-usual, (2) an ecological momentary intervention involving text message reminders to increase their fruits and vegetable consumption plus a voucher to purchase them, or (3) a fruit and vegetable intervention in which participants were given two additional daily servings of fresh fruits and vegetables to consume on top of their normal diet. Participants were asked to take a nightly survey on their smartphones. Participants were student volunteers at the University of Otago, New Zealand. At the end of the 14-day study, only participants in the third group showed improvements to their psychological well-being across the 14-days relative to the other groups. (Conner et al. 2017)

    1. What type of study is this?

    2. Identify the explanatory and response variables.

    3. Comment on whether the results of the study can be generalized to the population.

    4. Comment on whether the results of the study can be used to establish causal relationships.

    5. A newspaper article reporting on the study states, “The results of this study provide proof that giving young adults fresh fruits and vegetables to eat can have psychological benefits, even over a brief period of time.” How would you suggest revising this statement so that it can be supported by the study?

  1. Screens, teens, and psychological well-being. In a study of three nationally representative large-scale datasets from Ireland, the United States, and the United Kingdom (n = 17,247), teenagers between the ages of 12 to 15 were asked to keep a diary of their screen time and answer questions about how they felt or acted. The answers to these questions were then used to compute a psychological well-being score. Additional data were collected and included in the analysis, such as each child’s sex and age, and on the mother’s education, ethnicity, psychological distress, and employment. The study concluded that there is little clear-cut evidence that screen time decreases adolescent well-being. (Orben and Baukney-Przybylski 2018)

    1. What type of study is this?

    2. Identify the explanatory variables.

    3. Identify the response variable.

    4. Comment on whether the results of the study can be generalized to the population, and why.

    5. Comment on whether the results of the study can be used to establish causal relationships.

  1. The question “Over the last five years, what is the average time to complete a degree for Duke undergrads?” is only relevant to students who complete their degree; the average cannot be computed using a student who never finished their degree. Thus, only Duke undergrads who graduated in the last five years represent cases in the population under consideration. Each such student is an individual case. For the question “Does a new drug reduce the number of deaths in patients with severe heart disease?”, a person with severe heart disease represents a case. The population includes all people with severe heart disease.↩︎

  2. Answers will vary. From our own anecdotal experiences, we believe people tend to rant more about products that fell below expectations than rave about those that perform as expected. For this reason, we suspect there is a negative bias in product ratings on sites like Amazon. However, since our experiences may not be representative, we also keep an open mind.↩︎

  3. This is a different concept than a control group, which we discuss in the second principle and in Section 2.2.2.↩︎

  4. Also called a lurking variable, confounding factor, or a confounder.↩︎

  5. Human subjects are often called patients, volunteers, or study participants.↩︎

  6. There are always some researchers involved in the study who do know which patients are receiving which treatment. However, they do not interact with the study’s patients and do not tell the blinded health care professionals who is receiving which treatment.↩︎

  7. The researchers assigned the patients into their treatment groups, so this study was an experiment. However, the patients could distinguish what treatment they received because a stent is a surgical procedure. There is no equivalent surgical placebo, so this study was not blind. The study could not be double-blind since it was not blind.↩︎

  8. Ultimately, can we make patients think they got treated from a surgery? In fact, we can, and some experiments use a sham surgery. In a sham surgery, the patient does undergo surgery, but the patient does not receive the full treatment, though they will still get a placebo effect.↩︎

  9. Answers will vary. Population density may be important. If a county is very dense, then this may require a larger percentage of residents to live in housing units that are in multi-unit structures. Additionally, the high density may contribute to increases in property value, making homeownership unfeasible for many residents.↩︎

  10. Derived from similar figures in Chance and Rossman (2018) and Ramsey and Schafer (2012).↩︎

  11. The data used in this exercise comes from the General Social Survey, 2018.↩︎

  12. The cia_factbook data used in this exercise can be found in the openintro R package.↩︎

  13. The county_complete data used in this exercise can be found in the openintro R package.↩︎

中文

2  研究设计

在深入探讨处理数据的细节之前，我们先停下来思考数据是如何产生的。也就是说，如果要用数据得出广泛而完整的结论，那么重要的是要了解数据代表的是谁或什么。数据来源的一个重要方面是抽样。了解观察单元是如何从更大的实体中被选出的，将使我们可以将结论推广到数据所随机抽取自的总体。此外，通过理解研究的结构，可以将因果关系与那些仅存在关联的关系区分开来。在开始处理数据之前，先问自己一个好问题：“这些观测是如何收集的？”。通过了解数据的来源，你将对该数据有更多的了解。

2.1 抽样原则与策略

开展研究的第一步是确定要研究的主题或问题。一个清晰明确的研究问题有助于确定应该研究哪些对象或案例，以及哪些变量是重要的。同样重要的是要考虑 如何 收集数据，以使数据可靠并有助于实现研究目标。

2.1.1 总体与样本

考虑以下三个研究问题：

  1. 大西洋中剑鱼的汞含量平均值是多少？
  2. 在过去五年中，杜克大学本科生完成学位的平均时间是多少？
  3. 一种新药能否减少严重心脏病患者的死亡人数？

每个研究问题都指向一个目标 总体。在第一个问题中，目标总体是大西洋中的所有剑鱼，每条鱼代表一个个案。通常情况下，为总体中的每个个案收集数据并不可行。为整个总体收集数据称为 普查。普查之所以困难，是因为为整个总体收集数据的成本过高，但也可能是因为难以甚至无法确定所关注的整个总体！这时就会改为抽取样本。 样本 就是你所拥有的数据。理想情况下，样本是总体的很小一部分。例如，可能会从总体中选出60条剑鱼（或其他某个数量），这些样本数据可用于提供总体平均值的估计并回答研究问题。

对于上面的第二个和第三个问题，请确定目标总体以及什么代表一个个案。1

2.1.2 参数与统计量

在大多数统计分析过程中，当前的研究问题归根结底在于理解一个数值汇总。这个数字（或一组数字）可能是你已经熟悉的量（比如平均值），也可能是你通过本书学到的东西（比如最小二乘模型中的斜率和截距，见 第 7.2 节).

数值汇总既可以在观测样本上计算，也可以在整个总体上计算。然而，测量总体中的每一个单位通常是不可行的。因此，“典型的”数值汇总是从样本中计算得出的。不过，我们仍然可以在概念上设想计算阿根廷所有成年人的平均收入。

我们使用特定的术语来区分：一个数字是在数据样本上计算的（样本统计量），还是对整个总体进行计算或考虑对其进行计算的（总体参数）。统计量和参数这两个术语对于传达论断和模型非常有用，并将在后续深入探讨对总体进行推断的章节中被广泛使用。

2.1.3 轶事证据

考虑针对这三个研究问题的以下几种可能的回答：

  1. 新闻中报道一名男子因食用剑鱼而汞中毒，所以剑鱼中的汞平均浓度一定高得危险。
  2. 我认识两名在杜克大学花了超过7年才毕业的学生，所以在杜克大学毕业一定比在其他许多大学耗时更长。
  3. 我朋友的爸爸在服用一种新的心脏病药物后心脏病发作并去世了，所以这种药肯定无效。

每个结论都基于数据。然而，这里存在两个问题。第一，这些数据仅代表一两个个案。第二，也是更重要的是，这些个案是否真正代表总体并不清楚。以这种随意方式收集的数据被称为 轶事证据.

轶事证据。

要小心以随意方式收集的数据。这类证据可能是真实且可验证的，但它可能只代表特殊个案，因此不能很好地代表总体。

轶事证据通常由我们根据其显著特征而记住的特殊个案组成。例如，比起那六位四年毕业的人，我们更容易记住我们遇到过的两位花了7年才毕业的人。与其关注最特殊的个案，我们应该考察一个包含许多个案、能更好代表总体的样本。

2.1.4 从总体中抽样

我们或许会尝试通过收集毕业生样本来估计杜克大学本科生过去五年的毕业所需时间。过去五年的所有毕业生代表 总体，而被选中进行审查的毕业生统称为 样本。一般来说，我们总是力求 随机 地从总体中抽取一个样本。最基本的随机抽取方式就相当于抽奖的进行方式。例如，在挑选毕业生时，我们可以把每位毕业生的名字写在抽奖券上，然后抽出10张抽奖券。被抽中的名字就代表了一个由10名毕业生组成的随机样本。

A large circle contains many dots which indicate all the graduates. A smaller circle contains a few of the dots (i.e., graduates) which have been randomly selected from the larger circle.
图 2.1：从总体中随机抽取10名毕业生纳入样本。

假设我们请一位恰好主修营养学的学生为这项研究挑选几名毕业生。你认为他们可能会挑选哪些学生？你认为他们的样本能代表所有毕业生吗？

他们可能会挑选数量不成比例的健康相关领域的毕业生，如 图 2.2所示。在手工挑选样本时，我们可能会选到 有偏的 样本，即使我们的偏差并非有意为之。

A large circle contains many dots which indicate all the graduates, but some of the dots have been greyed out where others are dark dots from which the sample is taken. A smaller circle contains a few of the dots (i.e., graduates) which have been selected from the biased group of dark dots in the large circle.
图 2.2：在被要求挑选一个毕业生样本时，营养学专业的学生可能会在无意中挑选出数量不成比例的、来自健康相关专业的毕业生。

如果允许某人自行挑选究竟让哪些毕业生被纳入样本，那么样本完全有可能过度代表此人的兴趣，而这可能完全是无意为之。这就将 偏差 引入样本。随机抽样有助于解决这一问题。最基本的随机样本被称为 简单随机样本 ，这相当于从帽子里抽名字来选取个案。这意味着总体中的每个个案都有均等的机会被纳入样本，并且样本中的个案彼此之间互不相关。

采取简单随机抽样的做法有助于将偏差降至最低。然而，偏差还可能以其他方式出现。即使人们是随机挑选的，例如在调查中，也必须保持谨慎，如果 无应答率 很高。例如，如果一项调查随机抽取的人当中只有30%的人真正作出了回应，那么就不清楚这些结果是否 能代表 整个总体。这种 无应答偏倚 会使结果产生偏斜。

A large circle contains many dots which indicate the population of interest, but some of the dots have been greyed out where others are dark dots from which the sample is taken (where the grey dots are potentially due to non-response bias). A smaller circle contains a few of the dots which have been selected from the group of dark dots in the large circle who were individuals willing to respond to the survey.
图 2.3：由于存在无应答的可能性，调查研究可能只能触及总体中的某个特定群体。这个问题很难彻底解决，而且往往根本无法解决。

另一个常见的缺陷是 便利样本，即容易接触到的人更有可能被纳入样本。例如，如果一项政治调查是通过拦住在布朗克斯区（Bronx）街头行走的人来进行的，这将无法代表整个纽约市。通常很难辨别便利样本所代表的是哪个子总体。

我们可以通过网站轻松获取产品、卖家和公司的评分。这些评分仅基于那些特意主动提供评分的人。如果某产品的在线评论中有50%是负面的，你认为这意味着50%的买家对该产品不满意吗？为什么？2

2.1.5 四种抽样方法

几乎所有统计方法都基于隐含随机性这一概念。如果数据不是以随机的框架从总体中收集的，这些统计方法——估计值以及与估计值相关的误差——就不可靠。这里我们考虑四种随机抽样技术：简单随机抽样、分层抽样、整群抽样和多阶段抽样。 图 2.4 和 图 2.5 提供了这些技术的图形化表示。

简单随机抽样 可能是随机抽样中最直观的一种形式。以美国职业棒球大联盟（MLB）球员的薪水为例，每名球员都是联盟30支球队之一的成员。要抽取120名棒球球员及其薪水的简单随机样本，我们可以把这个赛季数百名球员的名字写在纸条上，将纸条放入一个桶中，摇动桶直到确信所有名字已充分混合，然后抽取纸条，直到得到120名球员的样本。一般来说，如果总体中的每个个案都有相同的机会被纳入最终样本，则该样本被称为“简单随机”样本 而且 知道某个案例被包含在样本中，并不能提供关于还有哪些其他案例被包含在内的有用信息。

The top box shows a population of dots (i.e., individuals) where a handful of the dots have been sampled randomly. The bottom box shows the same population of dots but grouped in such a way that there are six strata. From each stratum three dots (i.e., individuals) are randomly selected.
图 2.4：简单随机抽样与分层抽样的示例。在上图中，采用简单随机抽样随机选取了18个案例（以红色标示）。在下图中，采用了分层抽样：先将案例分组为层，然后在每个层内使用简单随机抽样随机选取3个案例。’

分层抽样 是一种分而治之的抽样策略。总体被划分为若干称为 层。这些层的选取方式是让相似的案例归为一组，然后在每个层内采用第二种抽样方法，通常为简单随机抽样。在棒球薪水的例子中，30支球队中的每一支都可以代表一个层，因为有些球队的资金要多得多（最多可达4倍！）。然后，我们可以从每支球队中随机抽取4名球员，组成由120名球员构成的样本。

分层抽样 在每一层中的个案就所关注的结果而言非常相似时尤为有用。缺点是，分析来自分层样本的数据比分析来自简单随机样本的数据更为复杂。要分析通过分层抽样收集的数据，需要将本书中介绍的分析方法加以扩展。

为什么每一层内的个案非常相似会比较好呢？

如果个案非常相似，我们也许能为某一层中的子总体得到更稳定的估计，从而使每组内的估计更加精确。当我们把这些估计合并成对整个总体的单一估计时，由于每一个单独的组估计本身更为精确，这个总体估计往往会更加精确。

在 整群样本中，我们把总体分成许多组，称为 群。然后，我们抽取固定数目的群，并将来自其中每个群的所有观测值都纳入样本。一个 多阶段样本 类似于整群样本，但不是保留每个群内的所有观测，而是在每个被选中的群内收集一个随机样本。

In the top figure, dots are grouped into clusters, three clusters are selected, and every dot (i.e., all individuals) from each of the three clusters are sampled. In the bottom figure, dots are again grouped into clusters and three clusters are selected. However, random sampling is applied so that a random sample from each of the three selected clusters is taken.
图 2.5：整群抽样与多阶段抽样的示例。上面的面板中使用的是整群抽样：数据被分入九个群，对其中三个群进行了抽样，且这三个群内的所有观测都被纳入了样本。下面的面板中使用的是多阶段抽样，它与整群抽样的唯一区别在于：我们随机选取每个群的一个子集纳入样本，而不是测量被抽样的每个群中的每一个个案。’

有时，整群抽样或多阶段抽样可能比其他备选抽样方法更经济。此外，与分层抽样不同，当一个群内个案之间的变异很大、而各个群之间看起来没有太大差异时，这些方法最为有用。例如，如果以社区作为群，那么当每个社区内的人口非常多样化时，整群抽样或多阶段抽样效果最好。这些方法的一个缺点是，分析数据通常需要更高级的技术，不过本书中的方法可以加以扩展来处理这类数据。

假设我们有兴趣估计印度尼西亚农村一片热带丛林茂密地区的疟疾感染率。我们了解到，在印度尼西亚丛林的这一区域有 30 个村庄，每个村庄彼此都大同小异，但村庄之间的距离相当远。我们想对 150 个人进行疟疾检测。我们应该采用哪种抽样方法？

简单随机抽样很可能会从全部 30 个村庄中抽取个体，这会使数据收集成本高昂。分层抽样则会是一个难题，因为我们不清楚该如何由相似的个体构建层。然而，整群抽样或多阶段抽样看起来是非常好的主意。采用多阶段抽样，我们可以随机选取一半的村庄，然后从每个村庄中随机选取 10 个人。与简单随机抽样相比，这可以大幅降低数据收集成本，而且整群样本仍能提供可靠的信息，即使我们需要用比本书所介绍的更高级的方法来分析数据。

2.2 实验

研究者将处理分配给个案的研究称为 实验。当这种分配包含随机化时，例如使用抛硬币来决定患者接受哪种治疗，就称为 随机实验。在试图证明两个变量之间的因果关系时，随机实验具有根本性的重要意义。

2.2.1 实验设计的原则

  1. 控制。 研究人员将处理分配给各个个案，并尽最大努力 控制 组之间的任何其他差异3。例如，当患者以药片形式服用药物时，有些患者只用一小口水送服药片，而另一些患者可能会用一整杯水送服。为了控制饮水量的影响，医生可能会指示每位患者随药片喝下一杯12盎司的水。
  1. 随机化。 研究人员将患者随机分配到各个治疗组，以考虑那些无法控制的变量。例如，由于饮食习惯不同，一些患者可能比其他患者更容易患某种疾病。在这个例子中，饮食习惯是一个 混淆变量4，其定义为一个与解释变量和响应变量都相关的变量。将患者随机分配到治疗组或对照组有助于均衡这类差异。

混杂变量。

一个 混淆变量 是指与解释变量和响应变量都相关联的变量。由于它同时与这两个变量相关联，因而使研究无法得出解释变量导致响应变量的结论。考虑一个有点滑稽的例子：将冰淇淋总销售额作为解释变量，将船只事故的数量作为响应变量（两者可能看起来高度相关）。室外温度与这两个变量都相关联，因此我们不能断定冰淇淋销量高是造成更多船只事故的原因。

混淆变量可能作为研究的一部分被测量，也可能不被测量。尽管如此，由于始终存在出现混淆变量的可能性，在观察性研究中很难得出因果结论。

  1. 重复。 研究者观察的案例越多，就越能准确地估计解释变量对响应变量的影响。在单一研究中，我们 重复 ，即通过收集足够大的样本。至于多大的样本才算足够大，则因实验而异，但至少我们希望每个处理组中都有多个受试对象（实验单位）。实现重复的另一种方法是重复整项研究，以验证先前的发现。术语 可重复性危机 指的是当前持续存在的一场方法论危机，即若干学科中以往科学研究的发现无法被重复验证。 伪重复 当不同处理条件下的个体观测值彼此高度相关时便会发生。例如，假设你在一项实验中有 50 名受试者，在整个研究过程中于 10 个时间点测量血压。到最后，你将得到 50 \(\times\) 10 = 500 次测量。如果报告称你有 500 个观测值，就会被认为属于伪重复，因为同一个体的血压测量值彼此之间并不独立。伪重复常常发生在复制了错误对象的情况下，并且报告的样本量被夸大了。

  1. 区组化。 研究人员有时会知道或怀疑，除处理之外的变量也会影响响应。在这种情况下，他们可能会先根据该变量将个体划分入 区组 然后将每个区组内的个案随机分配到各处理组。这种策略通常被称为 区组化。例如，如果我们想考察某种药物对心脏病发作的影响，我们可以先将研究中的患者划分为低风险区组和高风险区组，然后从每个区组中随机将一半患者分配到对照组，另一半分配到处理组，如图 图 2.6。这一策略确保每个处理组中低风险患者的数量相同，高风险患者的数量也相同。
Before randomly allocating, the red low risk patients and blue high risk patients are split into two separate groups. Subsequently, half of the red low risk patients are randomly chosen to receive the treatment, and half of the blue high risk patients are randomly chosen to receive the treatment.
图 2.6：按患者风险进行区组化。患者首先被划分为低风险区组和高风险区组，然后将每个区组中的患者均匀地随机分配到各个处理组中。这一策略确保来自两种风险类别的患者在每个处理组中都得到同等的代表。

将前三个实验设计原则融入任何研究都很重要，本书介绍了分析此类实验所得数据的适用方法。区组化是一种稍微进阶的技术，本书中的统计方法可以加以扩展，用于分析采用区组化收集的数据。

2.2.2 减少人体实验中的偏差

随机化实验长期以来一直被认为是数据收集的金标准，但它并不能在所有情况下都确保对因果关系的无偏考察。人体研究正是偏差可能在无意中产生的典型例子。这里我们重新审视一项使用新药治疗心脏病发作患者的研究。具体而言，研究人员想知道这种药物是否能降低患者的死亡人数。

这些研究人员之所以设计随机化实验，是因为他们想就这种药物的疗效得出因果结论。研究志愿者5 被随机分入两个研究组。其中一组，即 处理组，服用了该药物。另一组被称为 对照组，未接受任何药物治疗。

请把自己设想成研究中的一名参与者。如果你在处理组，你会得到一种新潮的新药，并期待它能对你有所帮助。另一方面，另一组的人则没有拿到药物，只能闲坐着，希望自己参与研究并不会增加死亡的风险。这些视角表明，这项研究中实际上存在两种效应：一种是研究者所关注的药物疗效，另一种是（不）服用药物所带来的情绪效应，而这种效应难以量化。

研究者通常对这种情绪效应并不感兴趣，而它可能会使研究产生偏倚。为了避免这一问题，研究者不希望患者知道自己属于哪一组。当研究者对患者隐瞒其所接受的治疗时，该研究就被称为 盲的。但存在一个问题：如果患者没有接受治疗，就会知道自己处于对照组。解决这一问题的方法是给对照组的患者使用假治疗。这被称为 安慰剂，而有效的安慰剂是使研究真正实现盲法的关键。安慰剂的一个经典例子是制成后外观与真正的治疗药丸一样的糖丸。然而，在某些实验中提供这种假治疗可能不符合伦理。例如，在医学实验中，对照组通常必须接受当前的标准治疗。安慰剂常常会使患者出现轻微但真实的改善。这种效应被称为 安慰剂效应.

需要设盲的不仅是患者：医生和研究人员也可能在无意中使研究产生偏倚。当医生知道某位患者接受的是真正的治疗时，可能会在无意中给予该患者比已知正在使用安慰剂的患者更多的关注或照顾。为了防范这种偏倚（它在某些情况下同样被发现具有可测量的影响），大多数现代研究采用 双盲 设计，即与患者互动的医生或研究人员与患者一样，不知道谁在接受治疗、谁没有接受治疗。6

回顾 第1.1节 研究人员在其中检验支架能否有效降低高危患者的中风风险。这是一项实验吗？该研究是否设盲？是否为双盲？7

对于 第1.1节中的研究，研究人员能否采用安慰剂？如果可以，这种安慰剂会是什么样的？8

对于通过假手术来制造安慰剂的做法，你可能有许多伦理方面的疑问。在一般的实验情境中，当一种可能有益的治疗被扣留而不给予对照组中的个体时，这些问题甚至可能已经在你的脑海中浮现过；主要区别在于，假手术往往会带来额外的风险，而扣留治疗只是使一个人维持原有的风险。

对于实验和安慰剂，总是存在多种观点，而且很少能显而易见地看出哪一种在伦理上是“正确”的。例如，当假手术会给患者带来风险时，使用假手术是否符合伦理？然而，如果我们不使用假手术，我们可能会推动使用一种并无实际效果却代价高昂的治疗；如果发生这种情况，资金和其他资源将从其他已知有益的治疗中被转移走。归根结底，这是一种困难的处境：我们无法同时完美地保护自愿参加研究的患者，以及未来可能从该治疗中受益（或不受益）的患者。

2.3 观察性研究

未明确施加处理（或明确不施加处理）的研究被称为 观察性研究。例如， 第 1.2 节 中所述的针对贷款数据和县级数据的研究均会被视为观察性研究，因为它们依赖于 观察性数据.

基于实验得出因果结论通常是合理的，因为我们可以随机分配解释变量（即处理）。然而，基于观察性数据得出同样的因果结论可能暗藏风险，并不被推荐。因此，观察性研究通常仅足以显示关联或形成假设，而这些假设日后可通过实验加以检验。

假设一项观察性研究对防晒霜的使用与皮肤癌进行了跟踪，结果发现一个人使用的防晒霜越多，其患皮肤癌的可能性就越大。这是否意味着防晒霜 导致 皮肤癌?

不!一些先前的研究告诉我们，使用防晒霜实际上会降低患皮肤癌的风险，所以也许存在另一个变量可以解释防晒霜使用与皮肤癌之间的这种假设关联，如图 图 2.7。缺失的一条重要信息是日晒。如果某人一整天都待在太阳底下，他更可能使用防晒霜 而且 更可能患皮肤癌。在这项简单的观察性调查中，日晒并未被考虑在内。

Three boxes are shown in a triangle arrangement representing: sun exposure, using sunscreen, and skin cancer. A solid arrow connects sun exposure as a causal mechanism to using sunscreen; a solid arrow also connects sun exposure as a causal mechanism to skin cancer. A questioning arrow indicates that the causal effect of using sunscreen on skin cancer is unknown.
图 2.7:日晒可能是使用防晒霜和患皮肤癌的共同根本原因。

在这个例子中，日晒就是一个混杂变量。正是混杂变量的存在，使得观察性研究难以作出因果论断。虽然为从观察性研究得出因果结论提供依据的一种方法是穷尽对混杂变量的搜索，但并不能保证所有混杂变量都能被检验或测量。

图 1.2 显示了某县住房自有率与多单元结构住房单元所占百分比之间的负相关关系。然而，断言这两个变量之间存在因果关系是不合理的。请提出一个可能解释这种负相关关系的变量。9

观察性研究有两种形式：前瞻性研究和回顾性研究。一项 前瞻性研究 会识别个体并在事件展开的过程中收集信息。例如，医学研究人员可能会识别出一组患者并进行多年随访，以评估行为对癌症风险可能产生的影响。这类研究的一个例子是护士健康研究。该研究始于1976年，并于1989年扩展，已收集了超过275,000名护士的数据，目前仍在招募参与者。这项前瞻性研究招募注册护士，然后通过问卷从她们那里收集数据。 回顾性研究 在事件发生之后收集数据，例如，研究人员可以查阅医疗记录中的既往事件。有些数据集可能同时包含以前瞻性和回顾性方式收集的变量，例如某些医学研究会在参与者进入研究之前收集其生活方面的信息，并在整个研究过程中持续收集参与者的数据。

2.4 本章复习

2.4.1 小结

一名熟练的分析师会对其所处理的数据类型以及如何将数据可视化有良好的把握，从而全面理解各个变量。然而，同样重要的是数据来源。在本章中，我们讨论了随机化实验以及如何从总体中抽取良好的、随机的、有代表性的样本。当我们讨论推断方法时（从 第 11 章）开始，能够得出的结论将取决于数据是如何收集的。 图 2.8 总结了抽样和分配方法与推断范围之间的关系。10 定期回顾 图 2.8 在根据某项给定的数据分析得出结论时将十分重要。

A two by two table describing the scenarios of random sample or not and random allocation or not. Selecting randomly from a population allows for generalization back to the population. Randomly allocating in an experiment allows for establishing causation.
图 2.8：应当根据数据的收集方式谨慎地作出分析结论。很少有数据集来自左上角的方框，因为伦理规范通常要求处理的随机分配只能给予志愿者。代表性（理想情况下为随机）抽样和实验（处理的随机分配）对于能否就总体作出统计结论都很重要。

2.4.2 术语

本章中介绍的术语列于 表 2.1。如果您不确定其中一些术语的含义，我们建议您回到正文中复习它们的定义。您应该能够很容易地发现它们是 粗体文本.

表 2.1：本章中介绍的术语。
轶事证据 实验 重复
偏差 多阶段样本 可重复性危机
盲的 无应答偏倚 能代表
区组化 无应答率 回顾性研究
普查 观察性研究 样本
群 安慰剂 样本偏差
整群抽样 安慰剂效应 样本统计量
混淆变量 总体 简单随机样本
控制 总体参数 简单随机抽样
对照组 前瞻性研究 层
便利样本 伪重复 分层抽样
双盲 随机实验 处理组

2.5 练习

奇数编号习题的答案见 附录 A.2.

  1. 参数与统计量。 判断哪个数值代表样本均值，哪个数值代表所声称的总体均值。

    1. 2007年，美国家庭在万圣节商品（如服装、装饰品和糖果）上的平均花费约为 $52。为了解这一数字是否发生了变化，研究人员在行业数据公布之前于2008年开展了一项新的调查。该调查涵盖1,500户家庭，发现每户家庭的万圣节平均支出为 $58。

    2. 2001年，一所私立大学学生的平均GPA为3.37。十年后，对该校203名学生样本进行的一项调查得出的平均GPA为3.59。

  1. 大学生的睡眠。 某大学报纸最近的一篇文章称，大学生每晚平均睡眠5.5小时。一名对该数值持怀疑态度的学生决定通过随机抽取25名学生来进行一项调查。受调查的学生平均每晚睡眠6.25小时。判断哪个数值代表样本均值，哪个数值代表所声称的总体均值。

  1. 空气污染与出生结局、推断范围。 研究人员收集数据以考察南加利福尼亚地区空气污染物与早产之间的关系。在研究期间，空气污染水平由空气质量监测站进行测量。研究人员收集了1989年至1993年间143,196例出生的妊娠时长数据，并计算了每次妊娠期间的空气污染暴露量。 (Ritz 等，2000)

    1. 指出本研究中感兴趣的总体和样本。

    2. 评论本研究的结果能否推广到总体，以及本研究的发现能否用于建立因果关系。

  1. 作弊者，推断范围。 研究诚实、年龄与自控力之间关系的研究人员对160名5至15岁的儿童进行了一项实验。研究人员要求每个孩子私下抛一枚均匀的硬币，并将结果（白色或黑色）记录在一张纸上，并说他们只会奖励报告白色的孩子。一半的学生被明确告知不许作弊，其余学生则未得到任何明确指示。在有指示组和无指示组之间观察到作弊率的差异，此外各组内不同儿童特征之间也存在一些差异。 (Bucciol 和 Piovesan，2011)

    1. 指出本研究中感兴趣的总体和样本。

    2. 评论本研究的结果能否推广到总体，以及本研究的发现能否用于建立因果关系。

  1. 游戏化与统计，推断范围。 研究人员为考察游戏化(在非游戏情境中应用游戏设计元素和游戏原则)对统计学学习的影响，将一门统计学课程中的365名大学生随机分配到四个组之一；其中一个组既无阅读练习也无游戏化，一个组有阅读但无游戏化，一个组有游戏化但无阅读，最后一个组既有游戏化又有阅读。所有组的学生还参加了讲座课。研究发现，与涉及阅读练习的传统教学方法相比，游戏化对学生的学习有积极影响。 (Legaki et al. 2020)

    1. 指出本研究中感兴趣的总体和样本。

    2. 评论本研究的结果能否推广到总体，以及本研究的发现能否用于建立因果关系。

  1. 偷窃者，推断范围。 在一项关于社会经济阶层与不道德行为之间关系的研究中，129名加州大学伯克利分校的本科生被要求通过与拥有最多(最少)金钱、受过最多(最少)教育以及从事最受(最不受)尊敬职业的他人进行比较，来确定自己是低社会阶层还是高社会阶层。研究人员还向他们展示了一罐独立包装的糖果，并告知这些糖果是给附近实验室的孩子们的，但如果他们想要，可以拿一些。在完成一些不相关的任务后，参与者报告了自己拿走的糖果数量。研究发现，被认定为上层阶级的人比其他人拿走了更多糖果。 (Piff et al. 2012)

    1. 指出本研究中感兴趣的总体和样本。

    2. 评论本研究的结果能否推广到总体，以及本研究的发现能否用于建立因果关系。

  1. 工作后的放松。 综合社会调查提出了这样一个问题， “在一个平均工作日之后，你大约有几个小时可以用来放松或从事自己喜爱的活动？” 该问题的询问对象是由1,155名美国人组成的随机样本。结果发现平均放松时间为1.65小时。请判断以下各项中哪一个是观测值、变量、样本统计量或总体参数。11

    1. 样本中的一名美国人。

    2. 一个平均工作日之后花在放松上的小时数。

    3. 1.65.

    4. 所有美国人在一个普通工作日结束后用于放松的平均小时数。

  1. YouTube上的猫。 假设你想估计YouTube上猫咪视频所占的百分比。你不可能看完YouTube上的所有视频，因此你使用一个随机视频选择器为你选出1000个视频。你发现其中2%的视频是猫咪视频。请判断以下各项分别是一个观测值、一个变量、一个样本统计量，还是一个总体参数。

    1. YouTube上所有视频中猫咪视频所占的百分比。

    2. 2%.

    3. 你样本中的一个视频。

    4. 一个视频是否是猫咪视频。

  1. 不同小班之间的课程满意度。 某大学的一门大型课程共有160名学生。所有160名学生一起上大课，但学生被分成4组，每组40名学生，参加由不同助教负责的实验小班。教授想就学生对课程的满意度进行一次调查，并且他认为学生所在的实验小班可能会影响该学生对课程的整体满意度。

    1. 这是一项什么类型的研究？

    2. 为开展这项研究提出一种抽样策略。

  1. 各宿舍间的住房提案。 在一所大型大学校园里，一年级和二年级学生住在位于校园东部的宿舍，三年级和四年级学生住在位于校园西部的宿舍。假设你想收集学生对学校管理层提出的新住宿方案的意见，并且希望确保你的调查能平等地反映各年级学生的意见。

    1. 这是一项什么类型的研究？

    2. 为开展这项研究提出一种抽样策略。

  1. 互联网使用与预期寿命。 以下散点图是一项研究的一部分，该研究评估了在有相关数据的208个国家中，出生时预期寿命估计值（截至2014年）与互联网用户比例（截至2009年）之间的关系。12

    1. 描述预期寿命与互联网用户百分比之间的关系。

    2. 这是一项什么类型的研究？

    3. 给出一个可能解释这种关系的混杂变量，并描述其潜在影响。

  1. 压力过大。 一项针对其他方面均健康的随机抽样的高中学生的调查研究发现，学生在有压力时更容易出现肌肉痉挛。该研究还指出，学生在有压力时会喝更多咖啡、睡得更少。

    1. 这是一项什么类型的研究？

    2. 这项研究能否用于得出压力增加与肌肉痉挛之间存在因果关系的结论？

    3. 给出可能解释所观察到的压力增加与肌肉痉挛之间关系的混杂变量。

  1. 评估抽样方法。 某大学想确定其本科生中有多少比例的人支持一项新的 $25 年度费用，用于改善学生会。对于下面提出的每种方法，请指出该方法是否合理。

    1. 对500名学生的简单随机样本进行调查。

    2. 按学生的专业领域分层，然后从每一层中抽取10%的学生。

    3. 按年龄将学生分群（例如，18岁的学生为一个群，19岁的学生为一个群，等等），然后随机抽取三个群，并对这些群中的所有学生进行调查。

  1. 随机数字拨号。 盖洛普民意调查采用一种称为随机数字拨号的方法，该方法根据美国所有区号的列表以及每个区号对应的居民住户数量来生成电话号码。请给出盖洛普民意调查选择使用随机数字拨号而不是从电话簿中挑选电话号码的一个可能原因。

  1. 研究证实：喷子总会喷。 一项发表于 人格与社会心理学杂志 的研究要求一组通过Amazon’s Mechanical Turk在线招募的200名随机抽样参与者评价他们对各种主题的感受，例如露营、医疗保健、建筑、动物标本制作、填字游戏和日本，以测量他们对大多相互独立的刺激的态度。然后，他们向参与者展示了关于一款新产品的信息：一台微波炉。这款微波炉并不存在，但参与者并不知道这一点，他们拿到的是三条正面和三条负面的虚假评论。在倾向性态度测量中对这些主题反应积极的人也往往对微波炉反应积极，而那些反应消极的人则往往对其反应消极。研究人员得出的结论是： “有些人倾向于喜欢事物，而另一些人则倾向于不喜欢事物，对这种倾向更透彻的理解将带来对态度心理学的更透彻的理解。” (Hepler 和 Albarracı́n 2013)

    1. 个案是什么？

    2. 本研究中的响应变量是什么（有哪些）？

    3. 本研究中的解释变量是什么（有哪些）？

    4. 该研究是否采用了随机抽样？请解释你的理由。

    5. 这是一项观察性研究还是实验？请解释你的理由。

    6. 我们能否在解释变量和响应变量之间建立因果联系？

    7. 该研究的结果能否推广到总体？

  1. 阅读论文。 以下是两篇发表于 纽约时报:

    1. 一篇题为 风险：研究发现吸烟者更易患痴呆症 的文章节选如下。根据这项研究，我们能得出吸烟会导致晚年患痴呆症的结论吗？请解释你的理由。 (Rabin 2010)

    “研究人员分析了23,123名健康计划成员的数据，这些成员在1978年至1985年间参加了自愿体检和健康行为调查，当时他们的年龄为50-60岁。23年后，该群体中约有25%的人患有痴呆症，其中包括1,136名阿尔茨海默病患者和416名血管性痴呆患者。在对其他因素进行调整后，研究人员得出结论：每天吸一包烟的人患痴呆症的可能性比不吸烟者高37%，且风险随吸烟量的增加而上升；每天吸一到两包的风险为44%；每天吸两包以上的风险则是两倍。”

    1. 一篇题为 校园恶霸很困倦 的文章节选如下。你的一位读过这篇文章的朋友说： “这项研究表明，睡眠障碍会导致学龄儿童出现欺凌行为。” 这个说法合理吗？如果不合理，你能如何最佳地描述从这项研究中可以得出的结论？ (Parker-Pope 2011)

    “密歇根大学的这项研究收集了家长关于每个孩子睡眠习惯的调查数据，并请家长和老师共同评估行为问题。在被研究的学生中，约三分之一被家长或老师认为存在破坏性行为或欺凌问题。研究人员发现，有行为问题的儿童以及被认定为欺凌者的儿童，表现出睡眠障碍症状的可能性是其他儿童的两倍。”

  1. 抽样策略。 一名统计学专业的学生对学生在社交网站上花费的时间与其在学校的表现之间的关系感到好奇，决定开展一项调查。下面描述了几种收集数据的研究策略。请针对每一种策略，指出所提出的抽样方法以及你可能预期的任何偏差。

    1. 他们从研究总体中随机抽取40名学生，向他们发放调查问卷，请他们填写，并于第二天交回。

    2. 他们只把调查问卷发给自己的朋友，并确保每个人都填写。

    3. 他们在 Facebook 上发布了一个在线调查的链接，并请他们的朋友填写。

    4. 他们随机抽取了5个班级，并从这些班级中随机抽取一部分学生填写调查问卷。

  1. 家庭规模。 假设我们想要估计家庭户规模，其中一个 “家庭户” 的定义是居住在同一住所中并共享生活设施的一群人。如果我们从一所小学随机抽取学生并询问他们的家庭规模，这会是衡量家庭户规模的好方法吗？我们的平均值会有偏差吗？如果有，会高估还是低估真实值？

  1. 光线与考试表现。 一项研究旨在测试光照水平对学生考试表现的影响。研究者认为，光照水平对戴眼镜的人和不戴眼镜的人可能产生不同的影响，因此他们希望确保这两组人在每种处理中都被均等代表。处理方式为：头顶荧光灯照明、头顶黄色灯光照明、无头顶照明（仅使用台灯）。

    1. 响应变量是什么？

    2. 什么是解释变量？它有哪些水平？

    3. 什么是区组变量？它有哪些水平？

  1. 维生素补充剂。 为了评估服用大剂量维生素C能否缩短普通感冒的病程，研究人员从一所大学的教职员工和学生中招募了400名健康志愿者。四分之一的受试者被分配服用安慰剂，其余的人平均分为三组，分别服用1g维生素C、3g维生素C或3g维生素C加添加剂，从感冒初发时开始服用，持续两天。所有药片的外观和包装完全相同。负责把处方药片发给患者的护士知道哪位患者接受了哪种治疗，但在患者患病时对其进行评估的研究人员并不知道。四组之间在感冒病程或严重程度的任何衡量指标上均未观察到有统计学意义的差异，并且安慰剂组的症状持续时间最短。 (Audera 等，2001)

    1. 这是一项实验还是观察性研究？为什么？

    2. 这项研究中的解释变量和响应变量是什么？

    3. 患者是否对自己所接受的治疗设盲？

    4. 这项研究是双盲的吗？

    5. 参与者最终可以自行选择是否服用开给他们的药片。我们可能会预期并非所有人都会坚持服药。这是否会给研究引入一个混杂变量？请解释你的理由。

  1. 光照、噪音与考试成绩。 一项研究旨在检验光照水平和噪音水平对学生考试成绩的影响。研究者认为，光照和噪音水平对戴眼镜的人和不戴眼镜的人可能产生不同的影响，因此他们希望确保这两类人在每种处理中都得到同等代表。所考虑的光照处理包括：头顶荧光灯照明、头顶黄色灯照明、无头顶照明（仅使用台灯）。所考虑的噪音处理包括：无噪音、施工噪音和人声嘈杂。

    1. 这是一项什么类型的研究？

    2. 这项研究考虑了多少个因子？请识别这些因子，并描述它们的水平。

    3. 戴眼镜这一变量在本研究中起什么作用？

  1. 音乐与学习。 你想在课堂上进行一项实验，看看学生在以下哪种情况下学习效果更好：不听任何音乐、听无歌词的音乐（器乐），还是听有歌词的音乐。请简要概述这项研究的设计。
  1. 汽水偏好。 你想在课堂上进行一项实验，看看你的同学更喜欢普通可乐还是健怡可乐的口味。请简要概述这项研究的设计。

  1. 锻炼与心理健康。 一位研究人员对锻炼对心理健康的影响感兴趣，并提出了以下研究：使用分层随机抽样，以确保从总体中抽取的18-30岁、31-40岁和41- 55岁年龄段具有代表性比例。接下来，从每个年龄组中随机分配一半受试者每周锻炼两次，并指示其余受试者不锻炼。在研究开始和结束时分别进行心理健康检查，并比较结果。

    1. 这是一项什么类型的研究？

    2. 这项研究中的处理组和对照组分别是什么？

    3. 这项研究是否使用了区组化？如果是，区组变量是什么？

    4. 本研究是否采用了盲法？

    5. 请评述该研究的结果能否用于建立运动与心理健康之间的因果关系，并说明这些结论能否推广到总体人群。

    6. 假设你的任务是判断这项拟议研究是否应该获得资助。你对这个研究提案是否有任何保留意见？

  1. 奇亚籽与减肥。 奇亚宠物（Chia Pets）——那些会长出毛茸茸绿色“头发”的陶土小人偶——使奇亚植物家喻户晓。但此后，奇亚作为膳食补充剂也声名鹊起。在2009年的一项研究中，招募了38名男性和38名女性，并将他们每个人随机分为两组：处理组或对照组。其中一组每天两次服用25克奇亚籽，另一组服用安慰剂。受试者均为自愿参加该研究。12周后，科学家发现两组在食欲或减肥方面没有统计学上可辨识的差异。 (Nieman et al. 2009)

    1. 这是一项什么类型的研究？

    2. 在这项研究中，实验处理和对照处理分别是什么？

    3. 本研究中是否使用了区组化（blocking）？如果使用了，区组变量是什么？

    4. 本研究中是否使用了盲法？

    5. 请评述我们能否作出因果性结论，并说明我们能否将结论推广到总体。

  1. 市议会调查。 某市议会要求在其城市的一个郊区开展住户调查。该区域被划分为许多各具特色的街区，有些街区包括大型住宅，有些只有公寓，还有一些是多种住房结构的混合。针对下面的每一小题，请指出所描述的抽样方法，并结合该市的实际情况描述该方法的统计优缺点。

    1. 从该市随机抽取200户住户。

    2. 将该市划分为20个街区，然后从每个街区抽取10户住户。

    3. 将该市划分为20个街区，随机抽取3个街区，然后从这3个街区中抽取所有住户。

    4. 将城市划分为20个街区，随机抽取8个街区，然后再从这些街区中随机抽取50户家庭。

    5. 抽取距离市议会办公室最近的200户家庭。

  1. 推理存在缺陷。 找出以下情境中推理的缺陷。解释研究中的人员如果想得出如此有力的结论，应该采取哪些不同的做法。

    1. 一所小学的学生收到一份问卷，要求他们在父母填写完毕后交回。其中一个问题是： “你是否觉得你的工作日程让你很难在放学后陪伴你的孩子？” 在回复的家长中，85%的人表示 “没有”。基于这些结果，学校官员得出结论：绝大多数家长在放学后与孩子共度时光没有困难。

    2. 对最近分娩的 1,000 名女性进行了一项简单随机抽样调查，询问她们在怀孕期间是否吸烟。3 年后进行了一项后续调查，询问孩子是否有呼吸系统问题。然而，只有 567 名女性能够通过原地址联系到。研究者报告称，这 567 名女性能够代表所有母亲。

    3. 一位骨科医生对其 30 名没有任何关节问题的患者进行了问卷调查，发现其中 20 人经常跑步。他得出结论：跑步会降低患关节问题的风险。

  1. 美国各县的收入与教育。 下面的散点图显示了 2019 年美国 3,142 个县中人均收入（以千美元为单位）与拥有学士学位的人口百分比之间的关系。13

    1. 解释变量和响应变量分别是什么？

    2. 描述这两个变量之间的关系。如有异常观测值，请务必加以讨论。

    3. 我们能由此得出拥有学士学位会提高个人收入的结论吗？

  1. 吃得健康，感觉更好。 在一项关于食用水果和蔬菜对年轻人心理健康影响的公共卫生研究中，参与者被随机分配到三个组：(1) 正常饮食组；(2) 一种生态瞬间干预，即通过短信提醒他们增加水果和蔬菜的消费，并提供用于购买这些食物的代金券；或 (3) 一种水果和蔬菜干预，即参与者在正常饮食之外，每天额外获得两份新鲜水果和蔬菜食用。参与者被要求每晚在智能手机上完成一项调查。参与者是新西兰奥塔哥大学的学生志愿者。在这项为期14天的研究结束时，只有第三组的参与者在整个14天期间相比其他组表现出心理健康状况的改善。 (Conner 等人，2017年)

    1. 这是一项什么类型的研究？

    2. 指出解释变量和响应变量。

    3. 请评述该研究的结果是否可以推广到总体。

    4. 评述该研究的结果能否用于建立因果关系。

    5. 一篇报道该研究的报纸文章指出：“这项研究的结果证明，给年轻人提供新鲜水果和蔬菜食用可以带来心理上的益处，即使时间很短也是如此。”你会建议如何修改这一说法，使其能够得到该研究的支持？

  1. 屏幕、青少年与心理幸福感。 在一项对来自爱尔兰、美国和英国的三个具有全国代表性的大规模数据集(n = 17,247)开展的研究中，12至15岁的青少年被要求记录屏幕使用时间日记，并回答有关自身感受或行为的问题。这些问题的答案随后被用于计算心理幸福感得分。研究还收集了其他数据并纳入分析，例如每个孩子的性别和年龄，以及母亲的受教育程度、种族、心理困扰和就业情况。该研究得出结论：几乎没有明确证据表明屏幕使用时间会降低青少年的幸福感。 (Orben 和 Baukney-Przybylski 2018)

    1. 这是一项什么类型的研究？

    2. 指出解释变量。

    3. 指出响应变量。

    4. 评述该研究的结果能否推广到总体，并说明理由。

    5. 评述该研究的结果能否用于建立因果关系。

  1. 问题 “在过去五年中，杜克大学本科生完成学位所需的平均时间是多久？” 仅与能够完成学位的学生相关；该平均值无法通过从未完成学位的学生来计算。因此，只有过去五年内毕业的杜克大学本科生才代表所考虑总体中的个案。每个这样的学生都是一个单独的个案。对于问题 “一种新药能否减少严重心脏病患者的死亡人数？”，一名患有严重心脏病的人代表一个个案。总体包括所有患有严重心脏病的人。↩︎

  2. 答案不唯一。根据我们自己的亲身经验，我们认为人们更倾向于对低于预期的产品大发牢骚，而不是对表现符合预期的产品大加赞赏。因此，我们怀疑在 Amazon 这类网站上的产品评分存在负面偏差。不过，由于我们的经验可能不具代表性，我们也对此持开放态度。↩︎

  3. 这是一个不同于 对照组的概念，我们将在第二个原则以及 第2.2.2节.↩︎

  4. 也称为 潜伏变量, 混杂因素，或一个 混杂因素.↩︎

  5. 人类受试者通常被称为 患者, 志愿者，或 研究参与者.↩︎

  6. 参与研究的研究人员中总有一些人知道哪些患者正在接受哪种治疗。然而，他们不会与研究中的患者互动，也不会告诉设盲的医疗保健专业人员谁正在接受哪种治疗。↩︎

  7. 研究人员将患者分配到各自的治疗组中，因此这项研究是一项实验。然而，患者能够分辨自己接受的是哪种治疗，因为支架置入是一种外科手术。并不存在与之等同的外科安慰剂，因此这项研究不是盲法研究。由于该研究并非盲法研究，它也就不可能做到双盲。↩︎

  8. 归根结底，我们能否让患者以为自己在手术中接受了治疗？事实上是可以的，有些实验会采用一种 假手术。在假手术中，患者确实会接受手术，但不会接受完整的治疗，不过他们仍然会产生安慰剂效应。↩︎

  9. 答案不唯一。人口密度可能是重要因素。如果一个县的人口非常稠密，那么这可能要求更大比例的居民居住在多单元建筑内的住房单元中。此外，高密度可能会推高房产价值，使许多居民难以拥有自己的住房。↩︎

  10. 改编自以下文献中的类似图形： Chance 和 Rossman (2018) 和 Ramsey 和 Schafer (2012).↩︎

  11. 本练习中使用的数据来自 2018年综合社会调查.↩︎

  12. 该 cia_factbook 本练习中使用的数据可在 openintro R 包中找到。↩︎

  13. 该 county_complete 本练习中使用的数据可在 openintro R 包中找到。↩︎