English

11 Hypothesis testing with randomization

Statistical inference is primarily concerned with understanding and quantifying the uncertainty of parameter estimates. While the equations and details change depending on the setting, the foundations for inference are the same throughout all of statistics.

We start with two case studies designed to motivate the process of making decisions about research claims. We formalize the process through the introduction of the hypothesis testing framework, which allows us to formally evaluate claims about the population.

Throughout the book so far, you have worked with data in a variety of contexts. You have learned how to summarize and visualize the data as well as how to model multiple variables at the same time. Sometimes the dataset at hand represents the entire research question. But more often than not, the data have been collected to answer a research question about a larger group of which the data are a (hopefully) representative subset.

You may agree that there is almost always variability in data – one dataset will not be identical to a second dataset even if they are both collected from the same population using the same methods. However, quantifying the variability in the data is neither obvious nor easy to do, i.e., answering the question “how different is one dataset from another?” is not trivial.

First, a note on notation. We generally use $p$ to denote a population proportion and $\hat{p}$ to denote a sample proportion. Similarly, we generally use $\mu$ to denote a population mean and $\bar{x}$ to denote a sample mean.

Suppose your professor splits the students in your class into two groups: students who sit on the left side of the classroom and students who sit on the right side of the classroom. If $\hat{p}_{L}$ represents the proportion of students who prefer to read books on screen who sit on the left side of the classroom and $\hat{p}_{R}$ represents the proportion of students who prefer to read books on screen who sit on the right side of the classroom, would you be surprised if $\hat{p}_{L}$ did not exactly equal $\hat{p}_{R}$?

While the proportions $\hat{p}_{L}$ and $\hat{p}_{R}$ would probably be close to each other, it would be unusual for them to be exactly the same. We would probably observe a small difference due to chance.

If we do not think the side of the room a person sits on in class is related to whether they prefer to read books on screen, what assumption are we making about the relationship between these two variables?¹

Studying randomness of this form is a key focus of statistics. Throughout this chapter, and those that follow, we provide three different approaches for quantifying the variability inherent in data: randomization, bootstrapping, and mathematical models. Using the methods provided in this chapter, we will be able to draw conclusions beyond the dataset at hand to research questions about larger populations that the samples come from.

The first type of variability we will explore comes from experiments where the explanatory variable (or treatment) is randomly assigned to the observational units. As you learned in Chapter 1, a randomized experiment can be used to assess whether one variable (the explanatory variable) causes changes in a second variable (the response variable). Every dataset has some variability in it, so to decide whether the variability in the data is due to (1) the causal mechanism (the randomized explanatory variable in the experiment) or instead (2) natural variability inherent to the data, we set up a sham randomized experiment as a comparison. That is, we assume that each observational unit would have gotten the exact same response value regardless of the treatment level. By reassigning the treatments many many times, we can compare the actual experiment to the sham experiment. If the actual experiment has more extreme results than any of the sham experiments, we are led to believe that it is the explanatory variable which is causing the result and not just variability inherent to the data. Using a few different case studies, let’s look more carefully at this idea of a randomization test.

11.1 Sex discrimination case study

We consider a study investigating sex discrimination in the 1970s, which is set in the context of personnel decisions within a bank. The research question we hope to answer is, “Are individuals who identify as female discriminated against in promotion decisions made by their managers who identify as male?” (Rosen and Jerdee 1974)

The sex_discrimination data can be found in the openintro R package.

This study considered sex roles, and only allowed for options of “male” and “female”. We should note that the identities being considered are not gender identities and that the study allowed only for a binary classification of sex.

11.1.1 Observed data

The participants in this study were 48 bank supervisors who identified as male, attending a management institute at the University of North Carolina in 1972. They were asked to assume the role of the personnel director of a bank and were given a personnel file to judge whether the person should be promoted to a branch manager position. The files given to the participants were identical, except that half of them indicated the candidate identified as male and the other half indicated the candidate identified as female. These files were randomly assigned to the bank managers.

Is this an observational study or an experiment? How does the type of study impact what can be inferred from the results?²

For each supervisor both the sex associated with the assigned file and the promotion decision were recorded. Using the results of the study summarized in Table 11.1, we would like to evaluate if individuals who identify as female are unfairly discriminated against in promotion decisions. In this study, a smaller proportion of female identifying applications were promoted than males (0.583 versus 0.875), but it is unclear whether the difference provides convincing evidence that individuals who identify as female are unfairly discriminated against.

Table 11.1: Summary results for the sex discrimination study.

	decision
sex	promoted	not promoted	Total
male	21	3	24
female	14	10	24
Total	35	13	48

The data are visualized in Figure 11.1 as a set of cards. Note that each card denotes a personnel file (an observation from our dataset) and the colors indicate the decision: red for promoted and white for not promoted. Additionally, the observations are broken up into groups of male and female identifying groups.

48 cards are laid out; 24 indicating male files, 24 indicated female files. Of the 24 male files 3 of the cards are colored white, and 21 of the cards are colored red. Of the female files, 10 of the cards are colored white, and 14 of the cards are colored red. — Figure 11.1: The sex discrimination study can be thought of as 48 red and white cards.

Statisticians are sometimes called upon to evaluate the strength of evidence. When looking at the rates of promotion in this study, why might we be tempted to immediately conclude that individuals identifying as female are being discriminated against?

The large difference in promotion rates (58.3% for female personnel versus 87.5% for male personnel) suggests there might be discrimination against women in promotion decisions. However, we cannot yet be sure if the observed difference represents discrimination or is just due to random chance when there is no discrimination occurring. Since we wouldn’t expect the sample proportions to be exactly equal, even if the truth was that the promotion decisions were independent of sex, we can’t rule out random chance as a possible explanation when simply comparing the sample proportions.

The previous example is a reminder that there will always be variability in data (making the groups differ), even if there are no underlying causes for that difference (e.g., even if there is no discrimination). Table 11.1 shows there were 7 fewer promotions for female identifying personnel than for the male personnel, a difference in promotion rates of 29.2% $\left( \frac{21}{24} - \frac{14}{24} = 0.292 \right).$ This observed difference is what we call a point estimate of the true difference. The point estimate of the difference in promotion rate is large, but the sample size for the study is small, making it unclear if the observed difference represents discrimination or is simply due to chance. Chance can be thought of as the claim due to natural variability; discrimination can be thought of as the claim the researchers set out to demonstrate. We label these two competing claims, $H_0$ and $H_A:$

$H_0:$ Null hypothesis. The variables sex and decision are independent. The difference in promotion rates of 29.2% was due to natural variability inherent in the population.
$H_A:$ Alternative hypothesis. The variables sex and decision are not independent. The difference in promotion rates of 29.2% was not due to natural variability, and equally qualified female personnel are less likely to be promoted than male personnel.

Hypothesis testing.

These hypotheses are part of what is called a hypothesis test. A hypothesis test is a statistical technique used to evaluate competing claims using data. Often times, the null hypothesis takes a stance of no difference or no effect. This hypothesis assumes that any differences observed are due to the variability inherent in the population and could have occurred by random chance.

If the null hypothesis and the data notably disagree, then we reject the null hypothesis in favor of the alternative hypothesis.

There are many nuances to hypothesis testing, so do not worry if you don’t feel like a master of hypothesis testing at the end of this section. We’ll discuss these ideas and details many times in this chapter as well as in the chapters that follow.

What would it mean if the null hypothesis, which says the variables sex and decision are unrelated, was true? It would mean each banker would decide whether to promote the candidate without regard to the sex indicated on the personnel file. That is, the difference in the promotion percentages would be due to the natural variability in how the files were randomly allocated to different bankers, and this randomization just happened to give rise to a relatively large difference of 29.2%.

Consider the alternative hypothesis: bankers were influenced by which sex was listed on the personnel file. If this was true, and especially if this influence was substantial, we would expect to see some difference in the promotion rates of male and female candidates. If this sex bias was against female candidates, we would expect a smaller fraction of promotion recommendations for female personnel relative to the male personnel.

We will choose between the two competing claims by assessing if the data conflict so much with $H_0$ that the null hypothesis cannot be deemed reasonable. If data and the null claim seem to be at odds with one another, and the data seem to support $H_A,$ then we will reject the notion of independence and conclude that the data provide evidence of discrimination.

11.1.2 Variability of the statistic

Table 11.1 shows that 35 bank supervisors recommended promotion and 13 did not. Now, suppose the bankers’ decisions were independent of the sex of the candidate. Then, if we conducted the experiment again with a different random assignment of sex to the files, differences in promotion rates would be based only on random fluctuation in promotion decisions. We can perform this randomization, which simulates what would have happened if the bankers’ decisions had been independent of sex but we had distributed the file sexes differently.³

In the simulation, we thoroughly shuffle the 48 personnel files, 35 labelled promoted and 13 labelled not promoted, together and we deal files into two new stacks. Note that by keeping 35 promoted and 13 not promoted, we are assuming that 35 of the bank managers would have promoted the individual whose content is contained in the file independent of the sex indicated on their file. We will deal 24 files into the first stack, which will represent the 24 “female” files. The second stack will also have 24 files, and it will represent the 24 “male” files. Figure 11.2 highlights both the shuffle and the reallocation to the sham sex groups.

The 48 red and white cards which denote the original data are shuffled and reassigned, 24 to each group indicating 24 male files and 24 female files. — Figure 11.2: The sex discrimination data are shuffled and reallocated to new groups of male and female files.

Then, as we did with the original data, we tabulate the results and determine the fraction of personnel files designated as “male” and “female” who were promoted.

Since the randomization of files in this simulation is independent of the promotion decisions, any difference in promotion rates is due to chance. Table 11.2 show the results of one such simulation.

Table 11.2: Simulation results, where the difference in promotion rates between male and female is purely due to random chance.

	decision
sex	promoted	not promoted	Total
male	18	6	24
female	17	7	24
Total	35	13	48

What is the difference in promotion rates between the two simulated groups in Table 11.2? How does this compare to the observed difference 29.2% from the actual study?⁴

Figure 11.3 shows that the difference in promotion rates is much larger in the original data than it is in the simulated groups (0.292 > 0.042). The quantity of interest throughout this case study has been the difference in promotion rates. We call the summary value the statistic of interest (or often the test statistic). When we encounter different data structures, the statistic is likely to change (e.g., we might calculate an average instead of a proportion), but we will always want to understand how the statistic varies from sample to sample.

The 48 red and white cards are show in three panels. The first panel represents the original data and original allocation of the male and female files (in the original data there are 3 white cards in the male group and 10 white cards in the female group). The second panel represents the shuffled red and white cards that are randomly assigned as male and female files. The third panel has the cards sorted according to the random assignment of female or male. In the third panel there are 6 white cards in the male group and 7 white cards in the female group. — Figure 11.3: We summarize the randomized data to produce one estimate of the difference in proportions given no sex discrimination. Note that the sort step is only used to make it easier to visually calculate the simulated sample proportions.

11.1.3 Observed statistic vs. null statistics

We computed one possible difference under the null hypothesis in Guided Practice, which represents one difference due to chance when the null hypothesis is assumed to be true. While in this first simulation, we physically dealt out files, it is much more efficient to perform this simulation using a computer. Repeating the simulation on a computer, we get another difference due to chance under the same assumption: -0.042. And another: 0.208. And so on until we repeat the simulation enough times that we have a good idea of the shape of the distribution of differences under the null hypothesis. Figure 11.4 shows a plot of the differences found from 100 simulations, where each dot represents a simulated difference between the proportions of male and female files recommended for promotion.

A stacked dot plot of the 100 simulated differences between the proportion of male and female files recommended for promotion. The differences were simulated under the null hypothesis that there was no discrimination. Two of the 100 simulations had a difference of 29.2% and are colored in blue to indicate that they are as or more extreme than the observed difference. — Figure 11.4: A stacked dot plot of differences from 100 simulations produced under the null hypothesis, $H_0,$ where the simulated sex and decision are independent. Two of the 100 simulations had a difference of at least 29.2%, the difference observed in the study, and are shown as solid blue dots.

Note that the distribution of these simulated differences in proportions is centered around 0. Under the null hypothesis our simulations made no distinction between male and female personnel files. Thus, a center of 0 makes sense: we should expect differences from chance alone to fall around zero with some random fluctuation for each simulation.

How often would you observe a difference of at least 29.2% (0.292) according to Figure 11.4? Often, sometimes, rarely, or never?

It appears that a difference of at least 29.2% under the null hypothesis would only happen about 2% of the time according to Figure 11.4. Such a low probability indicates that observing such a large difference from chance alone is rare.

The difference of 29.2% is a rare event if there really is no impact from listing sex in the candidates’ files, which provides us with two possible interpretations of the study results:

If $H_0,$ the Null hypothesis is true: Sex has no effect on promotion decision, and we observed a difference that is so large that it would only happen rarely.
If $H_A,$ the Alternative hypothesis is true: Sex has an effect on promotion decision, and what we observed was actually due to equally qualified female candidates being discriminated against in promotion decisions, which explains the large difference of 29.2%.

When we conduct formal studies, we reject a null position (the idea that the data are a result of chance only) if the data strongly conflict with that null position.⁵ In our analysis, we determined that there was only a $\approx$ 2% probability of obtaining a sample where $\geq$ 29.2% more male candidates than female candidates get promoted under the null hypothesis, so we conclude that the data provide strong evidence of sex discrimination against female candidates by the male supervisors. In this case, we reject the null hypothesis in favor of the alternative.

Statistical inference is the practice of making decisions and conclusions from data in the context of uncertainty. Errors do occur, just like rare events, and the dataset at hand might lead us to the wrong conclusion. While a given dataset may not always lead us to a correct conclusion, statistical inference gives us tools to control and evaluate how often these errors occur. Before getting into the nuances of hypothesis testing, let’s work through another case study.

11.2 Opportunity cost case study

How rational and consistent is the behavior of the typical American college student? In this section, we’ll explore whether college student consumers always consider the following: money not spent now can be spent later.

In particular, we are interested in whether reminding students about this well-known fact about money causes them to be a little thriftier. A skeptic might think that such a reminder would have no impact. We can summarize the two different perspectives using the null and alternative hypothesis framework.

$H_0:$ Null hypothesis. Reminding students that they can save money for later purchases will not have any impact on students’ spending decisions.
$H_A:$ Alternative hypothesis. Reminding students that they can save money for later purchases will reduce the chance they will continue with a purchase.

In this section, we’ll explore an experiment conducted by researchers that investigates this very question for students at a university in the southwestern United States. (Frederick et al. 2009)

11.2.1 Observed data

One-hundred and fifty students were recruited for the study, and each was given the following statement:

Imagine that you have been saving some extra money on the side to make some purchases, and on your most recent visit to the video store you come across a special sale on a new video. This video is one with your favorite actor or actress, and your favorite type of movie (such as a comedy, drama, thriller, etc.). This particular video that you are considering is one you have been thinking about buying for a long time. It is available for a special sale price of $14.99. What would you do in this situation? Please circle one of the options below.⁶

Half of the 150 students were randomized into a control group and were given the following two options:

Buy this entertaining video.

Not buy this entertaining video.

The remaining 75 students were placed in the treatment group, and they saw a slightly modified option (B):

Buy this entertaining video.

Not buy this entertaining video. Keep the $14.99 for other purchases.

Would the extra statement reminding students of an obvious fact impact the purchasing decision? Table 11.3 summarizes the study results.

The opportunity_cost data can be found in the openintro R package.

Table 11.3: Summary results of the opportunity cost study.

	decision
group	buy video	not buy video	Total
control	56	19	75
treatment	41	34	75
Total	97	53	150

It might be a little easier to review the results using a visualization. Figure 11.5 shows that a higher proportion of students in the treatment group chose not to buy the video compared to those in the control group.

Stacked bar plot with groups of control and treatment and filled using the proportion who did and did not buy the video. 74% of the control group bought the video as compared with a little over 50% of the treatment group who bought the video. — Figure 11.5: Stacked bar plot of results of the opportunity cost study.

Another useful way to review the results from Table 11.3 is using row proportions, specifically considering the proportion of participants in each group who said they would buy or not buy the video. These summaries are given in Table 11.4.

Table 11.4: The opportunity cost data are summarized using row proportions. Row proportions are particularly useful here since we can view the proportion of buy and not buy decisions in each group.

	decision
group	buy video	not buy video	Total
control	0.747	0.253	1
treatment	0.547	0.453	1

We will define a success in this study as a student who chooses not to buy the video.⁷ Then, the value of interest is the change in video purchase rates that results by reminding students that not spending money now means they can spend the money later.

We can construct a point estimate for this difference as ($T$ for treatment and $C$ for control):

\[\hat{p}_{T} - \hat{p}_{C} = \frac{34}{75} - \frac{19}{75} = 0.453 - 0.253 = 0.200\]

The proportion of students who chose not to buy the video was 20 percentage points higher in the treatment group than the control group. Is this 20% difference between the two groups so prominent that it is unlikely to have occurred from chance alone, if there is no difference between the spending habits of the two groups?

11.2.2 Variability of the statistic

The primary goal in this data analysis is to understand what sort of differences we might see if the null hypothesis were true, i.e., the treatment had no effect on students. Because this is an experiment, we’ll use the same procedure we applied in Section 11.1: randomization.

Let’s think about the data in the context of the hypotheses. If the null hypothesis $(H_0)$ was true and the treatment had no impact on student decisions, then the observed difference between the two groups of 20% could be attributed entirely to random chance. If, on the other hand, the alternative hypothesis $(H_A)$ is true, then the difference indicates that reminding students about saving for later purchases actually impacts their buying decisions.

11.2.3 Observed statistic vs. null statistics

Just like with the sex discrimination study, we can perform a statistical analysis. Using the same randomization technique from the last section, let’s see what happens when we simulate the experiment under the scenario where there is no effect from the treatment.

While we would in reality do this simulation on a computer, it might be useful to think about how we would go about carrying out the simulation without a computer. We start with 150 index cards and label each card to indicate the distribution of our response variable: decision. That is, 53 cards will be labeled “not buy video” to represent the 53 students who opted not to buy, and 97 will be labeled “buy video” for the other 97 students. Then we shuffle these cards thoroughly and divide them into two stacks of size 75, representing the simulated treatment and control groups. Because we have shuffled the cards from both groups together, assuming no difference in their purchasing behavior, any observed difference between the proportions of “not buy video” cards (what we earlier defined as success) can be attributed entirely to chance.

If we are randomly assigning the cards into the simulated treatment and control groups, how many “not buy video” cards would we expect to end up in each simulated group? What would be the expected difference between the proportions of “not buy video” cards in each group?

Since the simulated groups are of equal size, we would expect $53 / 2 = 26.5,$ i.e., 26 or 27, “not buy video” cards in each simulated group, yielding a simulated point estimate of the difference in proportions of 0%. However, due to random chance, we might also expect to sometimes observe a number a little above or below 26 and 27.

The results of a single randomization is shown in Table 11.5.

Table 11.5: Summary of student choices against their simulated groups. The group assignment had no connection to the student decisions, so any difference between the two groups is due to chance.

	decision
group	buy video	not buy video	Total
control	46	29	75
treatment	51	24	75
Total	97	53	150

From this table, we can compute a difference that occurred from the first shuffle of the data (i.e., from chance alone):

\[\hat{p}_{T, shfl1} - \hat{p}_{C, shfl1} = \frac{24}{75} - \frac{29}{75} = 0.32 - 0.387 = - 0.067\]

Just one simulation will not be enough to get a sense of what sorts of differences would happen from chance alone.

We’ll simulate another set of simulated groups and compute the new difference: 0.04.

And again: 0.12.

And again: -0.013.

We’ll do this 1,000 times.

The results are summarized in a dot plot in Figure 11.6, where each point represents the difference from one randomization.

A stacked dot plot of the 1000 simulated differences between the proportion of students who bought the video (treamtment minus control). The differences were simulated under the null hypothesis that there was no effect of the treatment. Six of the 1000 simulations had a difference of at least 20% and are colored in blue to indicate that they are as or more extreme than the observed difference. — Figure 11.6: A stacked dot plot of 1,000 simulated (null) differences produced under the null hypothesis, $H_0.$ Six of the 1,000 simulations had a difference of at least 20%, which was the difference observed in the study.

Since there are so many points and it is difficult to discern one point from the other, it is more convenient to summarize the results in a histogram such as the one in Figure 11.7, where the height of each histogram bar represents the number of simulations resulting in an outcome of that magnitude.

A histogram of the 1000 simulated differences between the proportion of students who bought the video (treamtment minus control). The differences were simulated under the null hypothesis that there was no effect of the treatment. Six of the 1000 simulations had a difference of at least 20% and their histogram bars are colored in blue to indicate that they are as or more extreme than the observed difference. A histogram is used instead of a dot plot as a more convenient representation of the data. — Figure 11.7: A histogram of 1,000 chance differences produced under the null hypothesis. Histograms like this one are a convenient representation of data or results when there are a large number of simulations.

Under the null hypothesis (no treatment effect), we would observe a difference of at least +20% about 0.6% of the time. That is really rare! Instead, we will conclude the data provide strong evidence there is a treatment effect: reminding students before a purchase that they could instead spend the money later on something else lowers the chance that they will continue with the purchase. Notice that we are able to make a causal statement for this study since the study is an experiment, although we do not know why the reminder induces a lower purchase rate.

11.3 Hypothesis testing

In the last two sections, we utilized a hypothesis test, which is a formal technique for evaluating two competing possibilities. In each scenario, we described a null hypothesis, which represented either a skeptical perspective or a perspective of no difference. We also laid out an alternative hypothesis, which represented a new perspective such as the possibility of a relationship between two variables or a treatment effect in an experiment. The alternative hypothesis is usually the reason the scientists set out to do the research in the first place.

Null and alternative hypotheses.

The null hypothesis $(H_0)$ often represents either a skeptical perspective or a claim of “no difference” to be tested.

The alternative hypothesis $(H_A)$ represents an alternative claim under consideration and is often represented by a range of possible values for the value of interest.

If a person makes a somewhat unbelievable claim, we are initially skeptical. However, if there is sufficient evidence that supports the claim, we set aside our skepticism. The hallmarks of hypothesis testing are also found in the US court system.

11.3.1 The US court system

In the US court system, jurors evaluate the evidence to see whether it convincingly shows a defendant is guilty. Defendants are considered to be innocent until proven otherwise.

The US court considers two possible claims about a defendant: they are either innocent or guilty.

If we set these claims up in a hypothesis framework, which would be the null hypothesis and which the alternative?

The jury considers whether the evidence is so convincing (strong) that there is no reasonable doubt regarding the person’s guilt. That is, the skeptical perspective (null hypothesis) is that the person is innocent until evidence is presented that convinces the jury that the person is guilty (alternative hypothesis).

Jurors examine the evidence to see whether it convincingly shows a defendant is guilty. Notice that if a jury finds a defendant not guilty, this does not necessarily mean the jury is confident in the person’s innocence. They are simply not convinced of the alternative, that the person is guilty. This is also the case with hypothesis testing: even if we fail to reject the null hypothesis, we do not accept the null hypothesis as truth.

Failing to find evidence in favor of the alternative hypothesis is not equivalent to finding evidence that the null hypothesis is true. We will see this idea in greater detail in Chapter 14.

11.3.2 p-value and statistical discernibility

In Section 11.1 we encountered a study from the 1970’s that explored whether there was strong evidence that female candidates were less likely to be promoted than male candidates. The research question – are female candidates discriminated against in promotion decisions? – was framed in the context of hypotheses:

$H_0:$ Sex has no effect on promotion decisions.
$H_A:$ Female candidates are discriminated against in promotion decisions.

The null hypothesis $(H_0)$ was a perspective of no difference in promotion. The data on sex discrimination provided a point estimate of a 29.2% difference in recommended promotion rates between male and female candidates. We determined that such a difference from chance alone, assuming the null hypothesis was true, would be rare: it would only happen about 2 in 100 times. When results like these are inconsistent with $H_0,$ we reject $H_0$ in favor of $H_A.$ Here, we concluded there was discrimination against female candidates.

The 2-in-100 chance is what we call a p-value, which is a probability quantifying the strength of the evidence against the null hypothesis, given the observed data.

p-value.

The p-value is the probability of observing data at least as favorable to the alternative hypothesis as our current dataset, if the null hypothesis were true. We typically use a summary statistic of the data, such as a difference in proportions, to help compute the p-value and evaluate the hypotheses. This summary value that is used to compute the p-value is often called the test statistic.

In the sex discrimination study, the difference in discrimination rates was our test statistic. What was the test statistic in the opportunity cost study covered in Section 11.2)?

The test statistic in the opportunity cost study was the difference in the proportion of students who decided against the video purchase in the treatment and control groups. In each of these examples, the point estimate of the difference in proportions was used as the test statistic.

When the p-value is small, i.e., less than a previously set threshold, we say the results are statistically discernible. This means the data provide such strong evidence against $H_0$ that we reject the null hypothesis in favor of the alternative hypothesis.⁸ The threshold is called the discernibility level and often represented by $\alpha$ (the Greek letter alpha). ⁹ The value of $\alpha$ represents how rare an event needs to be in order for the null hypothesis to be rejected. Historically, many fields have set $\alpha = 0.05,$ if the null hypothesis is to be rejected. The value of $\alpha$ can vary depending on the the field or the application.

Note that you may have heard the phrase “statistically significant” as a way to describe “statistically discernible.” Although in everyday language “significant” would indicate that a difference is large or meaningful, that is not necessarily the case here. The term “statistically discernible” indicates that the p-value from a study fell below the chosen discernibility level. For example, in the sex discrimination study, the p-value was found to be approximately 0.02. Using a discernibility level of $\alpha = 0.05,$ we would say that the data provided statistically discernible evidence against the null hypothesis. However, this conclusion gives us no information regarding the size of the difference in promotion rates!

Statistical discernibility.

We say that the data provide statistically discernible evidence against the null hypothesis if the p-value is less than some predetermined threshold (e.g., 0.01, 0.05, 0.1).

In the opportunity cost study in Section 11.2, we analyzed an experiment where study participants had a 20% drop in likelihood of continuing with a video purchase if they were reminded that the money, if not spent on the video, could be used for other purchases in the future. We determined that such a large difference would only occur 6-in-1,000 times if the reminder actually had no influence on student decision-making. What is the p-value in this study? Would you classify the result as “statistically discernible”?

The p-value was 0.006. Since the p-value is less than 0.05, the data provide statistically discernible evidence that US college students were actually influenced by the reminder.

What’s so special about 0.05?

We often use a threshold of 0.05 to determine whether a result is statistically discernible. But why 0.05? Maybe we should use a bigger number, or maybe a smaller number. If you’re a little puzzled, that probably means you’re reading with a critical eye – good job! We’ve made a video to help clarify why 0.05:

https://www.openintro.org/book/stat/why05/

Sometimes it’s also a good idea to deviate from the standard. We’ll discuss when to choose a threshold different than 0.05 in Chapter 14.

11.4 Chapter review

11.4.1 Summary

Figure 11.8 provides a visual summary of the randomization testing procedure.

48 red and white cards are show in three panels. The first panel represents original data and original allocation of Group 1 and Group 2 (in the original data there are 7 white cards in Group 1 and 10 white cards in Group 2). The second panel represents the shuffled red and white cards that are randomly assigned as Group 1 and Group 2. The third panel has the cards sorted according to the random assignment of Group 1 and Group 2. In the third panel there are 8 white cards in the Group 1 and 9 white cards in Group 2. — Figure 11.8: An example of one simulation of the full randomization procedure from a hypothetical dataset as visualized in the first panel. We repeat the steps hundreds or thousands of times.

We can summarize the randomization test procedure as follows:

Frame the research question in terms of hypotheses. Hypothesis tests are appropriate for research questions that can be summarized in two competing hypotheses. The null hypothesis $(H_0)$ usually represents a skeptical perspective or a perspective of no relationship between the variables. The alternative hypothesis $(H_A)$ usually represents a new view or the existence of a relationship between the variables.
Collect data with an observational study or experiment. If a research question can be formed into two hypotheses, we can collect data to run a hypothesis test. If the research question focuses on associations between variables but does not concern causation, we would use an observational study. If the research question seeks a causal connection between two or more variables, then an experiment should be used.
Model the randomness that would occur if the null hypothesis was true. In the examples above, the variability has been modeled as if the treatment (e.g., sexual identity, opportunity) allocation was independent of the outcome of the study. The computer generated null distribution is the result of many different randomizations and quantifies the variability that would be expected if the null hypothesis was true.
Analyze the data. Choose an analysis technique appropriate for the data and identify the p-value. So far, we have only seen one analysis technique: randomization. Throughout the rest of this textbook, we’ll encounter several new methods suitable for many other contexts.
Form a conclusion. Using the p-value from the analysis, determine whether the data provide evidence against the null hypothesis. Also, be sure to write the conclusion in plain language so casual readers can understand the results.

Table 11.6 is another look at the randomization test summary.

Table 11.6: Summary of randomization as an inferential statistical method.

Question	Answer
What does it do?	Shuffles the explanatory variable to mimic the natural variability found in a randomized experiment
What is the random process described?	Randomized experiment
What other random processes can be approximated?	Can also be used to describe random sampling in an observational model
What is it best for?	Hypothesis testing (can also be used for confidence intervals, but not covered in this text)
What physical object represents the simulation process?	Shuffling cards

11.4.2 Terms

The terms introduced in this chapter are presented in Table 11.7. If you’re not sure what some of these terms mean, we recommend you go back in the text and review their definitions. You should be able to easily spot them as bolded text.

Table 11.7: Terms introduced in this chapter.

alternative hypothesis	permutation test	statistical inference
discernibility level	point estimate	statistically discernible
hypothesis test	randomization test	statistically significant
independent	significance level	success
null hypothesis	simulation	test statistic
p-value	statistic

11.5 Exercises

Answers to odd-numbered exercises can be found in Appendix A.11.

Identify the parameter, I. For each of the following situations, state whether the parameter of interest is a mean or a proportion. It may be helpful to examine whether individual responses are numerical or categorical.
1. In a survey, 100 college students are asked how many hours per week they spend on the Internet.
2. In a survey, 100 college students are asked: “What percentage of the time you spend on the Internet is part of your course work?”
3. In a survey, 100 college students are asked whether they cited information from Wikipedia in their papers.
4. In a survey, 100 college students are asked what percentage of their total weekly spending is on alcoholic beverages.
5. In a sample of 100 recent college graduates, it is found that 85 percent expect to get a job within one year of their graduation date.

Identify the parameter, II. For each of the following situations, state whether the parameter of interest is a mean or a proportion.
1. A poll shows that 64% of Americans personally worry a great deal about federal spending and the budget deficit.
2. A survey reports that local TV news has shown a 17% increase in revenue within a two year period while newspaper revenues decreased by 6.4% during this time period.
3. In a survey, high school and college students are asked whether they use geolocation services on their smart phones.
4. In a survey, smart phone users are asked whether they use a web-based taxi service.
5. In a survey, smart phone users are asked how many times they used a web-based taxi service over the last year.

Hypotheses. For each of the research statements below, note whether it represents a null hypothesis claim or an alternative hypothesis claim.
1. The number of hours that grade-school children spend doing homework predicts their future success on standardized tests.
2. King cheetahs on average run the same speed as standard spotted cheetahs.
3. For a particular student, the probability of correctly answering a 5-option multiple choice test is larger than 0.2 (i.e., better than guessing).
4. The mean length of African elephant tusks has changed over the last 100 years.
5. The risk of facial clefts is equal for babies born to mothers who take folic acid supplements compared with those from mothers who do not.
6. Caffeine intake during pregnancy affects mean birth weight.
7. The probability of getting in a car accident is the same if using a cell phone than if not using a cell phone.

True null hypothesis. Unbeknownst to you, let’s say that the null hypothesis is actually true in the population. You plan to run a study anyway.
1. If the level of discernibility you choose (i.e., the cutoff for your p-value) is 0.05, how likely is it that you will mistakenly reject the null hypothesis?
2. If the level of discernibility you choose (i.e., the cutoff for your p-value) is 0.01, how likely is it that you will mistakenly reject the null hypothesis?
3. If the level of discernibility you choose (i.e., the cutoff for your p-value) is 0.10, how likely is it that you will mistakenly reject the null hypothesis?

Identify hypotheses, I. Write the null and alternative hypotheses in words and then symbols for each of the following situations.
1. New York is known as “the city that never sleeps”. A random sample of 25 New Yorkers were asked how much sleep they get per night. Do these data provide convincing evidence that New Yorkers on average sleep less than 8 hours a night?
2. Employers at a firm are worried about the effect of March Madness, a basketball championship held each spring in the US, on employee productivity. They estimate that on a regular business day employees spend on average 15 minutes of company time checking personal email, making personal phone calls, etc. They also collect data on how much company time employees spend on such non- business activities during March Madness. They want to determine if these data provide convincing evidence that employee productivity decreases during March Madness.

Identify hypotheses, II. Write the null and alternative hypotheses in words and using symbols for each of the following situations.
1. Since 2008, chain restaurants in California have been required to display calorie counts of each menu item. Prior to menus displaying calorie counts, the average calorie intake of diners at a restaurant was 1100 calories. After calorie counts started to be displayed on menus, a nutritionist collected data on the number of calories consumed at this restaurant from a random sample of diners. Do these data provide convincing evidence of a difference in the average calorie intake of a diners at this restaurant?
2. Based on the performance of those who took the GRE exam between July 1, 2004 and June 30, 2007, the average Verbal Reasoning score was calculated to be 462. In 2021 the average verbal score was slightly higher. Do these data provide convincing evidence that the average GRE Verbal Reasoning score has changed since 2021?

Side effects of Avandia. Rosiglitazone is the active ingredient in the controversial type 2 diabetes medicine Avandia and has been linked to an increased risk of serious cardiovascular problems such as stroke, heart failure, and death. A common alternative treatment is Pioglitazone, the active ingredient in a diabetes medicine called Actos. In a nationwide retrospective observational study of 227,571 Medicare beneficiaries aged 65 years or older, it was found that 2,593 of the 67,593 patients using Rosiglitazone and 5,386 of the 159,978 using Pioglitazone had serious cardiovascular problems. These data are summarized in the contingency table below.¹⁰ (Graham et al. 2010)

Treatment No Yes Total

Pioglitazone 154,592 5,386 159,978

Rosiglitazone 65,000 2,593 67,593

Total 219,592 7,979 227,571
1. Determine if each of the following statements is true or false. If false, explain why. Be careful: The reasoning may be wrong even if the statement’s conclusion is correct. In such cases, the statement should be considered false.
  1. Since more patients on Pioglitazone had cardiovascular problems (5,386 vs. 2,593), we can conclude that the rate of cardiovascular problems for those on a Pioglitazone treatment is higher.
  2. The data suggest that diabetic patients who are taking Rosiglitazone are more likely to have cardiovascular problems since the rate of incidence was (2,593 / 67,593 = 0.038) 3.8% for patients on this treatment, while it was only (5,386 / 159,978 = 0.034) 3.4% for patients on Pioglitazone.
  3. The fact that the rate of incidence is higher for the Rosiglitazone group proves that Rosiglitazone causes serious cardiovascular problems.
  4. Based on the information provided so far, we cannot tell if the difference between the rates of incidences is due to a relationship between the two variables or due to chance.
2. What proportion of all patients had cardiovascular problems?
3. If the type of treatment and having cardiovascular problems were independent, how many patients in the Rosiglitazone group would we expect to have had cardiovascular problems?
4. We can investigate the relationship between outcome and treatment in this study using a randomization technique. While in reality we would carry out the simulations required for randomization using statistical software, suppose we actually simulate using index cards. In order to simulate from the independence model, which states that the outcomes were independent of the treatment, we write whether each patient had a cardiovascular problem on cards, shuffled all the cards together, then deal them into two groups of size 67,593 and 159,978. We repeat this simulation 100 times and each time record the difference between the proportions of cards that say “Yes” in the Rosiglitazone and Pioglitazone groups. Use the histogram of these differences in proportions to answer the following questions.
  1. What are the claims being tested?
  2. Compared to the number calculated in part (b), which would provide more support for the alternative hypothesis, higher or lower proportion of patients with cardiovascular problems in the Rosiglitazone group?
  3. What do the simulation results suggest about the relationship between taking Rosiglitazone and having cardiovascular problems in diabetic patients?

Treatment	No	Yes	Total
Pioglitazone	154,592	5,386	159,978
Rosiglitazone	65,000	2,593	67,593
Total	219,592	7,979	227,571

Heart transplants. The Stanford University Heart Transplant Study was conducted to determine whether an experimental heart transplant program increased lifespan. Each patient entering the program was designated an official heart transplant candidate, meaning that they were gravely ill and would most likely benefit from a new heart. Some patients got a transplant and some did not. The variable transplant indicates which group the patients were in; patients in the treatment group got a transplant and those in the control group did not. Of the 34 patients in the control group, 30 died. Of the 69 people in the treatment group, 45 died. Another variable called survived was used to indicate whether the patient was alive at the end of the study.¹¹ (Turnbull et al. 1974)
1. Does the stacked bar plot indicate that survival is independent of whether the patient got a transplant? Explain your reasoning.
2. What do the box plots suggest about the efficacy of heart transplants.
3. What proportions of patients in the treatment and control groups died?
4. One approach for investigating whether the treatment is discernably effective is randomization testing.
  1. What are the claims being tested?
  2. The paragraph below describes the set up for a randomization test, if we were to do it without using statistical software. Fill in the blanks with a number or phrase.
  We write alive on $\rule{1.25cm}{0.5pt}$ cards representing patients who were alive at the end of the study, and deceased on $\rule{1.25cm}{0.5pt}$ cards representing patients who were not. Then, we shuffle these cards and split them into two groups: one group of size $\rule{1.25cm}{0.5pt}$ representing treatment, and another group of size $\rule{1.25cm}{0.5pt}$ representing control. We calculate the difference between the proportion of cards in the treatment and control groups (treatment - control) and record this value. We repeat this 100 times to build a distribution centered at $\rule{1.25cm}{0.5pt}$. Lastly, we calculate the proportion of simulations where the simulated differences in proportions are $\rule{1.25cm}{0.5pt}$. If this proportion is low, we conclude that it is unlikely to have observed such an outcome by chance and that the null hypothesis should be rejected in favor of the alternative.
  1. What do the simulation results shown below suggest about the effectiveness of heart transplants?

We would be assuming that these two variables are independent.↩︎
The study is an experiment, as subjects were randomly assigned a “male” file or a “female” file (remember, all the files were actually identical in content). Since this is an experiment, the results can be used to evaluate a causal relationship between the sex of a candidate and the promotion decision.↩︎
The test procedure we employ in this section is sometimes referred to as a randomization test. If the explanatory variable had not been randomly assigned, as in an observational study, the procedure would be referred to as a permutation test. Permutation tests are used for observational studies, where the explanatory variable was not randomly assigned.↩︎
$18/24 - 17/24=0.042$ or about 4.2% in favor of the male personnel. This difference due to chance is much smaller than the difference observed in the actual groups.↩︎
This reasoning does not generally extend to anecdotal observations. Each of us observes incredibly rare events every day, events we could not possibly hope to predict. However, in the non-rigorous setting of anecdotal evidence, almost anything may appear to be a rare event, so the idea of looking for rare events in day-to-day activities is treacherous. For example, we might look at the lottery: there was only a 1 in 176 million chance that the Mega Millions numbers for the largest jackpot in history (October 23, 2018) would be (5, 28, 62, 65, 70) with a Mega ball of (5), but nonetheless those numbers came up! However, no matter what numbers had turned up, they would have had the same incredibly rare odds. That is, any set of numbers we could have observed would ultimately be incredibly rare. This type of situation is typical of our daily lives: each possible event in itself seems incredibly rare, but if we consider every alternative, those outcomes are also incredibly rare. We should be cautious not to misinterpret such anecdotal evidence.↩︎
This context might feel strange if physical video stores predate you. If you’re curious about what those were like, look up “Blockbuster”.↩︎
Success is often defined in a study as the outcome of interest, and a “success” may or may not actually be a positive outcome. For example, researchers working on a study on COVID prevalence might define a “success” in the statistical sense as a patient who has COVID-19. A more complete discussion of the term success will be given in Chapter 16.↩︎
Many texts use the phrase “statistically significant” instead of “statistically discernible”. We have chosen to use “discernible” to indicate that a precise statistical event has happened, as opposed to a notable effect which may or may not fit the statistical definition of discernible or significant.↩︎
Here, too, we have chosen “discernibility level” instead of “significance level” which you will see in some texts.↩︎
The avandia data used in this exercise can be found in the openintro R package.↩︎
The heart_transplant data used in this exercise can be found in the openintro R package.↩︎

中文

11 基于随机化的假设检验

统计推断主要关注理解和量化参数估计的不确定性。虽然方程和细节会因具体情境而变化，但推断的基础在整个统计学中都是相同的。

我们从两个案例研究开始，旨在说明对研究结论做出决策的过程。我们通过引入 假设检验框架将这一过程形式化，它使我们能够对关于总体的结论进行正式评估。

到目前为止，在本书中你已经在各种情境下处理过数据。你已经学会了如何汇总和可视化数据，以及如何同时对多个变量进行建模。有时，手头的数据集本身就代表了整个研究问题。但更常见的情况是，收集数据是为了回答关于一个更大群体的研究问题，而数据只是该群体的一个（希望具有代表性的）子集。

你可能会同意，数据中几乎总是存在变异性——即使两个数据集是用相同的方法从同一个总体中收集的，它们也不会完全相同。然而，量化数据中的变异性既不直观也不容易，也就是说，回答“如何一个数据集与另一个数据集有多不同？”这个问题并非易事。

首先，关于符号的说明。我们通常用 $p$ 表示总体比例，用 $\hat{p}$ 表示样本比例。类似地，我们通常用 $\mu$ 表示总体均值，用 $\bar{x}$ 表示样本均值。

假设你的教授将班上的学生分成两组：坐在教室左边的学生和坐在教室右边的学生。如果 $\hat{p}_{L}$ 表示坐在教室左边且更喜欢在屏幕上阅读书籍的学生比例， $\hat{p}_{R}$ 表示坐在教室右边且更喜欢在屏幕上阅读书籍的学生比例，如果 $\hat{p}_{L}$ 不恰好等于 $\hat{p}_{R}$?

，你会感到惊讶吗？ $\hat{p}_{L}$ 和 $\hat{p}_{R}$ 虽然比例 之间可能彼此接近，但它们完全相同是不太寻常的。我们可能会观察到由于.

偶然因素¹

造成的小差异。

如果我们认为一个人在教室里坐的位置与他们是否更喜欢在屏幕上阅读书籍无关，那么我们对这两个变量之间的关系做了什么假设？第 1研究这种形式的随机性是统计学的一个核心焦点。在本章及后续章节中，我们提供了三种不同的方法来量化数据中固有的变异性：随机化、自助法和数学模型。使用本章提供的方法，我们将能够超越当前数据集得出结论，以回答关于样本所来自的更大总体的研究问题。 随机化检验.

11.1 中学到的，随机化实验可用于评估一个变量（解释变量）是否引起另一个变量（响应变量）的变化。每个数据集都有一定的变异性，因此为了判断数据中的变异性是由于 (1) 因果机制（实验中随机化的解释变量），还是 (2) 数据固有的自然变异性，我们设置一个虚假的随机化实验作为比较。也就是说，我们假设每个观测单元无论处于哪个处理水平都会得到完全相同的响应值。通过多次重复重新分配处理，我们可以将实际实验与虚假实验进行比较。如果实际实验的结果比任何虚假实验的结果都更极端，我们就会认为结果是解释变量造成的，而不仅仅是数据固有的变异性。通过几个不同的案例研究，让我们更仔细地看看这个

随机化检验（randomization test）的概念。 (Rosen 和 Jerdee 1974)

该 sex_discrimination 数据可以在 openintro R 包中找到。

这项研究考虑的是生理性别，只提供了“男性”和“女性”两个选项。需要注意的是，这里考虑的身份并不是性别认同，而且该研究只允许对生理性别进行二元分类。

11.1.1 观测数据

这项研究的参与者是48名自我认同为男性的银行主管，他们于1972年参加北卡罗来纳大学的管理学院培训。他们被要求扮演银行的人事总监，并获得一份人事档案，需要判断该员工是否应被晋升为分行经理。发给参与者的人事档案完全相同，唯一的区别是一半档案标明候选人自我认同为男性，另一半标明候选人自我认同为女性。这些档案被随机分配给各银行主管。

这是一项观察性研究还是实验研究？研究类型如何影响我们从结果中可以做出的推断？²

对于每位主管，研究者记录了所分配档案对应的性别以及晋升决定。利用表 11.1中总结的研究结果，我们想评估自我认同为女性的个体在晋升决定中是否受到不公平的歧视。在这项研究中，自我认同为女性的申请者获得晋升的比例低于男性（0.583 对 0.875），但目前尚不清楚这一差异是否提供了自我认同为女性者受到不公平歧视的 令人信服的证据 。

表 11.1：性别歧视研究的汇总结果。

	决策
sex	晋升	未晋升	总计
男性	21	3	24
女性	14	10	24
总计	35	13	48

数据在图 11.1 中以一组卡片的形式呈现。请注意，每张卡片代表一份人事档案（我们数据集中的一个观测值），颜色表示决定：红色表示获得晋升，白色表示未获晋升。此外，观测值被分为自我认同为男性和女性的两组。

统计学家有时会被要求评估证据的强度。在观察本研究中的晋升率时，为什么我们可能会倾向于立即得出结论，认为自我认同为女性的人受到了歧视？

晋升率的巨大差异（女性人员为58.3%，男性人员为87.5%）表明在晋升决策中可能存在对女性的歧视。然而，我们还不能确定观察到的差异是代表了歧视，还是仅仅由于在不存在歧视的情况下偶然的随机因素造成的。因为即使事实是晋升决策与性别无关，我们也不会期望样本比例恰好完全相等，所以在仅仅比较样本比例时，我们不能排除随机偶然因素作为一种可能的解释。

前面的例子提醒我们，数据中总是存在变异性（使各组之间产生差异），即使这种差异没有潜在的原因（例如，即使不存在歧视）。表 11.1 显示女性人员的晋升人数比男性人员少7人，晋升率差异为29.2% $\left( \frac{21}{24} - \frac{14}{24} = 0.292 \right).$ 这个观察到的差异就是我们所称的真实差异的 点估计 。晋升率差异的点估计值很大，但研究的样本量很小，因此不清楚观察到的差异是代表了歧视，还是仅仅由于偶然因素造成的。偶然因素可以被视为归因于自然变异性的主张；歧视可以被视为研究人员试图证明的主张。我们给这两个相互竞争的主张贴上标签， $H_0$ 和 $H_A:$

$H_0:$ 原假设。变量 sex 和 decision 是独立的。29.2%的晋升率差异是由总体中固有的自然变异性造成的。
$H_A:$ 备择假设。变量 sex 和 decision 是不独立的。29.2%的晋升率差异并非由自然变异性造成，且条件相同的女性人员比男性人员更不可能获得晋升。

假设检验。

这些假设是所谓的 假设检验。假设检验是一种利用数据来评估相互竞争的主张的统计技术。通常，原假设采取的立场是 无差异 或 无效应。该假设假定任何观察到的差异都是由于总体中固有的变异性所致，可能是随机偶然产生的。

如果原假设与数据明显不符，我们就拒绝原假设，转而支持备择假设。

假设检验有许多细微之处，所以即使在本节结束时你并不觉得自己掌握了假设检验，也不必担心。在本章以及后续章节中，我们会多次讨论这些概念和细节。

如果原假设——即变量 sex 和 decision 之间不相关——为真，这意味着什么？这意味着每位银行主管在决定是否晋升候选人时，完全不考虑人事档案中标注的性别。也就是说，晋升百分比的差异是由于档案被随机分配给不同银行主管时产生的自然变异性，而这种随机化恰好产生了29.2%这一相对较大的差异。

考虑备择假设：银行主管受到了人事档案中所列性别的影响。如果这是真的，尤其是当这种影响相当大时，我们会预期男性和女性候选人的晋升率存在一定差异。如果这种性别偏见是针对女性候选人的，我们会预期女性人事档案获得晋升推荐的比例低于男性人事档案。

我们将通过评估数据与 $H_0$ 的冲突程度是否大到使原假设无法被认为是合理的，来在两个相互竞争的主张之间做出选择。如果数据与原假设主张似乎相互矛盾，而数据似乎支持 $H_A,$ ，那么我们将拒绝独立性的观点，并得出结论：数据提供了歧视的证据。

11.1.2 统计量的变异性

表 11.1 显示，35位银行主管推荐晋升，13位未推荐。现在，假设银行主管的决定与候选人的性别无关。那么，如果我们以不同的性别随机分配方式重新进行该实验，晋升率的差异将仅源于晋升决策的随机波动。我们可以进行这种 随机化，它模拟了如果银行主管的决定与 sex 无关、但我们以不同方式分配档案性别时会发生的情况。³

在模拟中，我们将 48 份人事档案彻底混洗，其中 35 份标记为 promoted ，13 份标记为 not promoted，然后将档案分成两叠新的档案堆。注意，通过保留 35 份晋升和 13 份未晋升，我们假设有 35 位银行经理会晋升档案独立中所标示性别的那位内容对应的人。我们将把 24 份档案放入第一叠，代表 24 份“女性”档案。第二叠也将有 24 份档案，代表 24 份“男性”档案。图 11.2 展示了混洗以及重新分配到虚假性别组的过程。

然后，正如我们对原始数据所做的那样，我们汇总结果，并确定被指定为“男性”和“女性”的人事档案中获得晋升的比例。

由于此模拟中档案的随机化与晋升决定相互独立，因此晋升率的任何差异都是偶然造成的。表 11.2 展示了其中一次模拟的结果。

表 11.2：模拟结果，其中男性和女性晋升率之间的差异纯粹是由随机机会造成的。

	决策
sex	晋升	未晋升	总计
男性	18	6	24
女性	17	7	24
总计	35	13	48

在表 11.2中，两个模拟组的晋升率差异是多少？这与实际研究中观察到的 29.2% 的差异相比如何？⁴

图 11.3 表明，原始数据中的晋升率差异远大于模拟组中的差异（0.292 > 0.042）。在本案例研究中，我们一直关注的量是晋升率的差异。我们将该汇总值称为感兴趣的 统计量 （或通常称为 检验统计量）。当我们遇到不同的数据结构时，统计量可能会发生变化（例如，我们可能计算平均值而不是比例），但我们始终希望了解统计量在样本之间的变化情况。

11.1.3 观测统计量与零假设统计量的对比

我们在引导练习中计算了原假设下的一个可能差异，它表示在假定原假设为真时由偶然造成的一个差异。虽然在第一次模拟中我们是实际分发文件的，但使用计算机进行模拟要高效得多。在计算机上重复模拟，我们得到在相同假设下由偶然造成的另一个差异：-0.042。再一个：0.208。如此继续，直到我们重复模拟足够多次，从而对原假设下 差异分布 的形状有一个很好的了解。图 11.4 展示了 100 次模拟所得差异的图，其中每个点代表一次模拟中推荐晋升的男性文件比例与女性文件比例之间的模拟差异。

请注意，这些模拟比例差异的分布以 0 为中心。在原假设下，我们的模拟对男性和女性人事文件没有做任何区分。因此，以 0 为中心是合理的：我们应该预期仅由偶然造成的差异在零附近波动，每次模拟都有一些随机波动。

根据图 11.4，你观察到至少 64.3%（0.643）差异的频率是多少？是经常、有时、很少，还是从不？

，你观察到至少 29.2%（0.292）差异的频率是多少？根据图 11.4。如此低的概率表明，仅凭偶然观察到如此大的差异是很罕见的。

如果候选人档案中列出性别确实没有影响，那么29.2%的差异就是一个罕见事件，这为研究结果提供了两种可能的解释：

如果 $H_0,$ 这 原假设 为真：性别对晋升决定没有影响，而我们观察到的差异如此之大，以至于它只会罕见地发生。
如果 $H_A,$ 这 备择假设 为真：性别对晋升决定有影响，而我们观察到的现象实际上是由于资历相同的女性候选人在晋升决定中受到歧视，这解释了29.2%的巨大差异。

当我们进行正式研究时，如果数据与零假设（即数据仅是偶然结果的观点）强烈冲突，我们就拒绝零假设。⁵ 在我们的分析中，我们确定在零假设下， $\approx$ 获得男性候选人比女性候选人晋升比例高出29.2%的样本的概率仅为 $\geq$ 2%，因此我们得出结论：数据提供了强有力的证据，表明男性主管在晋升决定中存在对女性候选人的性别歧视。在这种情况下，我们拒绝零假设而支持备择假设。

统计推断 是在不确定性背景下根据数据做出决策和结论的做法。错误确实会发生，就像罕见事件一样，手头的数据集可能会引导我们得出错误的结论。虽然给定的数据集并不总能引导我们得出正确的结论，但统计推断为我们提供了控制和评估这些错误发生频率的工具。在深入探讨假设检验的细微之处之前，让我们先研究另一个案例。

11.2 机会成本案例研究

典型的美国大学生的行为有多理性、多一致？在本节中，我们将探讨大学生消费者是否总是考虑以下问题：现在没有花掉的钱以后可以花。

我们特别感兴趣的是，提醒学生这个关于金钱的众所周知的事实，是否会让他们变得更节俭一些。怀疑者可能认为这样的提醒不会产生任何影响。我们可以使用原假设和备择假设的框架来总结这两种不同的观点。

$H_0:$ 原假设。提醒学生他们可以为以后的购买省钱，这不会对学生的消费决策产生任何影响。
$H_A:$ 备择假设。提醒学生他们可以为以后的购买省钱，会降低他们继续购买的可能性。

在本节中，我们将探讨研究人员进行的一项实验，该实验针对美国西南部一所大学的学生研究了这个问题。 (Frederick et al. 2009)

11.2.1 观测数据

该研究招募了150名学生，并给每位学生以下陈述：

想象一下，你一直在额外存一些钱用于购物，在你最近一次去音像店时，你碰巧遇到一张新录像带的特价促销。这张录像带的主演是你最喜欢的演员，类型也是你最喜欢的电影类型（如喜剧、剧情片、惊悚片等）。你正在考虑的这张录像带是你已经想买很久的。它现在特价14.99美元。在这种情况下你会怎么做？请圈出下面的一个选项。⁶

150名学生中有一半被随机分配到对照组，并得到以下两个选项：

购买这张有趣的录像带。

不购买这张有趣的录像带。

其余75名学生被分配到处理组，他们看到的选项（B）略有修改：

购买这张有趣的录像带。

不购买这张有趣的录像带。把14.99美元留作其他消费。

提醒学生一个显而易见事实的额外陈述会影响购买决策吗？表 11.3 总结了研究结果。

该 opportunity_cost 数据可以在 openintro R 包中找到。

表 11.3：机会成本研究的汇总结果。

	决策
组别	购买视频	不购买视频	总计
控制	56	19	75
治疗	41	34	75
总计	97	53	150

使用可视化图表来查看结果可能会更容易一些。图 11.5 图中显示，处理组中选择不购买视频的学生比例高于对照组。

查看表 11.3 研究结果的另一种有用方法是使用行比例，特别是考虑每组中表示会购买或不购买视频的参与者比例。这些汇总数据见表 11.4.

。 表 11.4：使用行比例汇总的机会成本数据。行比例在这里特别有用，因为我们可以查看每组中 和购买不购买

	决策
组别	购买视频	不购买视频	总计
控制	0.747	0.253	1
治疗	0.547	0.453	1

在本研究中，我们将把成功定义为一个选择不购买视频的学生。⁷ 然后，我们关注的数值是提醒学生“现在不花钱意味着以后可以花钱”所导致的视频购买率的变化。

我们可以为这一差异构建一个点估计（$T$ 为处理组， $C$ 为对照组）：

\[\hat{p}_{T} - \hat{p}_{C} = \frac{34}{75} - \frac{19}{75} = 0.453 - 0.253 = 0.200\]

选择不购买视频的学生比例在处理组比对照组高20个百分点。如果两组的消费习惯没有差异，那么这两组之间20%的差异是否足够显著，以至于不太可能仅由偶然因素造成？

11.2.2 统计量的变异性

这项数据分析的主要目标是了解如果原假设为真，即处理对学生没有影响，我们可能会看到什么样的差异。由于这是一个实验，我们将使用在第11.1节中应用的相同方法：随机化。

让我们在假设的背景下思考这些数据。如果原假设 $(H_0)$ 为真且处理对学生的决策没有影响，那么两组之间观察到的20%的差异可以完全归因于随机偶然。另一方面，如果备择假设 $(H_A)$ 为真，那么这一差异表明，提醒学生为以后的消费存钱实际上会影响他们的购买决策。

11.2.3 观测统计量与零假设统计量的对比

就像性别歧视研究一样，我们可以进行统计分析。使用上一节中相同的随机化技术，让我们看看在处理没有效果的情形下模拟实验时会发生什么。

虽然实际上我们会在计算机上进行这种模拟，但思考一下如何在没有计算机的情况下进行模拟可能会很有用。我们从150张索引卡片开始，并在每张卡片上标注以表示响应变量的分布： decision也就是说，53张卡片将标注为“不购买视频”，代表53名选择不购买的学生，另外97张卡片将标注为“购买视频”，代表其余97名学生。然后我们将这些卡片彻底洗牌，并分成两叠各75张，代表模拟的处理组和对照组。因为我们把两组的卡片混在一起洗牌，假设他们的购买行为没有差异，那么“不购买视频”卡片比例（即我们之前定义的成功)之间观察到的任何差异都可以完全归因于偶然。

如果我们将卡片随机分配到模拟的处理组和对照组，那么我们预期每个模拟组中会有多少张“不购买视频”卡片？每组中“不购买视频”卡片比例之间的预期差异是多少？

由于模拟组的规模相等，我们预期每个模拟组中有 $53 / 2 = 26.5,$ ，即26或27张“不购买视频”卡片，从而得到比例差异的模拟点估计为0%。然而，由于随机机会，我们也可能有时会观察到略高于或低于26和27的数字。

单次随机化的结果显示在表 11.5.

表11.5：学生选择与其模拟组的汇总。组分配与学生决策没有关联，因此两组之间的任何差异都是由于偶然。

	决策
组别	购买视频	不购买视频	总计
控制	46	29	75
治疗	51	24	75
总计	97	53	150

从这张表中，我们可以计算出第一次洗牌数据（即仅由偶然）产生的差异：

\[\hat{p}_{T, shfl1} - \hat{p}_{C, shfl1} = \frac{24}{75} - \frac{29}{75} = 0.32 - 0.387 = - 0.067\]

仅一次模拟不足以了解仅由偶然会产生什么样的差异。

我们将再模拟一组模拟组并计算新的差异：0.04。

再次：0.12。

再次：-0.013。

我们将这样做1,000次。

结果汇总在图 11.6的散点图中，其中每个点代表一次随机化所得的差异。

由于点太多，很难将一个点与另一个点区分开来，因此更方便的做法是将结果汇总为直方图，例如图 11.7中的直方图，其中每个直方图条形的高度表示产生该量级结果的模拟次数。

在原假设（无处理效应）下，我们大约有0.6%的时间会观察到至少+20%的差异。这真的非常罕见！因此，我们将得出结论：数据提供了强有力的证据表明存在处理效应：在购买前提醒学生他们可以稍后把钱花在其他事情上，会降低他们继续购买的可能性。请注意，由于该研究是一项实验，我们能够对其做出因果陈述，尽管我们并不知道为什么提醒会降低购买率。

11.3 假设检验

在最后两节中，我们使用了 假设检验，这是一种评估两种相互竞争的可能性的正式技术。在每种情形中，我们都描述了一个 原假设,它代表怀疑的观点或无差异的观点。我们还提出了 备择假设,它代表一种新观点,例如两个变量之间可能存在关系,或实验中可能存在处理效应。备择假设通常是科学家最初开展研究的原因。

原假设与备择假设。

该 原假设 $(H_0)$ 通常代表怀疑的观点,或待检验的“无差异”声明。

该 备择假设 $(H_A)$ 代表正在考虑的另一种声明,通常用感兴趣数值的一系列可能取值来表示。

如果一个人提出了一个有些难以置信的声明,我们起初会持怀疑态度。然而,如果有足够的证据支持该声明,我们就会放下怀疑。假设检验的特征在美国法院系统中也有体现。

11.3.1 美国法院系统

在美国法院系统中,陪审员评估证据,以判断证据是否令人信服地表明被告有罪。被告在被证明有罪之前被视为无罪。

美国法院考虑关于被告的两种可能声明:他们要么无罪,要么有罪。

如果我们在假设框架中设定这些声明,哪个是原假设,哪个是备择假设?

陪审团考虑证据是否足够令人信服(有力),以至于对该人的罪行不存在合理怀疑。也就是说,怀疑的观点(原假设)是该人在证据使陪审团相信其有罪(备择假设)之前是无罪的。

陪审员审查证据,以判断证据是否令人信服地表明被告有罪。注意,如果陪审团认定被告无罪,这并不一定意味着陪审团确信该人无罪。他们只是不相信另一种说法,即该人有罪。假设检验也是如此: 即使我们未能拒绝原假设,我们也不将原假设接受为真.

未能找到支持备择假设的证据并不等同于找到了证明原假设为真的证据。我们将在第 14.

11.3.2 p值与统计可辨别性

在第11.1节中遇到一项20世纪70年代的研究,该研究探讨了是否有强有力的证据表明女性候选人比男性候选人更不可能获得晋升。研究问题——女性候选人在晋升决策中是否受到歧视?——是在假设的背景下提出的:

$H_0:$ 性别对晋升决策没有影响。
$H_A:$ 女性候选人在晋升决策中受到歧视。

原假设 $(H_0)$ 是一种在晋升方面没有差异的观点。关于性别歧视的数据提供了男性和女性候选人之间推荐晋升率差异29.2%的点估计。我们确定,假设原假设为真,仅凭偶然出现这样的差异将是罕见的:大约每100次中只会发生2次。当这样的结果与 $H_0,$ 不一致时,我们拒绝 $H_0$ 而支持 $H_A.$ 在这里,我们得出结论:存在对女性候选人的歧视。

这个100分之2的概率就是我们所说的 p值,它是一个概率,用于在给定观测数据的情况下量化反对原假设的证据强度。

p值。

该 p值是在原假设为真的情况下，观察到至少与我们的当前数据集一样有利于备择假设的数据的概率。我们通常使用数据的一个汇总统计量，例如比例差异，来帮助计算p值并评估假设。这个用于计算p值的汇总值通常被称为 检验统计量.

在性别歧视研究中，歧视率的差异就是我们的检验统计量。在第 11.2 节)?

中介绍的Opportunity Cost（机会成本）研究中，检验统计量是什么？ 点估计 机会成本研究中的检验统计量是处理组和对照组中决定不购买视频的学生比例之差。在上述每个例子中，比例差异的

都被用作检验统计量。 统计上可辨识的统计上可辨别的 $H_0$ 。这意味着数据提供了足够强的反对⁸ 的证据，以至于我们拒绝原假设而支持备择假设。 可辨别性水平 这个阈值被称为 $\alpha$ ，通常用 表示（希腊字母). ⁹ alpha（α））。 The value of $\alpha$ 表示事件需要多么罕见才能拒绝原假设。历史上，许多领域都将 $\alpha = 0.05,$ 设定为拒绝原假设的标准。 $\alpha$ 的值可能因领域或应用的不同而有所变化。

请注意，你可能听说过“统计显著”这个短语，用来描述“统计上可辨别”。虽然在日常语言中，“显著”意味着差异很大或有意义，但在这里并不一定如此。“统计上可辨别”这一术语表示研究得到的 p 值低于所选的可辨别性水平。例如，在性别歧视研究中，p 值约为 0.02。使用 $\alpha = 0.05,$ 的可辨别性水平，我们可以说数据提供了反对原假设的统计上可辨别的证据。然而，这一结论并没有告诉我们晋升率差异的大小！

统计可辨别性。

如果 p 值小于某个预先设定的阈值（例如 0.01、0.05、0.1），我们就说数据提供了反对原假设的 统计上可辨识的 证据。

在第 11.2 节的机会成本研究中，我们分析了一个实验：如果提醒研究参与者，不用于购买视频的钱可以在未来用于其他消费，他们继续购买视频的可能性会下降 20%。我们确定，如果提醒实际上对学生的决策没有影响，如此大的差异只会以千分之六的概率出现。这项研究中的 p 值是多少？你会将该结果归类为“统计上可辨别”吗？

p 值为 0.006。由于 p 值小于 0.05，数据提供了统计上可辨别的证据，表明美国大学生实际上受到了提醒的影响。

0.05 有什么特别之处？

我们经常使用 0.05 作为判断结果是否统计上可辨别的阈值。但为什么是 0.05？也许我们应该使用更大的数字，或者更小的数字。如果你对此有些困惑，那可能说明你在带着批判的眼光阅读——干得好！我们制作了一个视频来帮助澄清 为什么是 0.05:

https://www.openintro.org/book/stat/why05/

有时偏离标准也是好主意。我们将在第 14.

11.4 本章复习

11.4.1 小结

图 11.8 中讨论何时选择不同于 0.05 的阈值。

图 11.8：第一个面板中可视化的假设数据集的完整随机化过程的一次模拟示例。我们将这些步骤重复数百或数千次。

我们可以将随机化检验过程总结如下： 用假设的形式提出研究问题。 $(H_0)$ 假设检验适用于可以归纳为两个相互竞争的假设的研究问题。原假设 $(H_A)$ 通常代表怀疑的观点或变量之间没有关系的观点。备择假设
通过观察性研究或实验收集数据。 通常代表新的观点或变量之间存在关系。
如果一个研究问题可以形成两个假设，我们就可以收集数据进行假设检验。如果研究问题关注变量之间的关联但不涉及因果关系，我们会使用观察性研究。如果研究问题寻求两个或多个变量之间的因果关系，则应使用实验。对原假设为真时会出现的随机性进行建模。
在上面的例子中，变异性被建模为处理（例如，性身份、机会）的分配与研究结果相互独立。计算机生成的零分布是许多不同随机化的结果，它量化了当原假设为真时所预期的变异性。分析数据。
形成结论。 选择适合数据的分析技术并确定 p 值。到目前为止，我们只见过一种分析技术：随机化。在本教科书的其余部分，我们将遇到适用于许多其他情境的几种新方法。

表 11.6 利用分析得到的 p 值，判断数据是否提供了反对原假设的证据。此外，务必用通俗的语言写出结论，以便普通读者能够理解结果。

是对随机化检验总结的另一种呈现方式。

问题	答案
它的作用是什么？	打乱解释变量以模拟随机化实验中出现的自然变异性
所描述的随机过程是什么？	随机化实验
还可以近似哪些其他随机过程？	也可用于描述观察模型中的随机抽样
它最适合用于什么？	假设检验（也可用于置信区间，但本教材未涉及）
什么实物可以代表这一模拟过程？	洗牌

11.4.2 术语

本章中介绍的术语列于表 11.7。如果您不确定其中一些术语的含义，我们建议您回到正文中复习它们的定义。您应该能够很容易地发现它们是 粗体文本.

表 11.7：本章介绍的术语。

备择假设	置换检验	统计推断
可辨别性水平	点估计	统计上可辨识的
假设检验	随机化检验	统计上显著的
独立	显著性水平	成功
原假设	模拟	检验统计量
p值	统计量

11.5 练习

奇数编号习题的答案见附录 A.11.

确定参数，I。 对于以下每种情形，判断感兴趣的参数是均值还是比例。观察个体响应是数值型还是分类型可能会有所帮助。
1. 在一项调查中，100名大学生被问及他们每周花多少小时上网。
2. 在一项调查中，100名大学生被问及：“你上网的时间中有多少百分比用于课程学习？”
3. 在一项调查中，100名大学生被问及他们是否在论文中引用了维基百科的信息。
4. 在一项调查中，100名大学生被问及他们在酒精饮料上的支出占其每周总支出百分比是多少。
5. 在100名近期大学毕业生的样本中，发现85%的人预计在毕业后一年内找到工作。

识别参数，II。 对于以下每种情况，说明感兴趣的参数是均值还是比例。
1. 一项民意调查显示，64%的美国人个人非常担心联邦支出和预算赤字。
2. 一项调查报告显示，地方电视新闻在两年内收入增长了17%，而同期报纸收入下降了6.4%。
3. 在一项调查中，高中生和大学生被问及是否在智能手机上使用地理位置服务。
4. 在一项调查中，智能手机用户被问及是否使用基于网络的出租车服务。
5. 在一项调查中，智能手机用户被问及在过去一年中使用基于网络的出租车服务的次数。

假设。 对于下面的每个研究陈述，注明它是零假设陈述还是备择假设陈述。
1. 小学儿童做作业的小时数可以预测他们在标准化考试中的未来成功。
2. 王猎豹的平均奔跑速度与普通有斑点的猎豹相同。
3. 对于某个学生来说，正确回答一道5选项选择题的概率大于0.2（即比瞎猜要好）。
4. 非洲象象牙的平均长度在过去100年中发生了变化。
5. 服用叶酸补充剂的母亲所生婴儿与未服用叶酸补充剂的母亲所生婴儿发生面部畸形（唇腭裂）的风险相同。
6. 孕期摄入咖啡因会影响平均出生体重。
7. 使用手机时发生车祸的概率与不使用手机时相同。

真实的零假设。 假设在总体中零假设实际上是正确的，而你并不知情。你仍然计划进行一项研究。
1. 如果你选择的显著性水平（即 p 值的临界值）为 0.05，你错误地拒绝零假设的可能性有多大？
2. 如果你选择的显著性水平（即 p 值的临界值）为 0.01，你错误地拒绝零假设的可能性有多大？
3. 如果你选择的显著性水平（即 p 值的临界值）为 0.10，你错误地拒绝零假设的可能性有多大？

识别假设，I。 针对以下每种情况，用文字和符号写出零假设和备择假设。
1. 纽约被称为“不夜城”。一项随机样本调查了 25 名纽约人每晚的睡眠时间。这些数据是否提供了令人信服的证据，表明纽约人平均每晚睡眠少于 8 小时？
2. 一家公司的雇主担心“疯狂三月”（美国每年春季举行的篮球锦标赛）对员工生产力的影响。他们估计，在正常工作日，员工平均花费 15 分钟的公司时间查看私人邮件、打私人电话等。他们还收集了员工在“疯狂三月”期间在这类非工作活动上花费的公司时间数据。他们想确定这些数据是否提供了令人信服的证据，表明员工生产力在“疯狂三月”期间有所下降。

识别假设，II。 针对以下每种情况，用文字和符号写出零假设和备择假设。
1. 自 2008 年以来，加利福尼亚州的连锁餐厅被要求在菜单上标明每道菜品的卡路里含量。在菜单标明卡路里含量之前，该餐厅用餐者的平均卡路里摄入量为 1100 卡路里。在菜单开始标明卡路里含量之后，一位营养师从该餐厅用餐者的随机样本中收集了卡路里摄入量的数据。这些数据是否提供了令人信服的证据，表明该餐厅用餐者的平均卡路里摄入量发生了变化？
2. 根据 2004 年 7 月 1 日至 2007 年 6 月 30 日期间 GRE 考试考生的表现，计算得出平均语文推理得分为 462。2021 年的平均语文得分略高。这些数据是否提供了令人信服的证据，表明 GRE 语文推理平均得分自 2021 年以来发生了变化？

文达敏（Avandia）的副作用。 罗格列酮（Rosiglitazone）是备受争议的2型糖尿病药物文达敏（Avandia）中的有效成分，已被证实与严重心血管问题（如中风、心力衰竭和死亡）风险增加有关。一种常见的替代治疗药物是吡格列酮（Pioglitazone），它是糖尿病药物爱妥糖（Actos）中的有效成分。在一项针对227,571名65岁及以上联邦医疗保险（Medicare）受益人的全国性回顾性观察研究中，发现使用罗格列酮的67,593名患者中有2,593人出现严重心血管问题，而使用吡格列酮的159,978名患者中有5,386人出现严重心血管问题。这些数据汇总在下方的列联表中。¹⁰ (Graham et al. 2010)

治疗否是总计

吡格列酮（Pioglitazone） 154,592 5,386 159,978

罗格列酮（Rosiglitazone） 65,000 2,593 67,593

总计 219,592 7,979 227,571
1. 判断以下每个陈述是真是假。如果是假的，请解释原因。 注意： 即使陈述的结论是正确的，其推理过程也可能是错误的。在这种情况下，该陈述应被视为错误。
  1. 由于使用吡格列酮的患者中出现心血管问题的人数更多（5,386 对比 2,593），我们可以得出结论：接受吡格列酮治疗的患者出现心血管问题的比例更高。
  2. 数据表明，服用罗格列酮的糖尿病患者更容易出现心血管问题，因为该治疗组患者的心血管问题发生率为 (2,593 / 67,593 = 0.038) 3.8%，而吡格列酮组患者仅为 (5,386 / 159,978 = 0.034) 3.4%。
  3. 罗格列酮组的发生率更高这一事实证明了罗格列酮会导致严重的心血管问题。
  4. 根据目前提供的信息，我们无法判断发生率之间的差异是由于两个变量之间的关系，还是由于偶然因素造成的。
2. 所有患者中出现心血管问题的比例是多少？
3. 如果治疗类型与出现心血管问题相互独立，那么我们预期罗格列酮组中会有多少患者出现心血管问题？
4. 我们可以使用随机化技术来研究本研究中结果与治疗之间的关系。虽然实际上我们会使用统计软件进行随机化所需的模拟，但假设我们实际上使用索引卡进行模拟。为了从独立性模型（即结果与治疗相互独立的模型）进行模拟，我们在卡片上写下每位患者是否出现心血管问题，将所有卡片混在一起洗牌，然后将它们分成大小分别为67,593和159,978的两组。我们重复这个模拟100次，每次记录罗格列酮组和吡格列酮组中写着“是”的卡片比例之间的差异。利用这些比例差异的直方图回答以下问题。
  1. 正在检验的假设是什么？
  2. 与在(b)部分计算出的数值相比，哪个数值能为备择假设提供更多支持，更高或下限罗格列酮组中出现心血管问题的患者比例是多少？
  3. 模拟结果对于糖尿病患者服用罗格列酮与出现心血管问题之间的关系有何提示？

治疗	否	是	总计
吡格列酮（Pioglitazone）	154,592	5,386	159,978
罗格列酮（Rosiglitazone）	65,000	2,593	67,593
总计	219,592	7,979	227,571

心脏移植。 斯坦福大学心脏移植研究旨在确定一项实验性心脏移植项目是否能延长寿命。每位进入该项目的患者都被指定为正式的心脏移植候选人，这意味着他们病情严重，很可能从新的心脏中获益。有些患者接受了移植，有些则没有。变量 transplant 表示患者所在的组别；治疗组的患者接受了移植，而对照组的患者则没有。在对照组的34名患者中，有30人死亡。在治疗组的69人中，有45人死亡。另一个名为 survived 的变量用于表示患者在研究结束时是否存活。¹¹ (Turnbull 等，1974)
1. 堆叠条形图是否表明存活情况与患者是否接受移植无关？请解释你的理由。
2. 箱线图对心脏移植的疗效有何提示？
3. 治疗组和对照组中死亡的患者比例各是多少？
4. 研究治疗效果是否明显有效的一种方法是随机化检验。
  1. 正在检验的假设是什么？
  2. 下面的段落描述了随机化检验的设置（如果我们不使用统计软件来进行的话）。请用一个数字或短语填空。
  我们写下存活于 $\rule{1.25cm}{0.5pt}$ 张卡片上，代表研究结束时仍然存活的患者，并且 已故患者 于 $\rule{1.25cm}{0.5pt}$ 张卡片代表非已故患者。然后，我们将这些卡片洗牌并分成两组：一组大小为 $\rule{1.25cm}{0.5pt}$ ，代表治疗组，另一组大小为 $\rule{1.25cm}{0.5pt}$ ，代表对照组。我们计算治疗组和对照组中卡片比例之间的差值（治疗组 - 对照组）并记录该值。我们重复此过程100次，以构建一个以 $\rule{1.25cm}{0.5pt}$为中心的分布。最后，我们计算模拟中比例差异 $\rule{1.25cm}{0.5pt}$的模拟所占的比例。如果这个比例很低，我们得出的结论是：偶然观察到这样的结果是不太可能的，因此应拒绝原假设而支持备择假设。
  1. 下面显示的模拟结果对心脏移植的有效性有何提示？

我们将假设这两个变量是独立.↩︎
该研究是一项实验，因为受试者被随机分配了一份“男性”档案或一份“女性”档案（请记住，所有档案的内容实际上完全相同）。由于这是一项实验，其结果可用于评估候选人性别与晋升决策之间的因果关系。↩︎
我们在本节中使用的检验程序有时被称为 随机化检验。如果解释变量不是随机分配的，例如在观察性研究中，该程序则被称为 置换检验。置换检验用于解释变量未经随机分配的观察性研究。↩︎
$18/24 - 17/24=0.042$ ，即约4.2%，偏向男性人员。这一由偶然造成的差异远小于在实际组别中观察到的差异。↩︎
这一推理通常不适用于轶事观察。我们每个人每天都在观察到极其罕见的事件，这些事件是我们根本无法预测的。然而，在轶事证据这种非严格的情境中，几乎任何事情都可能显得是罕见事件，因此在日常活动中寻找罕见事件的想法是不可靠的。例如，我们可以看看彩票：历史上最大奖池（2018年10月23日）的Mega Millions开奖号码为(5, 28, 62, 65, 70)、Mega ball为(5)的概率仅为1.76亿分之一，但这些号码确实开出来了！然而，无论开出什么号码，其概率都同样是极其罕见的。也就是说， 我们可能观察到的任何一组号码最终都是极其罕见的。这种情况在我们的日常生活中很典型：每个可能的事件本身似乎都极其罕见，但如果我们考虑每一种替代结果，那些结果也同样极其罕见。我们应谨慎行事，不要误读这类轶事证据。↩︎
如果实体音像租赁店出现的年代早于你，这个背景可能会让你感到陌生。如果你好奇那些店是什么样的，可以搜索“Blockbuster”（百视达）。↩︎
在一项研究中，“成功”通常被定义为所关注的结果，而“成功”可能是也可能不是真正积极的结果。例如，从事COVID患病率研究的学者可能会在统计意义上将“成功”定义为一名患有COVID-19的患者。关于该术语更完整的讨论成功将在第 16.↩︎
许多教材使用“统计显著”而非“统计可辨别”这一表述。我们选择使用“可辨别”来表示一个精确的统计事件已经发生，而不是指一个显著效应——后者可能符合也可能不符合可辨别或显著的统计定义。↩︎
同样，我们在这里也选择使用“可辨别性水平”而非某些教材中出现的“显著性水平”。↩︎
该 avandia 本练习中使用的数据可在 openintro R 包中找到。↩︎
该 heart_transplant 本练习中使用的数据可在 openintro R 包中找到。↩︎