| Survived | No | Yes | Total |
|---|---|---|---|
| Class | |||
| 1st | 122 | 203 | 325 |
| 2nd | 167 | 118 | 285 |
| 3rd | 528 | 178 | 706 |
| Crew | 673 | 212 | 885 |
| Total | 1490 | 711 | 2201 |
6 Conditional distributions
So far, we’ve looked at the distribution of one variable at a time: the frequency table and bar chart for a categorical variable, and the histogram for a quantitative variable. But most interesting statistical questions are about the relationship between two variables: does survival on the Titanic depend on ticket class? Does a cereal’s sugar content depend on which shelf it’s stocked on?
To answer these questions, we examine the distribution of one variable within each category of another variable. This is called a conditional distribution.
NoteMotivating example: Titanic passengers
The Titanic dataset (see here for a brief description) records, for each of the 2,201 people aboard, their ticket Class (1st, 2nd, 3rd, or Crew) and whether they Survived.
Overall, about 32.3% of people aboard survived. But does that percentage apply to every class, or is it different for 1st-class passengers than for the crew?
NoteMotivating example: US breakfast cereals
In the breakfast cereals dataset, we already looked at the distribution of sugars (grams of sugar per serving) across all 77 cereals. Cereals are also stocked on one of three shelves (1 = bottom, 2 = middle, 3 = top).
Is the distribution of sugars the same on every shelf, or do some shelves tend to hold sweeter cereals than others?
6.1 Contingency tables
To compare Class and Survived, we cross-tabulate them: for each combination of class and survival status, we count how many people fall into that cell. This is a contingency table.
The row and column labelled “Total” are the marginal distributions: the distribution of Class alone (ignoring Survived), and the distribution of Survived alone (ignoring Class). They’re called “marginal” simply because they sit in the table’s margins.
The conditional distribution of Survived for a given class is the distribution of Survived restricted to that one row. For example, the conditional distribution of Survived for 1st class passengers only considers the 325 people in the “1st” row (122 who didn’t survive, 203 who did), not the other 1,876 people aboard.
In case we want to compare the row-by-row shape, not the raw counts, conditional distributions can also be reported as percentages. Here, there were far more crew than 1st class passengers.
| Survived | No | Yes | Total |
|---|---|---|---|
| Class | |||
| 1st | 37.5% | 62.5% | 100.0% |
| 2nd | 58.6% | 41.4% | 100.0% |
| 3rd | 74.8% | 25.2% | 100.0% |
| Crew | 76.0% | 24.0% | 100.0% |
The survival rate drops steadily from 62.5% in the 1st class to 24.0% among the crew. Because the conditional distribution of Survived changes a lot from row to row, Class and Survived are clearly related.
Warning
A contingency table can be conditioned in two directions: by row (the conditional distribution of the column variable, given the row) or by column (the conditional distribution of the row variable, given the column). Always say which variable you’re conditioning on: “the distribution of Survived, for each Class” is not the same computation as “the distribution of Class, for each Survived value”.
Conditioning by column instead normalises each column to sum to 100%, so it’s the row labelled “Total” that’s now all 100%, not a column:
| Survived | No | Yes |
|---|---|---|
| Class | ||
| 1st | 8.2% | 28.6% |
| 2nd | 11.2% | 16.6% |
| 3rd | 35.4% | 25.0% |
| Crew | 45.2% | 29.8% |
| Total | 100.0% | 100.0% |
A contingency table is often called a pivot table.
6.2 Independence (intuitive definition)
Two variables are independent if the conditional distribution of one is (roughly) the same across every category of the other. If knowing someone’s class tells you nothing about their chance of survival, Class and Survived would be independent.
That’s clearly not the case here: the conditional distributions above (62.5%, 41.4%, 25.2%, 24.0%) differ markedly from each other and from the marginal survival rate of 32.3%. Class and Survived are not independent: your ticket class was strongly associated with whether you survived.
6.3 Lurking variables
A lurking variable is a variable that isn’t part of the analysis, but that affects the apparent relationship between the variables.
For instance, the Titanic data also records Sex. Women were much more likely to survive than men, but class and sex were themselves related (the 1st class had a different mix of men and women than the 3rd class or the crew). Whenever a two-variable relationship looks surprising, it’s worth asking whether a third, lurking variable might be driving it.
6.4 Visualising conditional distributions
6.4.1 Categorical variable: side-by-side bar chart
A side-by-side bar chart places one group of bars per category of the conditioning variable, with each group showing the conditional distribution of the other variable. Here, one group of bars per Class, coloured by Survived:
6.4.2 Quantitative variable: overlapping density plots
To compare a quantitative variable’s distribution across categories, overlay one density plot per category, using colour and some transparency so they remain readable where they overlap:
Cereals on shelf 2 tend to have more sugar (mean 9.6 g, median 12.0 g) than those on shelf 1 (mean 5.1 g, median 3.0 g) or shelf 3 (mean 6.5 g, median 6.0 g).
6.5 Quick reference: fractions and their percentage equivalents
| Fraction | Percentage | Fraction | Percentage | |
|---|---|---|---|---|
| 1/2 | 50% | 1/5 | 20% | |
| 1/3 | 33.3% | 2/5 | 40% | |
| 2/3 | 66.7% | 3/5 | 60% | |
| 1/4 | 25% | 4/5 | 80% | |
| 3/4 | 75% | 1/8 | 12.5% | |
| 1/10 | 10% | 3/8 | 37.5% | |
| 1/20 | 5% | 5/8 | 62.5% | |
| 1/100 | 1% | 7/8 | 87.5% |