11 Discrete probability distributions
Imagine a basketball player with a 70% shooting percentage. How many shots do you expect they make during a game? What’s the variance? How many misses before they make one, on average?
What if we consider an 85% hit ratio? Instead of recomputing everything, we can use well-known probability models.
11.1 Bernoulli trials
Bernoulli trials are a collection of independent experiments, each with exactly two outcomes: success with probability \(p\) or failure with probability \(1-p\).
For example, 10 free throws with shooting percentage \(p = 0.70\) are Bernoulli trials.
11.2 Model 1: Geometric model
The question is: how many trials before the first success?
(How many shots until you make your first free throw?)
Let \(X\) = the trial number when you get your first success.
| What you observe | Probability |
|---|---|
| Make on shot 1 | \(p = 0.70\) |
| Miss shot 1, make shot 2 | \((1-p) p = (0.30)(0.70) = 0.21\) |
| Miss shots 1 and 2, make shot 3 | \((1-p)^2 p = (0.30)^2 (0.70) ≈ 0.063\) |
| Miss shots 1 to \(n−1\), make shot \(n\) | \((1-p)^{n−1} p\) |
Probability Mass Function (PMF): The probability of a specific value (here, making your first shot on trial \(n\)).
\[P(X = n) = (1-p)^{n-1}p\]
Cumulative Distribution Function (CDF): The probability of success on or before trial \(n\).
\[P(X \leq n) = 1 - (1-p)^n\]
Example: Probability you make your first shot by the 3rd attempt: \[P(X \leq 3) = 1 - (0.30)^3 = 1 - 0.027 = 0.973\] About 97% of the time, you’ll make one within three shots.
Expected value and variance
\[E(X) = \frac{1}{p} = \frac{1}{0.70} \approx 1.43 \text{ shots}\]
On average, you make your first shot before your second attempt.
\[\text{Var}(X) = \frac{(1-p)}{p^2} = \frac{0.30}{(0.70)^2} \approx 0.612\]
\[\text{SD}(X) = \sqrt{0.612} \approx 0.78\]
Notice how a higher \(p\) concentrates the distribution near 1 (success comes quickly), while a lower \(p\) spreads it out (more trials needed).
11.3 Model 2: Binomial
The question is: how many successes are there in \(n\) trials?
(If you shoot 10 free throws, how many will you make?)
Let \(X\) = the number of successes in \(n\) = 10 independent trials.
You could make 0, 1, 2, …, or all 10. The Binomial model tells you the probability of each.
Probability Mass Function: Probability of exactly \(k\) successes in \(n\) trials.
\[P(X = k) = \binom{n}{k} p^k q^{n-k}\]
where \(\binom{n}{k} = \frac{n!}{k!(n-k)!}\) counts the number of orderings.
Example: Probability of making exactly 7 out of 10 shots:
\[P(X = 7) = \binom{10}{7} (0.70)^7 (0.30)^3\]
\[= 120 \cdot 0.0823 \cdot 0.027 \approx 0.267\]
About 26.7% of the time, you’ll make exactly 7 out of 10.
Cumulative Distribution Function: Probability of \(k\) or fewer successes.
\[P(X \leq k) = \sum_{i=0}^{k} \binom{n}{i} p^i q^{n-i}\]
Example: Probability of making 7 or fewer (missing at least 3):
\[P(X \leq 7) \approx 0.953\]
Expected value and variance
\[E(X) = np = 10 \cdot 0.70 = 7\]
You expect to make 7 out of 10 shots on average.
\[\text{Var}(X) = npq = 10 \cdot 0.70 \cdot 0.30 = 2.1\]
\[\text{SD}(X) = \sqrt{2.1} \approx 1.45\]
So most of the time, you’ll make between about 5.5 and 8.5 shots (within one SD of the mean).
The binomial distribution is more symmetric and bell-shaped as \(n\) increases. A higher \(p\) shifts the distribution toward the right (we expect more successes).
Note
Don’t confuse Geometric and Binomial: geometric asks “how long until success?”, binomial asks “how many successes in a fixed set?”
11.4 Model 3: Poisson
The question is: how many events occur during a fixed time period or space?
(During a 30-minute shooting practice, you attempt shots at random times. On average, you attempt 12 shots per practice. What’s the probability you attempt exactly 8?)
Poisson is useful as an approximation to the Binomial model. When \(n\) is very large and \(p\) is very small, computing the Binomial PMF is cumbersome (factorials with huge numbers). The Poisson model is a simpler alternative.
For example: suppose you’re counting defects in 1 million items, where only 0.0001% have a defect. Computing the exact Binomial probability involves \(\binom{1,000,000}{k}\) (impractical!). Instead, set \(\lambda = np = 1{,}000{,}000 \times 0.000001 = 1\) and use the Poisson model.
When to use Poisson:
- You’re counting rare or random events in a fixed time/space (attempts per practice, typos per page, phone calls per hour)
- The events occur independently at a roughly constant rate
Let \(X\) = the number of shots in 30 minutes, with average rate \(\lambda = 12\).
Note
The intensity \(\lambda\) is the mean number of events in the same time unit you’re considering.
Probability Mass Function:
\[P(X = k) = \frac{e^{-\lambda} \lambda^k}{k!}\]
where \(e \approx 2.71828\).
Example: Probability of exactly 8 shots:
\[P(X = 8) = \frac{e^{-12} \cdot 12^8}{8!} = \frac{(0.0000614) \cdot (429,981,696)}{40,320} \approx 0.065\]
About a 6.5% chance of exactly 8 shots.
Cumulative Distribution Function: Probability of \(k\) or fewer shots.
\[P(X \leq k) = \sum_{i=0}^{k} \frac{e^{-\lambda} \lambda^i}{i!}\]
Expected value and variance
\[E(X) = \lambda = 12\]
\[\text{Var}(X) = \lambda = 12\]
\[\text{SD}(X) = \sqrt{\lambda} = \sqrt{12} \approx 3.46\]
Note: The mean and variance are the same for a Poisson distribution
A higher \(\lambda\) means more events on average, shifting the distribution to the right. With larger \(\lambda\), the Poisson distribution becomes more symmetric.
11.5 Summary of models
| Model | Question | Example |
|---|---|---|
| Geometric(\(p\)) | “How many trials until the first success?” | How many shots until I make one? |
| Binomial(\(n\), \(p\)) | “How many successes in \(n\) fixed trials?” | How many makes in 10 shots? |
| Poisson(\(\lambda\)) | “How many events in a fixed time/space?” | How many shots taken in 30 minutes? |
11.6 Understanding PMF vs CDF
Probability Mass Function (PMF). The probability of a specific value.
- Always non-negative
- For a discrete variable, the sum over all possible values equals 1
- Tells you the “height” of a single outcome
Example: \(P(X = 7) = 0.267\) means there’s a 26.7% chance of exactly 7 successes.
Cumulative Distribution Function (CDF)
The probability of a value or less (accumulating from the start).
- Ranges from 0 to 1
- Always increasing (or flat)
- \(P(X \leq k) =\) sum of all PMF values from 0 up to \(k\)
Example: \(P(X \leq 7) = 0.953\) means there’s a 95.3% chance of making 7 or fewer shots.
Connection between PMF and CDF.
\[P(X = k) = P(X \leq k) - P(X \leq k-1)\]
The PMF is the “jump” between consecutive CDF values.
11.7 Computing expected value and variance
Expected value. The long-run average if you repeated the experiment many times.
\[E(X) = \sum_{k} k \cdot P(X = k)\]
For Binomial: \(E(X) = np\)
For Poisson: \(E(X) = \lambda\).
Variance. How spread out the values are around the mean.
\(\text{Var}(X) = \sum_{k} (k - E[X])^2 \cdot P(X = k)\)
For Binomial: \(\text{Var}(X) = npq\).
For Poisson: \(\text{Var}(X) = \lambda\).
Standard deviation. \[\text{SD}(X) = \sqrt{\text{Var}(X)}\]
11.8 Summary of formulas
| Model | PMF | E(X) | Var(X) |
|---|---|---|---|
| Geom(\(p\)) | \((1-p)^{n-1}p\) | \(1/p\) | \((1-p)/p^2\) |
| Binom(\(n\), \(p\)) | \(\binom{n}{k}p^k (1-p)^{n-k}\) | \(np\) | \(np(1-p)\) |
| Poisson(\(\lambda\)) | \(\frac{e^{-\lambda}\lambda^k}{k!}\) | \(\lambda\) | \(\lambda\) |