8 Probability
8.1 What is randomness?
Note
Your Pikachu is facing a random opponent. What type will it be? Water? Flying? Grass? You can’t know for sure, but you know that certain types appear more often than others. That’s randomness: the outcome is unknown, but not completely hidden.
Here are the key terms:
- A trial is one instance of the random phenomenon (encountering a Pokémon, or checking its type).
- An outcome is the result of a trial: the specific type of Pokémon we encounter.
- The sample space \(S\) is the set of all possible outcomes, e.g., the 18 Pokémon types: Normal, Fire, Water, etc.
- An event is a subset of the sample space, e.g., “the Pokémon is a Water-type,” or “the Pokémon is either Electric-type or Dragon-type”.
Randomness means we know the sample space but not the outcome. To measure how likely each outcome is, we count:
\[P(\text{Pokémon type}) = \frac{\text{count(type)}}{\text{total count}}\]
For example:
\[P(\text{Water-type}) = \frac{\text{number of Water-type Pokémon}}{\text{total number of Pokémon}}\]
8.2 Probability rules
8.2.1 Rule 1: A probability is between 0 and 1
The universe has size 1: \(P(\text{universe}) = 1\). For any event \(A\), we have \(0 \le P(A) \le P(\text{universe}) = 1\). A probability of 1 means you’re completely confident that the event will occur. A probability can’t be negative.
In the barplot below, each probability is between 0 and 1.
8.2.2 Rule 2: Probabilities over the entire sample space sum to 1
In other words, \(P(S) = 1\).
Every Pokémon has a type. When we add up the probabilities of all types, they sum to exactly 1. If probabilities don’t sum to 1, something is wrong.
Number of Pokémon types: 18
Sum of all probabilities: 1.00000000008.2.3 Rule 3: The complement rule
For any event \(A\), we have \(P(A^c) = 1 - P(A)\), where \(A^c\) is “not \(A\).”
If the probability that a Pokémon is Water-type is 6%, then the probability that it is anything but Water-type is 94%.
P(Water-type) = 0.1328
P(not Water-type) = 0.8672
Sum: 1.00008.2.4 Rule 4: Addition rule for disjoint events
If events \(A\) and \(B\) cannot both happen at the same time (they are disjoint), then: \[P(A \text{ or } B) = P(A) + P(B).\]
A Pokémon is either Water-type, Electric-type, or Grass-type, but not two primary types at once. The probability that a Pokémon is Water-type or Electric-type is just the sum of their individual probabilities.
P(Water-type) = 0.1328
P(Electric-type) = 0.0643
P(Water-type or Electric-type) = 0.19708.2.5 Rule 5: Multiplication rule for independent events
If events \(A\) and \(B\) are independent (knowing \(A\) happened doesn’t change the probability of \(B\)), then: \[P(A \text{ and } B) = P(A) \cdot P(B).\]
Intuitively, if you restrict your universe to \(A\), then the proportion of \(B\) in your restricted universe (which is \(P(A \text{ and } B) / P(A)\)) remains the same. The same goes if you restrict to \(B\).
To fix ideas, consider the following example.
| P(A) | P(B) | P(A and B) | P(A and B) / P(A) | P(A and B) / P(B) | |
|---|---|---|---|---|---|
| Dependent | 0.3 | 0.25 | 0.15 | 0.5 | 0.6 |
| Independent | 0.3 | 0.25 | 0.075 | 0.25 | 0.3 |
NotePokémon type and attack stat are not independent
Suppose we naively assumed that a Pokémon’s type and its attack stat were unrelated. We’d compute: \(P(\text{high attack and fighting-type}) = P(\text{high attack}) \cdot P(\text{fighting-type})\).
But fighting-type Pokémon are designed with high attack stats more often than a random Pokémon. If you know a Pokémon is a fighting-type, the probability it has a high attack is much higher than if you knew nothing. Type and attack are not independent.
P(high attack) overall = 0.4889
P(high attack | fighting-type) = 0.8387
If independent: P(high attack and fighting) = 0.0160
But we observe P(high attack and fighting) = 0.02748.3 The true definition of randomness: a measure over inputs
During a battle, the Pokémon you face is a complex function of the trainer state of mind, his history with their Pokémon, etc. We’ll never know this exact function, but we don’t need it to develop a battle strategy. We only need the size of the events that will lead to a specific type.
A random variable is a function from some space (the universe) to a sample space (here, Pokémon type). Let’s call it \(X(w)\) = “the Pokémon type” if the universe is in state \(w\). You can think of \(w\) as a long list of information about the trainer, their Pokémon roster, etc. We don’t know its type before we encounter it because we don’t have access to \(w\). But we do know that out of all possible outcomes, only 6% will be Fire-type and 13% will be Water-type. So it’s safer to prepare for a Water-type.
Again, information about the size of the events is enough to make informed decisions. That’s the core intuition behind probability and statistics.