Conditional Probability, Independence and Bayes' Theorem

For events A and B with $P(B) > 0$, the conditional probability of A given B is

• $P(A\mid B) = \dfrac{P(A\cap B)}{P(B)}$ — the share of B's probability that also lies in A.
• Equivalently, rearranged, $P(A\cap B) = P(B)\,P(A\mid B) = P(A)\,P(B\mid A)$ — the multiplication rule.
• When outcomes are equally likely, this reduces to counting: $P(A\mid B) = \dfrac{n(A\cap B)}{n(B)}$.

The general multiplication rule for a chain of events:

• $P(E_1\cap E_2\cap E_3) = P(E_1)\,P(E_2\mid E_1)\,P(E_3\mid E_1\cap E_2)$.
• Without replacement: after each draw the counts shrink, so denominators drop by 1 and the relevant numerator drops by 1 as well. Drawing tickets one at a time is exactly this.
• One item from each of several independent sources: the joint probability is just the product of each source's single-draw probability; add over all the ways a target composition can occur.

Apply $P(A\mid B) = \dfrac{P(A\cap B)}{P(B)}$ to composite events:

• Distribution table: $P(a\le X<b\mid X\le c)$ is the sum of the P(X) values in the numerator range that also satisfy $X\le c$, divided by $P(X\le c)$.
• 'At least one' condition: for family/coin problems, $P(\text{all girls}\mid \text{at least one girl}) = \dfrac{P(\text{all girls})}{P(\text{at least one girl})}$, and 'at least one' is best found as $1 - P(\text{none})$.
• Counting-based: $P(A\mid B) = \dfrac{n(A\cap B)}{n(B)}$ — list the outcomes in B, then count how many also lie in A.

The independence identity and its consequences:

• $P(A\cap B) = P(A)\,P(B)$ (definition); then $P(A\cup B) = P(A)+P(B)-P(A)P(B)$.
• If A, B are independent, so are the pairs $(A',B)$, $(A,B')$, $(A',B')$; hence $P(A'\mid B) = P(A')$ and $P(B'\mid A') = P(B')$.
• Odds: 'odds in favour $a:b$' means $P = \dfrac{a}{a+b}$; 'odds against $a:b$' means $P = \dfrac{b}{a+b}$.
• The complement-conditional sum $P(A'\mid B') + P(B'\mid A') = \dfrac{P(A'\cap B')}{P(B')} + \dfrac{P(A'\cap B')}{P(A')}$, using $P(A'\cap B') = 1 - P(A\cup B)$.

For independent events $A_1,\dots,A_n$ with success probabilities $p_i$:

• At least one succeeds: $P(\text{at least one}) = 1 - \prod_i (1-p_i)$ — one minus 'all fail'.
• Exactly one (of two): $P = P(A)P(B') + P(A')P(B)$.
• A composite pattern like 'hit by P or Q but not R' is a sum of the independent triple-products matching that description, e.g. $P Q' R' + P' Q R' + P Q R'$ (all with R failing).

If $H_1,\dots,H_n$ are mutually exclusive and exhaustive (they partition the sample space) with $P(H_i) > 0$, then for any event E:

• $P(E) = \displaystyle\sum_{i=1}^{n} P(H_i)\,P(E\mid H_i)$.
• Each term is one route's prior times its conditional; the routes' priors sum to 1.
• Polya-urn / draw-then-add problems fit here: the first draw's colour defines the partition, and the second-draw probability is conditional on the updated bag.

For a partition $H_1,\dots,H_n$ and observed event E with $P(E) > 0$:

• $P(H_k\mid E) = \dfrac{P(H_k)\,P(E\mid H_k)}{\displaystyle\sum_{i} P(H_i)\,P(E\mid H_i)}$.
• Numerator = the chosen route's prior × likelihood; denominator = total probability of E (the sum of ALL routes' prior × likelihood).
• With equal priors $P(H_i) = \tfrac1n$, the priors cancel and the posterior is just $\dfrac{P(E\mid H_k)}{\sum_i P(E\mid H_i)}$ — a ratio of likelihoods.