Probability – Advanced University‑Level Notes
(For the MAT (Quantitative Ability) Program – MIT‑style)
Author: Expert Teacher
Last updated: 2026‑10‑20
These notes start from the very first principles of probability theory, move through the classical axiomatic framework, and then introduce the measure‑theoretic machinery that underpins modern probability. Every definition is accompanied by a proof (or sketch of a proof) where appropriate, and we emphasise subtle edge cases that often trip students up in exams.
TL;DR –
- A probability space is a measure space ((\Omega,\mathcal{F},P)).
- Probability satisfies the Kolmogorov axioms.
- Conditional probability, independence, and Bayes’ theorem follow from the axioms.
- Random variables are measurable functions; expectations are Lebesgue integrals.
- Key limit theorems (LLN, CLT) describe the asymptotic behaviour of sums of i.i.d. random variables.
- Edge cases: zero‑probability events, non‑measurable sets, conditional probability given a σ‑algebra.
The challenging problem at the end will test both combinatorial intuition and rigorous probability reasoning.
1. Axiomatic Foundations
1.1 Probability Space
Definition 1.1 (Sample Space & σ‑Algebra)
• ( \Omega ) – non‑empty set of all possible outcomes.
• ( \mathcal{F} \subseteq 2^{\Omega} ) – a σ‑algebra:
- ( \Omega \in \mathcal{F} ).
- If ( A \in \mathcal{F} ) then ( A^c \in \mathcal{F} ).
- If ( (A_n){n\ge1} \subseteq \mathcal{F} ) then ( \bigcup{n=1}^{\infty} A_n \in \mathcal{F} ).
Definition 1.2 (Probability Measure)
A function (P:\mathcal{F}\to[0,1]) such that
- (P(\Omega)=1).
- Countable additivity: for any disjoint (A_1,A_2,\dots\in\mathcal{F}),
[ P!\left(\bigcup_{n=1}^{\infty}A_n\right)=\sum_{n=1}^{\infty}P(A_n). ]
The triple ((\Omega,\mathcal{F},P)) is called a probability space.
Remark 1.3 – In a finite setting, the σ‑algebra can be taken as the power set (2^{\Omega}). In infinite settings we must be careful to exclude “pathological” subsets that would violate countable additivity.
1.2 Basic Properties (Derived from the Axioms)
| Property | Proof Sketch |
|---|---|
| (P(\emptyset)=0) | ( \emptyset = \Omega^c \Rightarrow P(\emptyset)=1-P(\Omega)=0 ). |
| Monotonicity: (A\subseteq B \implies P(A)\le P(B)) | (B=A\cup (B\setminus A)) and disjointness. |
| Inclusion–exclusion for two events | (P(A\cup B)=P(A)+P(B)-P(A\cap B)). |
| Complementarity | (P(A^c)=1-P(A)). |
| Subadditivity | (P(\bigcup_{i}A_i)\le\sum_iP(A_i)). |
1.3 Conditional Probability
Definition 1.4 (Given a σ‑Algebra)
For (A\in\mathcal{F}) with (P(A)>0),
[ P(B\mid A)=\frac{P(A\cap B)}{P(A)}. ]
Theorem 1.5 (Multiplication Rule)
For any (A,B\in\mathcal{F}) with (P(A)>0),
[ P(A\cap B)=P(A)P(B\mid A). ]
Proof – Direct from the definition of conditional probability.
Edge Case – If (P(A)=0), the ratio is undefined. We will later see how to define conditional probability via regular conditional probabilities (Radon–Nikodym).
1.4 Independence
Definition 1.6 (Independence of Two Events)
(A) and (B) are independent iff
[ P(A\cap B)=P(A)P(B). ]
Definition 1.7 (Mutual Independence of a Collection)
Events (A_1,\dots,A_n) are mutually independent iff for every (k) and every choice of indices (i_1<\dots<i_k\