Measures of Central Tendency & Dispersion
MIT‑Level Study Notes
(Target: Advanced university students – BITSAT aspirants, statistics & probability majors)
Goal: Derive the classical measures from first principles, prove their key properties, explore edge cases, and extend the discussion to robust and optimal estimators.
Prerequisites: Elementary real analysis, basic probability theory, and elementary statistics.
1. Introduction
In any dataset (\mathcal{X}={x_1,\dots,x_n}\subset\mathbb{R}) we wish to summarize:
- Central tendency – a representative location.
- Dispersion – spread of the data around that location.
These summaries are the statistical functionals of the empirical distribution (F_n) induced by (\mathcal{X}).
We will:
| Concept | Notation | Definition |
|---|---|---|
| Empirical CDF | (F_n(x)) | (\frac{1}{n}\sum_{i=1}^n \mathbf{1}_{{x_i\le x}}) |
| Sample mean | (\bar{x}) | (\frac{1}{n}\sum x_i) |
| Sample median | (\tilde{x}) | (\text{argmin}_m \sum |
| Sample mode | (\hat{x}) | Value maximizing frequency |
| Sample variance | (s^2) | (\frac{1}{n-1}\sum (x_i-\bar{x})^2) |
| Sample standard deviation | (s) | (\sqrt{s^2}) |
| Range | (R) | (\max x_i-\min x_i) |
| Inter‑quartile range | (IQR) | (Q_3-Q_1) |
Remark: All quantities except the mean are functionals of (F_n); for continuous populations they become functionals of the true distribution (F).
2. Central Tendency
2.1 Mean (Arithmetic)
Definition
[
\bar{x} = \frac{1}{n}\sum_{i=1}^n x_i.
]
Derivation from First Principles
The mean is the least‑squares minimizer of the quadratic loss:
[ \bar{x} = \arg\min_{m\in\mathbb{R}}\sum_{i=1}^{n}(x_i-m)^2. ]
Proof:
[
\sum_{i=1}^{n}(x_i-m)^2
= \sum (x_i^2 -2mx_i+m^2)
= \sum x_i^2 -2m\sum x_i + nm^2.
]
Differentiating with respect to (m) and setting derivative to zero:
[
-2\sum x_i + 2nm = 0\quad\Rightarrow\quad m = \frac{1}{n}\sum x_i = \bar{x}.
]
Properties
- Linearity: (\overline{aX+b} = a\bar{X}+b).
- Additivity: (\overline{X+Y}=\bar{X}+\bar{Y}).
- Unbiased for IID samples: (\mathbb{E}[\bar{X}]=\mu).
- Minimum variance among unbiased linear estimators (Gauss–Markov).
Edge Cases
- Outliers: The mean is highly sensitive; a single extreme value can shift (\bar{x}) dramatically.
- Skewed distributions: The mean may lie outside the bulk of data.
2.2 Median
Definition
[
\tilde{x} = \arg\min_{m\in\mathbb{R}}\sum_{i=1}^{n}|x_i-m|.
]
Derivation
The absolute loss is minimized by any median (when (n) odd, unique; when (n) even, any value in ([x_{(n/2)},x_{(n/2+1)}])).
Proof:
Let (x_{(1)}\le\dots\le x_{(n)}).
For (m) in ([x_{(k)},x_{(k+1)}]), exactly (k) observations are (\le m) and (n-k) are (\ge m).
The derivative of (\sum |x_i-m|) w.r.t. (m) equals ((n-k)-k).
Setting derivative to zero gives (k=n/2), i.e. median.
Properties
- Robustness: Influence function bounded ((\pm 1)).
- Breakdown point: (1/2) – up to (50%) contamination can be tolerated.
- Distribution‑free: For continuous (F), (\tilde{X}) has variance (\frac{1}{4n[f(\mu)]^2}).
Edge Cases
- Discrete data: Median may not be unique; any value in the interval works.
- Ties: If many equal observations, the median may be that value.
2.3 Mode
Definition
[
\hat{x} = \arg\max_{x}#{i: x_i=x}.
]
- For continuous distributions, mode is defined via density maximization.
- For discrete data, it is the most frequent value.
Properties
- Non‑unique: Multiple modes possible (multimodal distributions).
- Sensitive to binning (for continuous data).
- Non‑linear functional –