Jensen’s Inequality & Convex Functions
(MIT‑level study notes for an Algebra/Analysis course)
Scope – We treat convexity from first principles, prove Jensen’s Inequality in full generality, explore edge cases, and finish with a challenging research‑style problem.
1. Preliminary Notions
1.1 Real‑valued functions on (\mathbb{R}^n)
Let (f: \Omega \to \mathbb{R}\cup{+\infty}) with (\Omega\subseteq\mathbb{R}^n) convex.
Throughout we assume (f) is proper (not identically (+\infty)) and finite on a non‑empty open subset of (\Omega).
1.2 Convex sets and epigraphs
A set (C\subseteq\mathbb{R}^n) is convex iff [ \forall x,y\in C,;\forall \lambda\in[0,1]\quad \lambda x+(1-\lambda)y\in C. ]
The epigraph of (f) is [ \operatorname{epi}f={(x,t)\in\mathbb{R}^n\times\mathbb{R}\mid t\ge f(x)}. ] A function is convex iff its epigraph is convex.
2. Convex Functions – First Principles
2.1 Definition
Definition (Convexity).
(f:\Omega\to\mathbb{R}\cup{+\infty}) is convex if for all (x,y\in\Omega) and all (\lambda\in[0,1]), [ f(\lambda x+(1-\lambda)y)\le\lambda f(x)+(1-\lambda)f(y). ] Equality for all (\lambda) means (f) is affine on the segment.
2.2 Equivalent characterizations
| Property | Statement | Notes |
|---|---|---|
| 1 | Epigraph convexity | Immediate from definition. |
| 2 | Secant inequality | Same as definition. |
| 3 | First derivative monotonicity | If (f) is differentiable, (f(x)\ge f(y)+\nabla f(y)\cdot(x-y)). |
| 4 | Second derivative positive semidefinite | If (f\in C^2), (\nabla^2 f(x)\succeq 0). |
| 5 | Supporting hyperplane | For each (x_0\in\Omega) there exists (v) such that (f(x)\ge f(x_0)+v\cdot(x-x_0)). |
| 6 | Jensen’s inequality | For any probability measure (\mu) on (\Omega), (f!\bigl(\int x,d\mu\bigr)\le \int f(x),d\mu). |
Proof sketch of (3) ⇔ (6): Using the subgradient inequality from (3) and integrating over (\mu) gives (6). Conversely, differentiate (6) w.r.t a Dirac mass.
2.3 Important Examples
- (f(x)=|x|^p) for (p\ge1) on (\mathbb{R}^n).
- (f(x)=e^{\langle a,x\rangle}), (a\in\mathbb{R}^n).
- The logarithm is concave; its negative is convex.
- Indicator functions of convex sets: (\chi_C(x)=0) if (x\in C), (+\infty) otherwise.
3. Jensen’s Inequality – The Core Result
3.1 Discrete Version
Theorem (Discrete Jensen).
Let (f:\Omega\to\mathbb{R}\cup{+\infty}) be convex, (x_1,\dots,x_m\in\Omega), and (a_1,\dots,a_m\ge0) with (\sum a_i=1).
Then [ f!\Bigl(\sum_{i=1}^m a_i x_i\Bigr);\le;\sum_{i=1}^m a_i f(x_i). ]
Proof (by induction on (m)).
Base case (m=2) is the definition.
Assume true for (m-1). For (m) points set
[
\bar x=\frac{1}{1-a_m}\sum_{i=1}^{m-1}a_i x_i,\qquad \lambda=1-a_m.
]
Then
[
\sum_{i=1}^m a_i x_i=\lambda\bar x+ a_m x_m.
]
Apply convexity to (\bar x,,x_m