Cache Memory – Study Notes
(Program: NCO – Cyber Olympiad – Computer Fundamentals & Hardware)
1. Simple Explanation
Cache memory is a small, very fast type of volatile memory that sits between the CPU and the main memory (RAM).
Its job is to hold the most frequently or recently used data and instructions so the CPU can fetch them quickly, reducing the average time it takes to read from RAM.
Think of it as a “short‑term memory” that keeps a copy of the “hot” information the CPU needs right now.
2. Key Points
| Concept | What It Means | Why It Matters |
|---|---|---|
| Hierarchy | L1 → L2 → L3 (and sometimes L4) | Each level is larger but slower; the CPU prefers the fastest. |
| Size | 16 KB – 1 MB (per core) | Larger cache can hold more data but costs more silicon and power. |
| Speed | 1–10 × faster than RAM | Reduces memory latency from ~100 ns to ~1–3 ns. |
| Associativity | Direct‑Mapped, Set‑Associative, Fully‑Associative | Determines how a memory block is mapped to a cache line. |
| Replacement Policy | LRU, FIFO, Random | Decides which cache line to evict when a new block must be loaded. |
| Write Policies | Write‑Through, Write‑Back | Control when data is written to main memory. |
| Cache Coherence | MESI protocol, Snooping | Keeps multiple CPU caches consistent. |
| Primary vs. Secondary Cache | L1 (instruction + data) | L2/L3 shared across cores |
| Typical Hits/Misses | 90–99 % hit ratio on modern CPUs | High hit ratio translates to higher performance. |
3. Worked Example
Scenario
A CPU needs to execute an instruction that loads data from address 0x7FF0.
The cache configuration:
- L1 Data Cache: 32 KB, 4‑way set associative, 64 B line size.
- Cache line formula:
- Index bits = log2(number of sets)
- Tag bits = remaining address bits after offset & index
- Offset bits = log2(line size)
Step‑by‑Step
| Step | What Happens | Calculation |
|---|---|---|
| 1 | Determine offset | 64 B line ⇒ 6 offset bits (2⁶ = 64). |
| 2 | Determine index | 32 KB cache = 32 × 1024 B = 32768 B. Lines per set = 4 ⇒ number of lines = 32768/64 = 512. Sets = 512/4 = 128 ⇒ 7 index bits (2⁷ = 128). |
| 3 | Determine tag | 32‑bit address → 32 − 6 (offset) − 7 (index) = 19 tag bits. |
| 4 | Extract fields | |
0x7FF0 in binary: 0000 0111 1111 1111 0000 |
- Offset: last 6 bits →
000000(address is line‑aligned). - Index: next 7 bits →
1111110(binary126). - Tag: remaining 19 bits →
000000011111111. | | 5 | Lookup | CPU checks set 126 for a line with that tag. | | 6 | Result | • Hit: If tag matches → data returned instantly (≈1 ns).
• Miss: If no match → cache line fetched from RAM (≈100 ns), placed in set 126 (evicting an existing line if set is full). |
Takeaway: Even a single memory address requires the CPU to parse the address into offset, index, and tag to decide whether a cached copy exists.
4. Quick Revision Summary
- Cache = fast, small memory between CPU and RAM.
- Levels: L1 (fastest, smallest) → L2 → L3 → (sometimes L4).
- Associativity controls mapping; higher associativity reduces conflict misses.
- Replacement policies (LRU, FIFO, Random) decide evictions.
- Write‑back saves bandwidth by updating RAM only on eviction; write‑through writes immediately.
- Cache coherence (MESI) maintains consistency in multi‑core systems.
- Hit ratio is the most critical performance metric; aim for >95 % on modern CPUs.
- Typical sizes: L1 ~32 KB, L2 ~256 KB–1 MB per core, L3 ~8–64 MB shared.
Pro Tip: When debugging performance, always check cache‑miss rates first; a high miss rate often indicates poor spatial/temporal locality in your code.
Happy studying, and good luck on the NCO Cyber Olympiad!