Library / Computer / CMP-03
Computer
Average memory access time
Typical delay per memory access with a cache in front.
- Ref
- CMP-03
- Rev
- v1
- Field
- Computer
- Result
- tavg [ns]
thit the time for a cache hit, m the miss probability (fraction of accesses that miss) and tpenalty the extra time on a miss. One cache level.
Most memory accesses hit the fast cache, but a few miss and must wait for slow main memory. The average time is the hit time plus the miss rate times the extra penalty.
Calculator
- Penalty share of average time83.33 %
- Slow-down against all hits6 ×
What drives the result
Each variable is moved across its typical range while the others stay at the values above. The longest bar is the key driver.
- A small miss rate still dominates: at 5% misses and a 100 ns penalty, the penalty adds five times the hit time.
- Lowering the miss rate (larger cache, better locality) usually pays more than speeding up the hit.
- Multi-level caches apply the formula recursively: the miss penalty of L1 is the average time of L2.
Chart data as tables
Ranked variables
| Variable | Typical range | Lowest result | Highest result | Swing (ns) | Local step | Change in result |
|---|---|---|---|---|---|---|
| m Miss rate | 0.1 to 30 % | 1.1 | 31 | 29.9 | +10% | +8.33 % |
| tpenalty Miss penalty | 20 to 300 ns | 2 | 16 | 14 | +10% | +8.33 % |
| thit Hit time | 0.3 to 5 ns | 5.3 | 10 | 4.7 | +10% | +1.67 % |
Sweep of Miss rate
| Miss rate (%) | Average access time (ns) |
|---|---|
| 0.1 | 1.1 |
| 3.837 | 4.838 |
| 7.575 | 8.575 |
| 11.31 | 12.31 |
| 15.05 | 16.05 |
| 18.79 | 19.79 |
| 22.52 | 23.52 |
| 26.26 | 27.26 |
| 30 | 31 |
Sweep one variable
Variables and how each one contributes
Average access time
What the equation gives you. Everything else feeds it.
Hit time
Time for an access that hits the cache. Adds directly.
Miss rate
Percentage of accesses that miss the cache. A strong driver: each extra percent adds tpenalty/100.
Miss penalty
Extra time to fetch from the next level. Scales with the miss rate.
Worked example with the values above
- Convert the miss rate: 5% = 0.05.
- Penalty contribution: 0.05 × 100 ns = 5 ns.
- Average time = 1 + 5 = 6 ns.
Assumptions and limits
- One cache level, constant miss penalty.
- Miss rate and penalty measured for the same workload.
- Same time units for both delays.
Common mistakes
- Entering the miss rate as a fraction when a percentage is expected.
- Counting hit time again inside the penalty.
- Ignoring the miss rate varies between programs.
Statistics and uncertainty
Miss rate is measured with a profiler and varies by program and phase. A ±20% error in miss rate becomes about ±17% in average time when the penalty share is 83%.
Where it is used
- Choosing cache size and memory technology.
- Estimating real-time deadlines in embedded systems.
- Tuning data layouts for speed.
Maths you need
- Probability: miss rate is the chance of a miss
- Averages: weighted by probability