The previous page argued that peak FLOPS is a ceiling you rarely reach. The roofline model, introduced by Williams, Waterman and Patterson in 2009, answers the natural follow-up: which ceiling is actually stopping you, and why. It does it with a single graph and one derived quantity — arithmetic intensity.
Arithmetic intensity: FLOPs per byte
Arithmetic intensity (AI) is, in NERSC's phrasing, "the ratio of total floating-point operations (FLOPs) performed by a given code or code section, to the total data movement (Bytes) required to support those FLOPs." Units: FLOPs per byte. It measures data reuse. A dense matrix multiply reuses each loaded value across a whole row and column, so its AI is high — tens to hundreds. A vector add (c = a + b) reads two numbers, writes one, does a single FLOP: AI near 1/12 in FP32. That difference decides everything.
Two roofs and a ridge
Plot achievable performance (FLOP/s, log scale) against arithmetic intensity (FLOPs/byte, log scale). The hardware imposes two limits:
- a flat compute roof — peak FLOP/s the chip can retire, a horizontal line;
- a slanted memory roof — peak memory bandwidth (bytes/s), which on this plot becomes a diagonal of slope 1, because
FLOP/s = AI × bytes/s.
The two lines meet at the ridge point (also called machine balance). Left of the ridge, the diagonal is lower — you are memory-bound: performance is capped by bandwidth, and adding compute does nothing. Right of the ridge, the flat line dominates — you are compute-bound: you're limited by the arithmetic units, and only more FLOP/s or better precision helps. Your kernel's AI is a vertical line; where it hits the roof is the best you can hope for.
perf
(GFLOP/s)
^ compute roof (peak FLOP/s)
| _________________________
| /
| memory / ridge point = peak FLOP/s ÷ peak BW
| roof / (machine balance, in FLOPs/byte)
| (slope 1)/
| /
+---------+----------------------------------> arithmetic intensity
MEMORY-BOUND | COMPUTE-BOUND (FLOPs/byte, log-log)
Ridge for an H100-class GPU ≈ ~990 TF (FP16) ÷ ~3.35 TB/s ≈ ~295 FLOPs/byte.
A kernel below that intensity CANNOT reach peak — bandwidth caps it first.
The ridge is a sobering number. On a modern accelerator with ~1 PFLOP/s of low-precision compute and a few TB/s of memory bandwidth, machine balance sits in the hundreds of FLOPs per byte. Any kernel below that intensity — most of them — is bandwidth-limited before the compute units are even warm.
Why this is the model for AI inference
LLM inference makes the point vividly. During autoregressive decode, you generate one token at a time; each step streams the entire weight matrix (and the KV cache) out of memory to multiply against a single activation vector. That is a matrix-times-vector, not matrix-times-matrix: almost no reuse, arithmetic intensity near 1. On the roofline it sits far left, pinned to the memory diagonal — the GPU's enormous compute roof is irrelevant. This is why decode throughput scales with memory bandwidth, why HBM generation matters more than TFLOPS for serving, and why batching (which raises AI by reusing loaded weights across many sequences) is the single biggest inference win. The roofline predicted all of it before you profiled a thing.
Why it matters: "make it faster" is not a plan; the roofline turns it into a decision. Memory-bound? Chase data movement — better layouts, fusion, higher batch, more bandwidth. Compute-bound? Chase FLOP/s — lower precision, better tensor-core utilization. Optimize against the wrong roof and you spend weeks moving a ceiling that was never the limit. Every serious GPU profiler (NVIDIA Nsight Compute) now draws this plot for you — because it's the fastest way to know what to fix.