Arithmetic intensity (FLOP/byte, log scale)
Attainable FLOP/s (log scale)
Memory roof
Compute roof
Gap below the roofs
Decode attention
memory-bound
Prefill GEMM
compute-bound
Fused block with
pipeline stalls