batch 1, 32 heads, d
k
= 128, FP16, causal · median of 20 runs · RTX 4090, PyTorch 2.12
explicit, eager (6 kernels)
explicit, torch.compile (3 kernels)
FlashAttention (1 kernel)