Paper, researchers, and primary source
Inference, evaluation & serving / inference_architecture
Multi-query attention shares one set of key and value heads across attention queries, reducing memory bandwidth during autoregressive decoding.
Contribution 1
Introduced shared key-value projections with multiple query heads.
Contribution 2
Reduced incremental-decoding cache size and memory traffic.
Contribution 3
Compared quality and decoding performance on sequence-to-sequence tasks.
Research context
inference_architecture / 2019
Fast Transformer Decoding: One Write-Head is All You Need places multi query attention inside the broader inference architecture discussion at Google Research, with kv cache supplying a second analytical lens. Read together, the source records three advances: Introduced shared key-value projections with multiple query heads; Reduced incremental-decoding cache size and memory traffic; and Compared quality and decoding performance on sequence-to-sequence tasks. Keeping those moves together prevents decoding from being detached from its evidence. For an implementation review, the relevant consequence is that sharing key-value heads can trade representational capacity for speed, and the original experiments predate current large decoder-only serving stacks.
Methods and evidence reading
1 cataloged method notes
Method 1
The experimental design in Fast Transformer Decoding: One Write-Head is All You Need tests introduced shared key-value projections with multiple query heads and reduced incremental-decoding cache size and memory traffic against the paper's documented baselines, datasets, model variants, or systems workloads.
How to read the evidence
The evidentiary value of Fast Transformer Decoding: One Write-Head is All You Need comes from the relationship among its reported moves. Two entries define the method-level claim: Introduced shared key-value projections with multiple query heads; and Reduced incremental-decoding cache size and memory traffic. The cataloged result is: Compared quality and decoding performance on sequence-to-sequence tasks. On that basis, sharing key-value heads can trade representational capacity for speed, and the original experiments predate current large decoder-only serving stacks. The catalog nevertheless records that evidence for multi query attention in Fast Transformer Decoding: One Write-Head is All You Need covers numerical precision, service-level objectives, reported models, comparison baselines, accelerator hardware, request shapes, and software revisions; behavior beyond that documented envelope remains untested. Reproduction work should separate genuine multi query attention transfer from behavior caused by a changed kv cache setup. An independent check of Fast Transformer Decoding: One Write-Head is All You Need needs a fixed multi query attention comparison, a declared kv cache variation, and saved cases where Reduced incremental-decoding cache size and memory traffic does not carry over.
Findings in the source record
1 paper-specific findings
- The reported evidence in Fast Transformer Decoding: One Write-Head is All You Need supports compared quality and decoding performance on sequence-to-sequence tasks.
Practical implication for AI builders
Google Research / 2019
Proposed BrokenGPT application
Research blueprint / proposed status
Proposed route placement / multi query attention
For a proposed BrokenGPT experiment based on Fast Transformer Decoding: One Write-Head is All You Need, compare multi-query and multi-head checkpoints at matched training quality, then record cache bytes per token, throughput, latency, and task regressions. Keep the multi query attention path isolated, versioned, and attributable to this research record.
Proposed acceptance test / kv cache
Validate the proposed multi query attention route against the paper's reported outcome: Compared quality and decoding performance on sequence-to-sequence tasks. For the Fast Transformer Decoding: One Write-Head is All You Need prototype, collect decode latency, cache footprint, throughput, and quality parity and audit kv cache slices independently before promoting the multi query attention configuration.
Proposed decision boundary / decoding
Balance memory bandwidth, conversion effort, and model fidelity before promoting the proposed decoding design. Because A deployment review should isolate safety checks, authentication, tokenization, workload drift, networking overhead, and failure recovery when translating the multi query attention contribution into a different system, adoption remains conditional on replay under BrokenGPT's selected model, runtime, and policy configuration.
Limitations, verification, and source
Boundaries recorded with the paper
Limitations
- Evidence for multi query attention in Fast Transformer Decoding: One Write-Head is All You Need covers numerical precision, service-level objectives, reported models, comparison baselines, accelerator hardware, request shapes, and software revisions; behavior beyond that documented envelope remains untested.
- A deployment review should isolate safety checks, authentication, tokenization, workload drift, networking overhead, and failure recovery when translating the multi query attention contribution into a different system.
PRIMARY SOURCES
- 01Fast Transformer Decoding: One Write-Head is All You Need
Google Research — Primary primary arXiv paper / 6 November 2019 / Noam Shazeer