Residuals Are All You Need: ReSQ for Shared-Reference Compression of Large Language Models

Abstract

Low-bit weight quantization reduces model storage and memory traffic, but its errors are not distributed uniformly. A small set of weights can dominate layer-output distortion, while the residual errors left by a base quantizer may repeat or cluster. We propose ReSQ, a shared-reference compression framework that represents a weight as a low-bit base value plus an optional residual selected from a compact dictionary. ReSQ couples that representation to activation-weighted selection, explicit position coding, and a fused execution path.

The framework has two branches. ReSQ-Q is lossy and optimizes output distortion subject to the complete serialized bit cost. ReSQ-E is exact: it predicts each stored 16-bit weight word and reference-codes the reversible XOR correction, with raw escapes when a dictionary entry is unavailable. We derive the rate objective, prove limited reconstruction properties, and state the conditions under which shared references can or cannot save space. We do not report empirical wins in this version. Instead, we publish hypotheses and a five-month validation protocol designed to reject the method if metadata cost, quality loss, or kernel overhead defeats it.

Let a transformer layer contain a weight matrix W and receive a calibration activation matrix X. A groupwise b-bit quantizer constructs a base matrix B. The residual matrix E = W − Bmeasures what the base representation loses. Ordinary quantization either accepts that loss or increases precision. ReSQ asks a narrower question: can high-value parts of E be recovered by reusing a small vocabulary of residual values?

The answer depends on two forms of structure. Value structure determines whether many residuals share useful representatives. Positional structure determines whether the locations and assignments can be described cheaply. A residual dictionary without an honest position bill can look compact on paper while producing a larger file.

Flatten a weight group into coordinates i = 1,…,n. Let qᵢ be its quantized integer, s its scale, mᵢ ∈ {0,1} a correction mask, cᵢ a dictionary assignment, and R = {r₁,…,rₖ} a shared scalar residual dictionary. The lossy reconstruction is:

The same construction extends to vector blocks, multiple residual stages, asymmetric zero-points, or layer-shared dictionaries. Version 0.1 starts with scalar, group-local residuals because that is the smallest design whose position cost and kernel behavior can be audited cleanly.

B = sQ

The low-bit base matrix consumed by an ordinary groupwise quantized multiply.

E = W − B

The residual error available to be approximated or exactly encoded.

M

A sparse binary mask identifying coordinates whose correction is worth storing.

C

Dictionary assignments, encoded only where the mask is active.

R

A small residual dictionary learned from selected errors.

Weight-space mean squared error is a weak proxy for model behavior. ReSQ therefore uses calibration activations to measure the layer-output error and a Lagrange multiplier to price the complete representation:

Effective bits per weight is Rtotal / n. Rmeta includes group boundaries, code lengths, offsets, escape tags, padding, and alignment. No result may quote only the low-bit payload. A target such as four bits per weight is achieved only by the final byte-identical serialized artifact, divided by the actual parameter count.

Sweeping λ traces an empirical rate-distortion frontier. The correct comparison is not merely against FP16, but against established methods at the same complete size and on the same held-out evaluation.

Let hᵢ approximate the activation-weighted sensitivity of coordinate i. Under a diagonal second-order approximation, correcting base residual eᵢ with dictionary value r[cᵢ] produces the estimated gain:

If a correction costs ℓᵢ bits after mask and index coding, the selection rule is not simply “largest residual.” It is a budgeted choice that favors positive Δᵢ / ℓᵢ, while accounting for the fact that nearby positions may share a cheaper block mask or run. The practical encoder alternates among dictionary fitting, assignment, position coding, and mask selection until the objective stops improving.

1. Quantize a group to obtain the base and residual errors.
2. Estimate activation statistics on a fixed calibration split.
3. Fit candidate residual dictionaries at several sizes.
4. Measure marginal distortion reduction and incremental encoded length.
5. Select corrections under a global or layerwise bit budget.
6. Re-encode, recompute the true serialized rate, and reject any non-improving configuration.

ReSQ evaluates several interchangeable codecs because sparsity alone does not imply compressibility. A random sparse set can require nearly as much addressing as a dense map; clustered or locally repeated assignments may be much cheaper. Each block chooses the smallest valid encoding, with the codec identifier included in its rate.

The encoder records both logical bits and physical bytes. Alignment required by a GPU, CPU SIMD path, or storage container is not free. Results will publish codec histograms by layer so an average cannot hide pathological layers.

Arithmetic residuals do not by themselves guarantee bitwise reconstruction because floating-point addition may round. The exact branch therefore operates on the stored 16-bit words. Let uᵢ be the original word andvᵢ a deterministic predicted word derived from the base representation. Define an exact correction symbol:

Frequent dᵢ symbols receive dictionary references; uncommon symbols use a raw 16-bit escape. The position or symbol stream may then be entropy coded. Every word has a valid path, so decoding is exact regardless of whether the data compresses. Exactness applies to stored weights. It does not promise identical end-to-end outputs across different hardware, kernel orders, or floating-point environments.

Proposition 1: exact weight-word reconstruction

For every original word uᵢ, if the encoder stores either a dictionary reference resolving todᵢ = uᵢ XOR vᵢ or a raw escape containing that dᵢ, the decoder returns uᵢ exactly. The proof is immediate because XOR is self-inverse: vᵢ XOR (uᵢ XOR vᵢ) = uᵢ.

Proposition 2: non-increasing diagonal proxy error

Fix the base, mask, and non-negative sensitivities. If the residual dictionary contains zero and each active coordinate chooses the entry minimizing hᵢ(eᵢ − r)², then the diagonal proxy error cannot exceed the base error. Zero reproduces the uncorrected choice, so the minimizer is never worse coordinate by coordinate; summing preserves the inequality.

Proposition 3: a sufficient condition for a useful correction

Under the Lagrangian objective, a candidate correction improves the local objective whenever its estimated distortion reduction exceeds its incremental rate penalty: Δᵢ > λℓᵢ. Shared block headers couple nearby decisions, so the implemented encoder must evaluate groups of corrections when their costs are not separable.

These propositions establish reconstruction and optimization properties only. They do not prove downstream accuracy, a universal compression ratio, generalization beyond calibration data, or a hardware speedup.

For activations A, ReSQ-Q decomposes the layer into a base multiply plus sparse shared corrections:

The intended kernel loads packed base values and scales, traverses correction metadata in tile order, fetches a small residual dictionary into fast local storage, and accumulates corrections before writing the output tile. It must not reconstruct the full high-precision matrix. Whether this is faster depends on batch size, memory bandwidth, sparsity, locality, decode branching, and the target device.

ReSQ-E has two deployment modes: decode once into the ordinary weight representation, which can save distribution and disk space but not steady-state inference memory; or execute from the exact compressed stream, which requires a separate kernel and must be benchmarked independently.

Evaluation matrix

• Models: publicly licensed open-weight transformer checkpoints in multiple parameter ranges; exact revisions disclosed in experiment artifacts and kept separate from BrokenGPT's private serving mixture.
• Baselines: FP16 plus reproducible GPTQ, AWQ, SpQR, SqueezeLLM, AQLM, and QuIP-family configurations where compatible implementations exist.
• Quality: perplexity, task accuracy, long-context retrieval, instruction following, and layer-output reconstruction on held-out data.
• Rate: physical file bytes, effective bits per weight, codebook and scale bytes, mask/index bytes, padding, and peak decode memory.
• Systems: load time, time to first token, tokens per second, peak memory, energy where measurable, and quantization time by batch regime.
• Statistics: fixed seeds, repeated runs, confidence intervals, paired comparisons, and explicit treatment of failed or timed-out runs.

The primary lossy endpoint is held-out quality at equal physical bytes. The primary exact endpoint is exhaustive bitwise equality plus physical byte count. Speed is a separate endpoint; storage compression will never be used as a proxy for runtime acceleration.

• Residual values may not cluster enough for a small dictionary to help.
• Sensitive corrections may be positionally random, making their addresses expensive.
• Calibration activations may overfit a domain and mis-rank corrections on held-out data.
• Existing quantizers may already remove most recoverable structure at the same byte budget.
• Dictionary fetches, branches, and irregular sparse work may erase memory-bandwidth savings.
• The exact symbol stream may have entropy too close to 16 bits per word to compress.
• Lossless disk compression may still require full-size decoded weights during inference.
• Shared dictionaries and sparse residuals have substantial prior art; novelty requires a separate, professional prior-art review and cannot be inferred from this proposal.

ReSQ sits near second-order post-training quantization, activation-aware scaling, dense-and-sparse outlier handling, incoherence processing, vector and additive codebooks, and entropy coding. The combination proposed here must be evaluated against those methods, not described as novel merely because it has a new name.

How to cite this proposal

BrokenGPT Research. “Residuals Are All You Need: ReSQ for Shared-Reference Compression of Large Language Models.” Version 0.1, 1 August 2026. brokengpt.com/research/residuals-are-all-you-need.