Drama3: BF16 decoded the same codes about three times faster

Hold the acoustic codes fixed

The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.

The measured candidate

BF16 was fastest in this test and reduced the Spanish standalone-process peak allocation from 2,630.3 to 1,707.5 MiB. Waveform SNR against FP32 was 36.87 dB in Spanish and 37.18 in English. These are signal-error measurements, not perceptual quality scores.

Weight-only INT8 dequantized for BF16 computation on each call. The W8A8 variant used custom integer kernels but remained slower than BF16 and had greater waveform departure. It introduced one Spanish substitution relative to the FP32 ASR transcript; agreement with that imperfect transcript is not zero error against ground truth.

What has not shipped

Timing includes code reconstruction and waveform copy, while excluding model loading, compilation, reference encoding and text-to-code generation. Other GPU work was active elsewhere in the machine. The production FP8 generator and FP32 decoder remained unchanged.

BF16 is the candidate for an integrated regression test. AutoRound INT4 and dynamic FP8 are proposals, with no result claimed here. Blinded listening and complete serving validation remain separate gates.

Evidence and publication boundary

This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.

• Source record 1
• Source record 2