Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
Key finding
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
One saved utterance per language, five timed repeats after two warmups. This replays the decoder only; production defaults stayed unchanged.
Data
Reviewed aggregate results with explicit units, sources and qualification limits. Different cohorts remain separate.
BF16 was the fastest tested waveform decoder
RTX 3090 · same Spanish codes · 21.5481 s audio · five timed repeats
- FP32390.07
- BF16129.54
- Mixed BF16173.79
- INT8 W8A16 · dequantized180.34
- INT8 W8A8 · custom kernels276.71
median decoder milliseconds
Decoder only, including waveform copy. Autoregressive generation, loading and compilation are excluded.
View data & source
| Configuration | Value |
|---|---|
| FP32 | 390.07 |
| BF16 | 129.54 |
| Mixed BF16 | 173.79 |
| INT8 W8A16 · dequantized | 180.34 |
| INT8 W8A8 · custom kernels | 276.71 |
Origin: reviewed report table. Revision 7e6fdbe3b1d1.
The English code replay showed the same ordering
RTX 3090 · same English codes · 15.8824 s audio · five timed repeats
- FP32292.42
- BF1696.88
- Mixed BF16129.68
- INT8 W8A16 · dequantized135.39
- INT8 W8A8 · custom kernels203.29
median decoder milliseconds
One utterance in this language. This is not a complete request, listening test or production deployment.
View data & source
| Configuration | Value |
|---|---|
| FP32 | 292.42 |
| BF16 | 96.88 |
| Mixed BF16 | 129.68 |
| INT8 W8A16 · dequantized | 135.39 |
| INT8 W8A8 · custom kernels | 203.29 |
Origin: reviewed report table. Revision 7e6fdbe3b1d1.
Write-ups
Report drama3-decoder-precision-20261016/MEASUREMENTS.md
Hold the acoustic codes fixed
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
The measured candidate
BF16 was fastest in this test and reduced the Spanish standalone-process peak allocation from 2,630.3 to 1,707.5 MiB. Waveform SNR against FP32 was 36.87 dB in Spanish and 37.18 in English. These are signal-error measurements, not perceptual quality scores.
Weight-only INT8 dequantized for BF16 computation on each call. The W8A8 variant used custom integer kernels but remained slower than BF16 and had greater waveform departure. It introduced one Spanish substitution relative to the FP32 ASR transcript; agreement with that imperfect transcript is not zero error against ground truth.
What has not shipped
Timing includes code reconstruction and waveform copy, while excluding model loading, compilation, reference encoding and text-to-code generation. Other GPU work was active elsewhere in the machine. The production FP8 generator and FP32 decoder remained unchanged.
BF16 is the candidate for an integrated regression test. AutoRound INT4 and dynamic FP8 are proposals, with no result claimed here. Blinded listening and complete serving validation remain separate gates.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Speech & codecs
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.
Longer Spanish phrases exposed the voice-transfer cost
Six phrases per language were synthesized with the production Supertonic stack, converted, then scored with an independent Parakeet recognizer. The experiment used public demo narrator references and did not alter the production application.
Speaker resemblance improved on longer phrases, but Spanish word error increased and conversion added over a second.