Spanish Chatterbox: smaller weights, uneven speech
The original BF16 baseline had recorded WER 0.42 and real-time factor 0.76 in this small Spanish screen. A dynamic, emulated FP8 path recorded WER 0.24 and an isolated RTF around 1.05. The user favored that candidate during listening, which is a personal…
Key finding
The emulated FP8 variant was a useful listening candidate; several smaller alternatives produced silence, tails or slower speech.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
Three Spanish prompts, one RTX 3060, and changing co-resident memory conditions. The FP8 path here is emulated, not a native FP8 speed result.
Write-ups
Report chatterbox-t3-quant-2026-10-08/README.md
Start with the spoken output
The original BF16 baseline had recorded WER 0.42 and real-time factor 0.76 in this small Spanish screen. A dynamic, emulated FP8 path recorded WER 0.24 and an isolated RTF around 1.05. The user favored that candidate during listening, which is a personal observation rather than a blinded preference study.
The failures are part of the comparison
GPTQ reached RTF around 0.63–0.74 but produced silence tails, with WER 0.44. The GPTQ plus RPO variant reached WER 0.55. AWQ produced silence and WER 1.0. W4A8 recorded WER 0.37 but an RTF around 12, far too slow for the intended use.
Memory readings came from different co-resident states. They cannot serve as a uniform peak-memory leaderboard, and this screen does not establish an architectural limit for the GPU.
What I learned
A quantization label is not an acceptance test. Keep intelligibility, silence detection, end-of-speech behavior and listening checks alongside memory and timing. The next comparison needs more speakers and text, fixed residency, matched timings and blinded listening before choosing a default.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Speech & codecs
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.