MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
Key finding
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
Three Spanish prompts, two seeds and one reference voice. Different bit budgets and a custom CUDA/Triton runtime prevent isolating the quantizer algorithm.
Data
Reviewed aggregate results with explicit units, sources and qualification limits. Different cohorts remain separate.
Q2 failed the tiny Spanish quality screen
Three Spanish prompts · two seeds · same cached codes and FP32 decoder
- Original BF161.53
- Sensitivity Q2 · rejected121.85
- Sensitivity Q31.45
- AWQ41.70
- AutoRoundQ32.04
normalized character error (%)
Insertions can push CER above 100%. Different bit budgets prevent isolating the quantizer algorithm.
View data & source
| Configuration | Value |
|---|---|
| Original BF16 | 1.53 |
| Sensitivity Q2 · rejected | 121.85 |
| Sensitivity Q3 | 1.45 |
| AWQ4 | 1.70 |
| AutoRoundQ3 | 2.04 |
Origin: reviewed report table. Revision 7e6fdbe3b1d1.
Write-ups
Report moss-tts-awq/experimentos-v15-oQ-AWQ-auto-round-3090/README.md
Quantize from the original
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
The useful and rejected candidates
Original BF16 had mean WavLM cosine 0.9837, UTMOSv2 3.192 and normalized pooled character error 1.53%. AWQ4 measured 0.9828, 3.166 and 1.70%. AutoRoundQ3 measured 0.9818, 3.241 and 2.04%.
Sensitivity Q2 fell to cosine 0.6845 and character error 121.85%; insertions can make character error exceed 100%. It was rejected. Nominal Q2/Q3 labels did not describe uniform effective storage: protected tensors and metadata raised effective bits per weight.
Memory and speed are separate
Recorded generator peaks ranged from 7.22 GiB for Q2 to 17.78 GiB for BF16. Those peaks included the resident reference decoder and exclude the separately measured extra allocation of FP32 decoding. The combined decoder RTF is a pooled timing estimate, not a single-process serving benchmark.
AWQ4 was a fidelity candidate and AutoRoundQ3 promising on automatic proxies. Neither substitutes for blinded human listening, and this CUDA prototype does not demonstrate Apple Silicon performance.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Speech & codecs
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.
Longer Spanish phrases exposed the voice-transfer cost
Six phrases per language were synthesized with the production Supertonic stack, converted, then scored with an independent Parakeet recognizer. The experiment used public demo narrator references and did not alter the production application.
Speaker resemblance improved on longer phrases, but Spanish word error increased and conversion added over a second.