Q6 saved space; predicted MOS did not settle naturalness
Q6 and Q8 quantized the same 651 main matrices and 37 external tokenizer matrices in execution. Unsupported tensors stayed FP32. The Q6 checkpoint was produced directly from the original FP32 source.
Key finding
Q6 reduced the main checkpoint by 18.2% versus Q8, while their English predicted-MOS difference remained uncertain.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
Six phrases, two seeds and fixed model order on a live M5 desktop. Spanish UTMOSv2 is exploratory; it is not a validated human rating.
Data
Reviewed aggregate results with explicit units, sources and qualification limits. Different cohorts remain separate.
A real reduction in main checkpoint storage
Main checkpoint only · external tokenizer excluded
- FP322,711.1
- Q8 G64877.8
- Q6 G64718.4
decimal MB
Storage is not runtime memory or uniform compute precision. Unsupported tensors remain FP32.
View data & source
| Configuration | Value |
|---|---|
| FP32 | 2,711.1 |
| Q8 G64 | 877.8 |
| Q6 G64 | 718.4 |
Origin: reviewed report table. Revision 7e6fdbe3b1d1.
Write-ups
Report chatterbox-multilingual/omlx-q6-q8-2026-10-09/README.md
A smaller multilingual checkpoint
Q6 and Q8 quantized the same 651 main matrices and 37 external tokenizer matrices in execution. Unsupported tensors stayed FP32. The Q6 checkpoint was produced directly from the original FP32 source.
Main files were about 2,711 MB for FP32, 878 for Q8 and 718 for Q6. Observed process RSS peaks were 3.226, 1.521 and 1.372 GiB. Across the six-prompt, two-seed corpus, aggregate RTF was 1.386, 0.740 and 0.705 respectively.
What the quality proxies could say
Strict ASR edits were 11/150 for FP32, 10/150 for Q8 and 13/150 for Q6. English predicted MOS differed by just +0.0036 for Q6 versus Q8, with a phrase-bootstrap interval spanning zero. Pitch range and within-clip variation were measured, but they do not establish which model sounds more natural.
An operational choice with limits
Q8 remained the preferred default, supported by the small intelligibility observation and user preference. Q6 offered a real storage reduction. Neither result proves a general perceptual winner, and the sampled chip-energy estimates should not be treated as calibrated whole-device energy.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Speech & codecs
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.