Native AWQ saved idle memory and cost speech time
This follow-up separates native AutoRound AWQ from the earlier rejected emulations. It measures whether reduced residency also improves the speech request.
Key finding
The native AWQ candidates saved roughly 0.40–0.69 GiB at idle but were slower on the tested speech workloads.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
Two warm repeats of one held-out 50-word story per language. English-only models do not become qualified Spanish systems by producing audio.
Data
Reviewed aggregate results with explicit units, sources and qualification limits. Different cohorts remain separate.
Idle-memory savings came with slower generation
English held-out 50-word story · two warm repetitions · RTX 3060
- Turbo · original0.446
- Turbo · native AWQ0.564
- English G64 · original0.888
- English G64 · native AWQ1.221
- Multilingual v2 · original1.014
- Multilingual v2 · native AWQ1.344
real-time factor
Lower is faster. These are separate model comparisons, not a multilingual quality leaderboard.
View data & source
| Configuration | Value |
|---|---|
| Turbo · original | 0.446 |
| Turbo · native AWQ | 0.564 |
| English G64 · original | 0.888 |
| English G64 · native AWQ | 1.221 |
| Multilingual v2 · original | 1.014 |
| Multilingual v2 · native AWQ | 1.344 |
Origin: reviewed report table. Revision 7e6fdbe3b1d1.
Write-ups
Report chatterbox/autoround-awq-rtx3060-2026-10-08/README.md
Test the native route separately
This follow-up separates native AutoRound AWQ from the earlier rejected emulations. It measures whether reduced residency also improves the speech request.
English Turbo RTF increased from 0.446 to 0.564. Regular English G64 changed from 0.888 to 1.221; Multilingual v2 from 1.014 to 1.344; the Spanish fine-tuned candidate from 0.416 to 0.543. Lower RTF is better, so these were regressions despite the idle-memory savings.
Intelligibility did not move uniformly
For supported Spanish Multilingual v2, RTF changed from 1.018 to 1.322 while local ASR edits fell from two to zero out of fifty words. That small observation does not establish quality parity over future sentences.
An original-FP32 default-voice study is retained as a separate cohort. Different voice conditioning, warm-up counts and model families should not be mixed into one ranking.
The choice
Use native AWQ when its specific memory saving justifies the measured latency cost and the intended language passes its own checks. Neither the smaller file nor a successful API response replaces listening and output validation.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Speech & codecs
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.