AWQ speech: fast enough, with a memory catch
AWQ speech delivered a useful render rate, but fitting a smaller GPU was still a separate question. The benchmark kept prompt health, pooled real-time factor, decode speed and peak memory in the same record.
Key finding
AWQ passed 33/33 prompts with pooled real-time factor 0.43 and 35.8 tokens per second.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
The 13.05-GiB decode peak blocks a direct 12GB deployment. Style and listening questions remain separate from pass counts.
Write-ups
Report moss-tts-awq/awq-3090-2026-09/README.md
The question
AWQ speech delivered a useful render rate, but fitting a smaller GPU was still a separate question. The benchmark kept prompt health, pooled real-time factor, decode speed and peak memory in the same record.
From the original notebook
Benchmark + samples for the 4-bit AWQ build
(lemuriandezapada/MOSS-TTS-v1.5-awq-int4, 6.3 GB) against the BF16 and HQ4
baselines. Same 33 prompts, seeds (1234) and sampling throughout; Marlin/bf16
kernels, SDPA, batch 1, one RTX 3090.
Code: groxaxo/moss-tts-hq4-quant/awq-3090
(full harness, env pins, reproduction commands).
Headline
- 33/33 pass, 0 silent — pooled RTF 0.43 (BF16 0.59, HQ4 0.88), 35.8 tok/s (BF16 25.5, HQ4 17.0), peak VRAM 13.05 GiB combined.
- Spanish: all 8 prompts healthy, no truncation/silence (see samples).
- Register control (instruction-guided female, F0 ≥195 Hz): 3/15 (HQ4 1/21, BF16 1/4) — usable control restored, both languages pass.
- Caveats: Marlin bf16 is run-to-run nondeterministic on sm86 (treat rates as single-draw estimates); a gptqmodel unload-retention quirk keeps decode peak at ~13 GiB (12GB deployment blocked until resolved).
Contents
docs/—comparison.json(BF16/HQ4/AWQ),benchmark_awq.json(per-prompt records), register reports,single_run.json,FINDINGS.mdsamples/— smoke (Spanish), 3 Spanish suite prompts, 3 register passes, plus the 33-promptprompts.txt
| Sample | What |
|---|---|
awq3060-single.wav |
Smoke test, Spanish |
es-conv-1.wav / es-long-1.wav / longform-es-1.wav |
Spanish short → 58 s longform |
register-en-3.wav (203.4 Hz) / register-es-3.wav (198.3 Hz) |
Seeds-1235 female passes |
register-es-3-seeds501.wav (219.2 Hz) |
Seeds-501 female pass (same seed HQ4 barely passed at 196.7) |
Keep exploring
Original experiment record. Workstation paths have been generalized. Detailed measurements below retain their original workload and validation boundaries.
Supporting notebooks
More in Speech & codecs
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.