Why a busy speech GPU felt slower
A fast isolated encoder does not guarantee a responsive shared service. I tested ASR, embeddings and speech synthesis together on the RTX 3060 and retained failures and health checks.
Key finding
Mixed speech, embedding and synthesis traffic increased latency even when every request passed.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
Repeated short phases reuse one 9.90-second ASR clip. This is saturation evidence, not a real conversation trace.
Write-ups
Report speech-to-text/parakeet-tensorrt-rtx3060-2026-10-05/concurrency-20261005/README.md
From isolated speed to shared traffic
A fast isolated encoder does not guarantee a responsive shared service. I tested ASR, embeddings and speech synthesis together on the RTX 3060 and retained failures and health checks.
The follow-up completed 9,407 requests across 24 phases. At mixed concurrency one, median ASR latency was about 240 ms, embeddings 48 ms and synthesis 234 ms. Their isolated observations were about 61, 25 and 118 ms.
More parallel work had a cost
At concurrency four, ASR latency reached 1,131 ms. ASR throughput was 34.6 times real time, below the 41.6 observed at mixed concurrency one. The phases lasted fifteen seconds with two repeats, so they describe a deliberately busy service rather than representative user traffic.
A separate bottleneck audit retained 4,044 requests and 1,671 health checks. These observations support resource contention as an operational issue; they do not prove that every slowdown came from one component.
The deployment lesson
Choose concurrency using both latency and throughput. Admission control and separate load budgets deserve a direct test before treating a benchmark maximum as a production setting.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Speech & codecs
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.