Index
All experiments
69 reviewed experiment stories. Search the findings, compare the recorded conditions, and follow each result back to its source.
69 shown ·
A benchmark harness that keeps dry runs out of the results
The new harness layer provides four kinds of runs: streamed API throughput, GGUF quant comparison, task evaluation through lm-eval, and command-level memory/load measurement. It uses standard-library Python for the core, with optional external model and…
The archive now includes standard-library harnesses for throughput, quant comparison, task evaluation and VRAM measurement.
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.
Longer Spanish phrases exposed the voice-transfer cost
Six phrases per language were synthesized with the production Supertonic stack, converted, then scored with an independent Parakeet recognizer. The experiment used public demo narrator references and did not alter the production application.
Speaker resemblance improved on longer phrases, but Spanish word error increased and conversion added over a second.
More diffusion steps did not reliably improve the voice match
The complete indexed collection contained fifty-one converted variants. Each was independently embedded against its exact selected narrator and source waveform. Twenty-nine used four steps, eleven eight and eleven twelve.
Matched six-sentence similarity intervals crossed zero for both eight-minus-four and twelve-minus-four steps.
Supertonic: half the storage did not mean faster CPU speech
Three variants ran the same twelve phrases per language at eight synthesis steps, twice each: FP32, FP16 with the duration component retained, and a mixed INT8 graph. The latter quantized only the verified vector-estimator path; a directory label was not…
FP16 roughly halved the model storage but increased measured median time to first audio on this CPU workload.
Voice resemblance improved before call readiness did
Seed-VC2 converted existing Supertonic speech with four, eight and twelve diffusion steps. The selected public audiobook references were thirty seconds long, but the upstream wrapper used only the first twenty-five seconds.
Timbre-only transfer improved speaker-embedding similarity on short greetings, while style transfer could destroy the words.
A faithful decoder can still be the slower pipeline
The complete community 6-bit MLX route had a 1.359-second warm median and RTF 0.225. The ordinary CoreML/ONNX Python adapter measured 4.974 seconds and RTF 0.749. Their payloads were 0.524 and 3.331 GiB respectively, with process RSS peaks of 1.181 and 4.963…
The community MLX pipeline was faster in this Python comparison; a controlled decoder diagnostic favored the other acoustic route.
A local LatAm conversion failed the content gate
A local MLX conversion of the LatAm package was compared with the existing multilingual Q8 and an original FP32 route. All active weights were loaded strictly, but that alone did not establish correct speech.
The deployed multilingual Q8 remained the choice: the local LatAm Q8 conversion added nine content edits over 142 words.
Holding speech tokens fixed made the Q8 advantage clearer
This follow-up applied the Turbo method to Multilingual FP32, Q8 and Q6. Thirty-six full generations and thirty-six controlled decoder outputs formed the primary cohort; warm timing and FP32 self-repeats were retained separately.
Q8 had lower controlled decoder error than Q6 on the paired English and Spanish clips.
The decoder swap passed its word-error gate
Both routes used the same community 6-bit MLX T3 generator. The hybrid replaced only the audio stage with the unchanged published ONNX conditional decoder. No CoreML execution occurred in this hybrid.
No hybrid output added word errors in 40 exact-token comparisons, but the ONNX decoder made the route slower.
MOSS: the best similarity score did not mean the fewest word errors
Nine generator profiles were tested against BF16 plus the same Argentina LoRA. The English study retained ninety complete generations, controlled decoder runs and a timing recheck. A separate Spanish extension used one generation per phrase.
Q6/Q8 built directly from BF16 had the fewest Spanish ASR edits, while the older Q4 had higher speaker cosine.
Q6 saved space; predicted MOS did not settle naturalness
Q6 and Q8 quantized the same 651 main matrices and 37 external tokenizer matrices in execution. Unsupported tensors stayed FP32. The Q6 checkpoint was produced directly from the original FP32 source.
Q6 reduced the main checkpoint by 18.2% versus Q8, while their English predicted-MOS difference remained uncertain.
The audio was identical. The benchmark numbers changed.
Repeating a benchmark can reveal measurement instability without changing the model output. This five-profile Chatterbox Turbo rerun preserved all thirty comparison WAVs byte for byte.
All 30 comparison WAV hashes matched the earlier run, while timings and four ASR transcripts changed.
What six bits preserved in Chatterbox Turbo
Five checkpoints were compared in two cohorts. Complete generations allow speech-token choices and pacing to change. Controlled decoder runs reuse the original tokens and conditioning, allowing a narrower acoustic comparison.
Community affine 6-bit had the closest controlled decoder output among the tested quantized candidates; oQ6 led the new oQ variants.
An unfinished gauntlet cannot name a winner
This campaign compared Qwen UD-Q4_K_XL with MTP2 against Bonsai through coding harnesses. It stopped at the 70 GiB free-space guard, retaining the incomplete evidence rather than risking the workstation.
The 15 matched cells tied: six passes and nine failures for each model.
Native AWQ saved idle memory and cost speech time
This follow-up separates native AutoRound AWQ from the earlier rejected emulations. It measures whether reduced residency also improves the speech request.
The native AWQ candidates saved roughly 0.40–0.69 GiB at idle but were slower on the tested speech workloads.
Spanish Chatterbox: smaller weights, uneven speech
The original BF16 baseline had recorded WER 0.42 and real-time factor 0.76 in this small Spanish screen. A dynamic, emulated FP8 path recorded WER 0.24 and an isolated RTF around 1.05. The user favored that candidate during listening, which is a personal…
The emulated FP8 variant was a useful listening candidate; several smaller alternatives produced silence, tails or slower speech.
The fastest kernel was not the qualified speech stack
The fitting setup combined HQ4, SDPA, the INT8 GPU codec, 0.8-second chunks, a 200-character budget and a 256-step cap. The BF16 comparison peaked at about 22.8 GiB, beyond the RTX 3060.
HQ4, SDPA and the GPU INT8 codec formed the qualified fitting stack; a faster kernel experiment lacked complete audio evidence.
144 recoveries, one carefully bounded task
The earlier recovery experiment proved that the pieces connected. This follow-up asks whether adding recent conversation history helps a worker resume after an independent reviewer has repaired the code.
All 144 supervised continuations completed; a smaller handoff used fewer worker tokens.
A context setting is not a context test
A server can accept a large context configuration without demonstrating useful work at that length. I separated configured capacity, an actual request, and a successful result before making another speed claim.
The largest verified successful coding prompt was 49,124 tokens; larger configured windows remained unproven.
Making MOSS fit: the codec and the chunk budget
The HQ4 generator occupied about 6.24 GiB and the codec about 3.31 GiB. A one-shot path reached roughly 13 GiB and ran out of memory; 0.8-second streaming chunks kept the resident stack near 9.55 GiB on the 12 GB RTX 3060.
GPU codec decoding cut several seconds from each tested render, but the token cap still required shorter text chunks.
A faster speech encoder with the same 50 transcripts
Could an RTX 3060 transcribe the same speech with less GPU memory by accelerating the encoder while retaining the existing decoder?
TensorRT FP16 encoding raised measured throughput from 128.80× to 159.76× real time with identical transcripts.
Why a busy speech GPU felt slower
A fast isolated encoder does not guarantee a responsive shared service. I tested ASR, embeddings and speech synthesis together on the RTX 3060 and retained failures and health checks.
Mixed speech, embedding and synthesis traffic increased latency even when every request passed.
A smaller speech model earns its place on the iPhone
The model that feels fastest after loading may still make you wait for its first words. This paired iPhone experiment compared English and Spanish transcription, kept a suspended timing sample out of both engines, and weighed accuracy against first-use…
Aidana transcribed the 47 matched clips 3.35 times faster, with observed English WER of 2.15% and Spanish WER of 5.37%.
A coding agent that knows when to ask for help
A coding agent can keep repeating a failed approach. This experiment gave a local Qwen worker an independent reviewer, then checked whether it could recover without weakening the final acceptance tests.
Three live recovery scenarios passed, each finishing with 10/10 checks.
Corrective data meets a fresh gameplay gate
Corrective trajectories were added to a fresh same-host training comparison on Ubuntu CPU. The initial and repeated results stay separate, including the control’s changed weights, so the result is not presented as a direct improvement over the earlier Mac…
On Ubuntu CPU, the corrected policy passed 10/10 full-kitchen trials against a same-host retraining control at 7/10.
When the retention gate says stop
Adding new behavior is only useful if the model keeps the behavior it already has. This training experiment put a public retention floor in front of promotion and required a rollback when a candidate damaged the parent.
All three registered learning rates crossed the retention floor after eight steps and rolled back.
A second dish, a new frozen test
A first-dish policy did not automatically know how to reach the second dish. The new paired frozen test rejected the first candidate, then evaluated the corrected candidate against the same unchanged baseline.
The corrected candidate passed 10/10 paired two-dish trials against a frozen baseline at 0/10.
Eight clears are still not a promotion
A full-kitchen clear needed survival and next-stage readiness as well as reaching the objective. The Mac protocols measured partial success, but the strict ten-of-ten gate kept both candidates from promotion.
The v3 protocol measured 8/10 full-stage clears versus 0/10 for its paired baseline; the strict gate rejected it.
Learning to play from pixels
The visual policy had to act from owned frames and recent controls rather than hidden emulator state. The first-dish baseline established a small replayable milestone; the later curricula ask harder questions without rewriting that first result.
The first-dish baseline produced six successful recorded evaluations across five starting positions, with exact replay.
The reusable RGBA learning pipeline
The reusable pipeline joins owned RGBA frames, bounded controls, teacher data, learned policies and immutable replay evidence. Pluto is the exercised target; the notebook does not turn planned support for other games into demonstrated gameplay.
Owned frames, bounded controls, saved state, teacher-data export and exact replay form one auditable pipeline.
What counts as a successful episode?
An episode can reach a goal and still fail its acceptance contract. These evaluator fixes tightened survival, terminal-state and replay checks while keeping historical outcomes attached to their original gates.
The evaluator tightened survival, terminal-state and exact-replay checks without rewriting historical experimental outcomes.
Finding a local model for the inbox
Inbox work needs accurate structured answers and reasonable response times. The benchmark ran the same 50 tasks across local model, runtime and hardware configurations, with thinking on and off, to make that tradeoff visible.
Bonsai-2-27B PQ2_0 with MTP passed 48/50 tasks with thinking on, at a 15.8-second median on an RTX 3060.
Teaching a 4B model to read tmux status
The tmux status problem is narrow enough to ask whether a small trained classifier can handle it locally. The project followed teacher-labeled examples through QLoRA, merged weights, quantization and a lazy service, while retaining the limits of that teacher…
The classifier matched its teacher on 192/202 examples, with macro-F1 0.863.
An image editor that respects the pixels you keep
A masked edit should leave the protected part of an image alone. The studio project checked a real object removal, compared decoded pixels outside the mask, and traced the edit through a working tool flow.
A real masked object removal finished in 24.39 seconds and preserved every unselected decoded pixel exactly.
Character LoRAs: identity, motion, and continuity
Character adapters need more than completed training. This project records the initial identity and motion runs, later action probes, continuity work and identity corrections, while leaving private reference media outside the article.
Three initial adapters completed: 600 LTX steps and two Qwen identity runs of 800 steps each.
Fifteen pictures, one photographic-style adapter
The style-adapter experiment started with a large mixed image folder and a much smaller selected dataset. Deduplication, actual selected images and training provenance matter more than calling the result a single-person identity adapter.
213 files became 97 unique decoded images; the curated random selection used 15 distinct photographs or photo-like images.
Making the keyboard lock key really sleep the phone
The keyboard lock key left the phone displaying lock-screen pixels. The device experiment traced the keyboard-specific handler and power transition, checked the display state directly, and left reboot persistence as a separate follow-up.
A keyboard-specific handler preserved the lock gesture and the display reached OFF roughly 0.7 seconds later.
Masked cleanup before a fresh LoRA
Cleanup happened before a fresh adapter was trained. The experiment saved lossless copies, verified unchanged pixels outside each mask, and documented that the resulting dataset still contained mixed subjects.
25 distinct images were cleaned into lossless copies, preserving dimensions and all decoded pixels outside each mask.
Moving speech decode onto the GPU
Moving the final audio decode onto the GPU shortened both that stage and total render time. The deployment A/B also records the extra GPU memory, so the speed improvement can be weighed against its cost.
Decode dropped from 3.36 seconds to 0.24; total render wall time dropped from 35.1 to 24.4 seconds.
Putting a watermark gate in front of training
A binary watermark screen became a gate in the training workflow. The project checked known positive controls and recorded the excluded image without treating a model label as infallible visual proof.
The binary screen accepted 24/25 cleaned images; both known positive controls were detected.
A complete image stack on one 24GB GPU
An image stack that fits on one GPU still has to produce useful pictures. This project measured the full pipeline and paired FP8 with opt-in INT8 across eight fixed scenes, keeping changed visual details beside the timing gain.
Eight matched 1024-square scenes averaged 93.352 seconds with FP8 and 46.219 with opt-in INT8.
Long speech, bounded memory
Long input needed a bounded rendering path. The live service split an eight-sentence request into chunks and checked the resulting audio, while documenting the separate CLI run that remained deferred.
A 551-character, eight-sentence input rendered in two chunks with 48.0 seconds of healthy audio.
Round-trip tests expose a broken encoder
A round-trip test challenged the assumption that a codec was healthy because its decoder worked. The record compares the encoder-plus-decoder path and qualifies the new FP8 implementation on the named clips.
The new 1.76GB FP8-E4M3 codec reached FP32-control parity in the measured round-trip test with its encoder intact.
The clone bug was in the reference encoder
The clone quality problem was narrowed to the reference encoder. Changing that stage to FP32 recovered the measured target similarity while the output decoder stayed INT8, leaving accent and listening questions open.
Using FP32 reference encoding lifted AWQ WavLM target similarity from 0.8475 to 0.9556 while retaining INT8 output decode.
The faster image pipeline that produced nothing
The two-GPU path appeared faster until the outputs were inspected. Every image was invalid, so the experiment records a failure and an unresolved sharding suspect rather than promoting the shorter timing.
The balanced two-GPU path took 48.1 seconds but every image was invalid flat gray noise.
The same story, six precision combinations
The same text made it possible to compare generator and codec precision separately. The six combinations exposed a small decoder similarity effect alongside a much more audible difference in pacing.
Generator precision affected the measured ranking; changing the output codec shifted similarity by at most about 0.001.
Balancing GPU experts when the model will not fit
The quant was larger than the three GPUs could hold on their own. The experiment tested explicit GPU expert splits with CPU offload and measured the decode effect under the same large-context workload.
An explicit 12/12/12 GPU expert split plus 12 CPU blocks measured 31.1 tokens per second.
Pipeline parallelism, INT4, and a 262K context
Pipeline parallelism and INT4 offered a different serving route for the large model. This record qualifies its long context, single-stream decode and two-stream aggregate throughput, with vision and MTP disabled.
The patched vLLM stack measured 67.4 tokens per second for one stream and 73.1 aggregate for two.
A speech service that sleeps between requests
A speech service need not hold all of its resources between requests. The lazy-server experiment checked wake-up, warm rendering, reap and re-wake, while preserving the known short-prompt silence behavior.
The recorded service used about 10.3GB resident memory; first render took 14.8 seconds and warm render 5.7.
A quantization job goes silent
A full-spec quantization job stopped making progress after only six blocks. The incident record follows the monitor, idle GPU and checked failure explanations without inventing a confirmed cause.
The full-spec job stopped after six of 36 blocks; the monitor detected the dead wrapper and idle GPU.
AWQ speech: fast enough, with a memory catch
AWQ speech delivered a useful render rate, but fitting a smaller GPU was still a separate question. The benchmark kept prompt health, pooled real-time factor, decode speed and peak memory in the same record.
AWQ passed 33/33 prompts with pooled real-time factor 0.43 and 35.8 tokens per second.
Do you really need the bigger embedding model?
The larger embedding model asks for more memory and compute. This paired bilingual retrieval experiment checked whether the extra cost bought a detectable quality improvement on the actual evaluation set.
The 1B encoder embedded 1,004 documents per second; the 8B encoder reached 181, with no proven retrieval-quality win.
FP32, BF16, INT8: separating codec drift from speech
Codec precision can change storage, waveform detail and reference encoding in different ways. The A/B/C experiment separated those effects using pure speech and clone cases, rather than letting transcription parity stand in for every kind of fidelity.
Pure TTS legs shared token sequences, and normalized transcription differences were zero across the tested codec legs.
Keeping the audio-sensitive layers intact
The HQ4 preview tried to save memory while protecting layers that mattered to audio. Text-token fidelity, codec behavior and staged-memory probes were checked separately, with calibration and physical-device limits still open.
The HQ4 preview matched 96.5% of text tokens in the greedy test and passed three staged-memory probes at 6.7–6.8 GiB.
Staging the speech stack under a 12GB envelope
The speech stack exceeded the target memory envelope before staging. This hardening experiment verified protected layers and quantized projections, then measured staged memory on the hardware actually available.
The staged run peaked at 6.63 GiB, down from 13.01 before staging; 252 INT4 projections passed the verifier.
Three draft tokens hit the sweet spot
More draft tokens can add work as well as save it. The sweep compared several windows on the same short-prose workload, then kept the longer prompt and concurrent-stream measurements separate.
Draft length three reached a short-prose mean of 88.16 tokens per second, versus 64.67 with drafting off.
From a model download to a working local service
A downloaded model does not yet make a useful service. This deployment notebook follows formats, cache settings, routing and resource constraints; its dated experiments explain which configuration produced each result.
The deployment notebook records the EXL3 serving family, cache choices and real runtime constraints.
The EXL3 quantization journey
The EXL3 work began with official BF16 weights and followed the conversion into a deployable artifact. The journey keeps the intermediate checks and remaining work visible rather than declaring success from a running process.
The notebook follows official BF16 weights through EXL3 conversion, artifact checks and deployment work.
A model proxy earns its promotion
A proxy must keep its serving contracts while managing several local model families. This promotion record follows the merge through regression checks and separates new live validation from code-reviewed operational notes.
The promoted merge passed 107 tests with one environmental skip.
The best checkpoint was not the last one
Training longer did not make every speech checkpoint better. The fixed Spanish transcription screen compared the saved steps, found an earlier best checkpoint, and retained the distinction between word accuracy and how a voice sounds.
Polish step 400 measured 6.1% word error rate; the final step 1500 measured 42.9%.
Rebuilding calibration for an agentic quant
Calibration was rebuilt around agentic work rather than assumed adequate from a generic matrix. The hard task, new quantization artifacts and subsequent draft-decoding probes stayed separate evidence gates.
The clean agentic importance matrix recovered the hard T3 task; later draft decoding improved measured throughput.
A draft head, a rollback fix, and a midnight incident
Draft decoding exposed a rollback issue, and the fix needed a fresh comparison at different sampling settings. The later disk incident is retained because it invalidated the scheduled long-context validation.
The rollback fix brought draft decoding to parity at temperature 0.7 and a 24% gain at greedy sampling.
Making room for a 200K context
Getting a large model to load is the first hurdle; completing a nearly full context is another. This experiment qualified a specific three-GPU profile with a warm decode measurement and a real long-context request.
The warm proxy decode median was 59.17 tokens per second; a 198,960-token request passed with GPU headroom.
DFlash2 on a single RTX 3090
Speculative decoding needs to earn its speedup on more than an easy completion. This single-GPU DFlash2 record separates predictable prose from a realistic prompt and records the effect of draft acceptance.
Ornith measured 210.16 tokens per second on the realistic 256-token prompt and 270.40 on predictable prose.
Less thinking, fewer tokens: a template A/B
A chat template can change how much a model writes, even when the weights stay the same. The paired test measured accuracy, completion tokens, total tokens and wall time across stock and Sharp templates.
Sharp reduced completion tokens by 20.6%, 20.1% and 16.0% across the three deployments.
The real cost of a five-second video
Five seconds of generated video can take many minutes of compute. The H3 archive groups real 20-step runs by resolution and keeps quantization, placement, mode and complete decode checks attached to each timing.
At 960×544, dual INT8 completed a 124-frame, 20-step run in 354.580 seconds.
No experiments match these filters.