Software Is NothingFounder & engineer (Software Is Nothing, LLC) · 2026
AI model training & distillation
Two translation systems I trained on my own GPU to replace a paid API: a full fine-tune of Opus-MT and a LoRA knowledge-distillation pipeline. I pick the one that ships by whether it keeps safety-critical facts intact, not by loss curves. German and Spanish are live in production.
What it is
The translation layer behind tailharbor.eu renders shelter listings and long-form guides into more than a dozen languages. Paying a per-character cloud API for that at scale is expensive, and it sends the data out of house. So I trained the models myself, on a GPU I own, and serve them locally. This is real gradient-descent training, not prompt-wrangling: two distinct tracks, each with its own reason to exist.
Two training tracks
- Full fine-tune (Opus-MT / Marian): a full fine-tune (not just an adapter) of the Opus-MT EN→DE and EN→ES models on roughly 104,000 (DE) and 147,000 (ES) sentence pairs, served with CTranslate2 int8 for fast, cheap CPU inference.
- LoRA knowledge distillation (EuroLLM): a EuroLLM-1.7B student trained to imitate a much larger teacher model through a LoRA adapter (~29M trainable parameters), then merged and exported to GGUF (Q8_0) for local serving via
llama.cpp. German and Spanish run in production.
The part that matters: choosing a model by what it must never get wrong
Translation quality metrics don't catch the failure that actually matters here: a fluent translation that quietly drops a critical instruction.
- Checkpoint selection isn't loss or BLEU. A metric-best checkpoint once silently dropped a safety clause (a "must be muzzled" instruction) from a translation. The selection gate now requires ≥ 97% recall on a set of safety-critical facts, plus an integrity tiebreak: a model that translates beautifully but loses a "muzzle required" line is rejected, however good its score.
- Adversarial review before merge. Every candidate passes an adversarial pre-merge review (one pass surfaced nine issues) and a three-part ship gate that includes a required human spot-check (≥ 27/30). No model ships on automated scores alone.
Feeding the pipeline from its own mistakes
- Reject-queue → retraining. Rejected translations are captured, repaired, and re-verified. The failures become the next round of training data instead of getting thrown away.
- Auditing the auditor. When an LLM judge was labelling training data, I measured its actual precision (~22%) instead of trusting it, replaced it with deterministic rules, and recovered on the order of 70,000 usable training rows that would otherwise have been discarded.
Why it's here
Anyone can call a translation API. The engineering is in owning the model: I trained it on hardware I control, I refused to ship it on a pretty score when a dropped word could mislead someone adopting an animal, and I built the data-quality discipline that keeps it improving. The result cut a recurring paid-API cost and kept the data in-house, and it's the clearest example of how I treat "AI" as an engineering problem with correctness gates, rather than a magic box.