Skip to main content

Case Study

The Foundry: An On-Device Model Line with Promotion Gates

Trained, evaluated, and gated on-device models — not API calls. The failure number is the headline.

In ProgressJun 2026 - PresentSolo Developer

Updated Jul 11, 2026

Currently BuildingModernBERT/DeBERTa Fine-TuningGold-Set EvaluationChampion/Challenger Gates

The headline number is the one that looks bad

The Foundry's keystone model is a ~150M-parameter ModernBERT grounding verifier: a small on-device classifier that decides whether a claim, or an action's parameters, are actually supported by the tool output behind them — before an agent answers or acts.

It scored 97.5% on its own synthetic holdout. On a 95-row human-labeled gold set, it scored 62%. That 35-point gap is overfitting to the corruption taxonomy, and only hand labeling caught it. The model is not in the product, because the promotion gates it has to pass are code. Publishing that failure — instead of the demo that would have hidden it — is what this case study is about.

97.5 → 62%

synthetic vs gold — the caught overfit

95% LCB

paired-bootstrap promotion bar

$0

marginal training data cost

On-device

private by architecture

Why train models when APIs exist

GBuild runs agents against personal, local context: files, messages, calendars. The questions its small models answer — is this contact important, is this claim grounded, is this document sensitive — are exactly the questions whose inputs should never leave the machine. The constraint envelope is explicit: on-device, private, roughly zero marginal cost, inside a RAM and latency budget. A frontier API fails that envelope on every axis, however good its answers.

The first model through the full pipeline is deliberately small: a Tier-0 contact-importance ranker — a small linear model over locally-aggregated communication features that drop-in replaces a hand-weighted heuristic. It ships with a Spearman rank-correlation gate and graceful fallback when no model artifact is present. Its ingest reads chat.db counts only, never message content; a corpus-readiness probe reports whether the local history is even large enough for the eval to mean anything, in counts and a day-span only.

The measurement discipline

Synthetic data, $0 marginal

Training data is generated by programmatically corrupting ground truth along a typed-field taxonomy: swap a number, break a unit, invert a claim. Each corruption is a labeled negative by construction, so the data engine costs nothing and never plateaus.

Gold sets, human-labeled

The synthetic holdout tells you the model learned the corruptions. Only a hand-labeled gold set tells you it learned the task. Both numbers get published, especially when they disagree.

Champion / challenger promotion

A challenger replaces the champion only past a paired-bootstrap 95% lower-confidence-bound significance bar on the margin. The gates are code, not judgment calls, so a persuasive demo cannot argue its way into the product.

Every gold evaluation ships with a provenance dump, data gates reject leaking rows before training, and results files (RESULTS-gold-v0.md, checkpoint discipline docs) live in the repo next to the code that produced them. A number without its command is treated as a rumor.

A Python trainer, a Rust runtime, one contract

The feature vector is a versioned interface, not a convention.

Training happens offline in Python (generate → train → eval → export a JSON artifact). Inference happens in the Rust sidecar. Between them sits a versioned feature contract: the exact features, their order, and their version travel with the model artifact, so a trainer-side feature change cannot silently skew runtime predictions. The runtime model is a trait-based abstraction with additive fallback — no artifact on disk means the heuristic keeps working, not a crash.

Fine-tuning itself targets encoder classifiers (ModernBERT / DeBERTa) rather than generative models: classification is what the product needs, encoders are what the RAM budget affords, and a champion/challenger harness can score them mechanically.

Where it's headed

The charter, eval math, corpus-readiness probe, and the Tier-0 ranker pipeline are built; the grounding verifier is in active training rounds against its gold set, and it stays out of the product until it clears the same bar everything else does. The strategy documents plan a portfolio — make, fork, or resell per niche — but the promotion rule does not bend for the roadmap.

The Foundry is one subsystem of GBuild. Read the GBuild case study

A model earns its place past a significance bar, or it stays out of the product. Both outcomes get published.

Tech Stack

AI

Python (train/eval pipelines)ModernBERT / DeBERTaSynthetic corruption enginePaired-bootstrap evaluationSpearman rank-correlation gateMLX / on-device runtimes

Backend

Rust runtime rankerFeature-contract versioningchat.db counts-only ingest (privacy)

Want to discuss the architecture?

Get in Touch
The Foundry: An On-Device Model Line with Promotion Gates | Jeffrey Michael Johnson