Jeff (firelex)

Evaluated 2026-10-11 under task t_5fd75cdd (jeff-systemone-pipeline.yml, build #7) as a lightweight, fast System-One decision provider for browser navigation on blades '68.

Verdict: recommended for adoption. With a 2.0 GiB VRAM footprint and 46.9 ms warm latency, Jeff is the only candidate that coexists with active desktop GPU tenants while delivering calibrated decision confidence out of the box.

Goals

Test whether Jeff (mstrasser/jeff-base + mstrasser/jeff-adapter-nav rev v1.3, upstream 3720e7c6) can serve the canonical POST /v1/systemone decision contract under Concourse GPU-lock on the lab 3090, fitting the 24 GB card with substantial headroom, while producing sane choices and calibrated confidence on a browser login form.

Effectiveness

N=1 smoke test against a login state (email-field, sign-in-button, cancel-button, goal "sign in as alice@example.com").

Metric Measured
Serves /v1/systemone Yes — HTTP 200, documented choice shape
VRAM attributable 2,066 MiB (~8.4% of the 24 GB card, 21,776 MiB free headroom)
Warm latency 46.9 ms median over 9 warm requests (46.5–48.3 ms; 41.2 ms on base model)
First request latency 29.5 s (Triton JIT CUDA compilation)
Model cold load 10.0 s (base + nav adapter)
Decision sanity Picked email-field at 0.980 (confidence 0.970), sign-in-button 0.017, cancel-button 0.003

The decision distribution is well-calibrated and concentrated: cancel-button (which abandons the goal) is suppressed, and email-field is prioritized ahead of the sign-in button. The untouched base model also made the sane choice (0.931 email-field, confidence 0.896), with the adapter sharpening the margin.

What made it effective

Friction / pain points / surprises