Laya (Convai)
Evaluated 2026-10-10 under task t_b34371c8 (concourse/laya-pipeline.yml, builds #1–#3) as a non-autoregressive System-One decision provider on blades '68.
Verdict: viable with caveats. Fastest raw decision latency (52 ms) and modest VRAM (2.75 GiB), but exports ambiguous confidence semantics that can mislead downstream agent deciders.
Goals
Test whether Convai's non-autoregressive Laya model (v0.4.2, laya-serve-cuda:latest, 421M params) can serve the canonical POST /v1/systemone contract on the lab 3090 inside a leased serving window managed by Concourse GPU-lock.
Effectiveness
N=1 smoke test against the standard login form (email, signin, cancel).
| Metric | Measured |
|---|---|
Serves /v1/systemone |
Yes — via unmodified upstream laya-serve |
| VRAM attributable | 2,812 MiB peak (2.75 GiB, sampled via nvidia-smi --query-compute-apps) |
| Warm latency | 52.6–55.0 ms steady-state (builds #2 and #3) |
| First request latency | 540–1488 ms |
| Model load / window open | Fully healthy within 2m30s wall clock lock-to-lock |
| Choice | Picked email at probability 0.5967, signin 0.2579, cancel 0.1455 |
The model picked the correct input field without hallucinating button clicks, and decision outputs were deterministic across all five requests in three separate builds.
What made it effective
- Non-Autoregressive Speed: Once warm, decisions take ~52 ms, the fastest in the tested cohort.
- Low Footprint: 2.75 GiB resident VRAM leaves over 21 GiB free on an idle card.
- Stock Upstream Distribution: Ran the official Docker Hub image without custom compilation or gated weights.
Friction / pain points / surprises
- The Confidence Trap: The payload returns three conflicting confidence metrics:
The field named plain"probabilities": {"email": 0.5967, "signin": 0.2579, "cancel": 0.1455}, "confidence": 0.1462, "answer_confidence": 0.5967, "x_jev_confidence": 0.3950confidenceis the lowest (0.1462) despiteemailbeing the correct choice with 0.60 probability. Any downstream agent client thresholding onconfidence >= 0.5will reject correct actions. Downstream deciders must explicitly map toanswer_confidenceor recompute marginals. - cu130 Forced: Upstream pins PyTorch 2.14.0 with CUDA 13.0 wheels, pulling a 9.3 GB container image that took 6.5 minutes to pull and build initially.
- Host Shim Coupling: Copied from AutoJev, the host-side shim required patching
queue_is_idle(): it previously reportedUNKNOWNwhen the ComfyUI container was absent, refusing to lease windows even though an absent renderer cannot hold GPU locks.