ai-toolkit (ostris)

LoRA training and generation for Krea 2 Turbo — one tool, both phases, no ComfyUI required.

Goals

Train a style LoRA on the David Goodsell sci-art gallery (50 images, real captions, trigger gdsll style.) for Krea 2 Turbo, then generate new images with it, entirely on a local RTX 3090 — no cloud GPU rental, no separate inference stack. Dogfooded 2026-07-31 through 2026-08-02: training completed 2000 steps; generation run against both a throwaway test prompt set and a curated Goodsell-subject set.

Effectiveness

Recommended. The same tool and the same YAML-job pattern (job: train vs job: generate) cover both training and inference — no ComfyUI graph to build, debug, or keep in sync with the training run's exact model/quantization settings. This mattered concretely: a same-day investigation into a Krea 2 Turbo ComfyUI template found it downloaded but never actually confirmed to run (no execution history, raw unedited JSON structure), while ai-toolkit's own generate job worked on the first real attempt.

What made it effective

• One YAML config format for both jobs — assistant_lora_path in a generate config points straight at the safetensors a train job just produced, no export/conversion step between training and using the result.
• qfloat8 + low-VRAM mode fit a 24GB 3090 for both training and generation without manual memory tuning.
• Checkpoints every 250 steps with sample images generated at each one, so training progress was visually inspectable without waiting for the full 2000-step run to finish.
• Per-prompt generation is genuinely just a list under generate.prompts in the YAML — batching multiple subjects in one invocation needed no scripting.

Friction / pain points / surprises

The base model is gated on HuggingFace and the failure mode doesn't say so plainly. krea/Krea-2-Turbo requires accepting a license and an HF_TOKEN — this blocked training for about two days until Gavin accepted the license and provided a token. The token then has to be passed explicitly at every invocation (HF_TOKEN=$(cat ~/code/goodsell-lora/hf-token) venv/bin/python run.py ...), not read from any persisted config — easy to forget on a fresh shell.

Quantization partially fails, silently, for a large fraction of the model, with no summary line telling you so. Model load logged dozens of individual Failed to quantize blocks.N.attn.wX: Float8Tensor dispatch: attempting to run unimplemented operator/function: func=<OpOverload(op='aten.abs'...)> lines — one per affected transformer block — but generation still succeeded. Nothing aggregates this into "N/M blocks quantized" so you're left scrolling a wall of per-block warnings to judge whether it matters.

Model load overhead dwarfs actual generation time on a cold run. ~3-4 minutes loading (text encoder download/quantization, VAE) before the first image, then ~2 minutes per image after that (25 steps, 1024×1024). A 3-image test run and a 4-image real run both spent roughly half their wall-clock time on load, not sampling.

Output filenames carry no link back to the prompt that produced them. Generated PNGs land in gen/ as <timestamp>_<idx>.png — matching an image to which of several prompts produced it means cross-referencing the run log's prompt order against file mtimes, not reading the filename.