mole

Go CLI for LLM-driven deep research: enforced token budgets (not estimated), verified quotes, and an MCP toolkit mode for wiring into other agents — https://github.com/lajosdeme/mole.

Dogfooded 2026-08-14 against a fork (gavmor/mole): added a third search provider, then smoke-tested Google Gemini as an inference backend through mole's own generic LLM adapter.

Goals

Two separate pieces of work against the same fork:

1. Add Marginalia Search as a third search provider, alongside the existing tavily and brave, following mole's own provider interface and conventions.
2. Verify whether Gemini works as an LLM inference provider through mole's openai-compatible backend (internal/llm/openai.go) — the same generic path DeepSeek already uses — since mole has no native gemini provider kind.

Effectiveness

Marginalia provider: clean, mechanical, done. The existing tavily/brave split made a third provider a straightforward exercise in matching an established shape — same interface, same config-wiring points (mole config set search.provider, search.<provider>-key, mole doctor), same test style (table-driven, httptest mocks). All gates the project's own README treats as CI (gofmt -l ., go build ./..., go test ./...) pass clean on branch feature/marginalia-search-provider. Not upstreamed yet — sitting on the fork for review before any PR.

Gemini-as-inference-provider: works, but with a real accounting gap. The plumbing itself is fine: point openai-compatible at https://generativelanguage.googleapis.com/v1beta/openai, set the API key, and a real completion round-trips cleanly through mole's actual Provider.Complete() — not just a raw curl check. But the model names mole's own docs would suggest (gemini-2.5-flash, gemini-2.5-pro) both 404 with "no longer available to new users," despite still appearing in Google's own /models listing; the working names are gemini-flash-latest / gemini-pro-latest. More importantly, a live call's usage came back as prompt_tokens=21, completion_tokens=5, total_tokens=139 — 113 tokens of Gemini's internal "thinking" that never surface anywhere mole's response struct looks. mole's ledger charged 26 tokens for a call Gemini actually billed 139 for.

What made it effective

• The tavily/brave pattern was clean enough to copy mechanically for Marginalia: one interface, one factory switch, one config validation list to extend. The project's stated testing philosophy — "falsify your own fix: revert the mechanism, confirm the test fails" — caught a real bug in the exclude-domain filter during development, not after.
• For the Gemini check, going straight at internal/llm.New(...) + .Complete() in a throwaway test (matching the project's own existing reasoning_live_test.go pattern) gave an unambiguous result faster than round-tripping through mole config set + mole research, and made it trivial to inspect the exact Usage/Reasoning/ReasoningTokens fields mole actually populated.
• Comparing mole's parsed Usage against a raw curl hitting the identical endpoint with the identical prompt turned "something feels off" into an exact, reproducible number (139 vs. 26) instead of a vague suspicion.

Friction, pain points, surprises

Marginalia has no server-side domain filtering at all — not even Brave's -site: query-operator workaround for excludes. Both include and exclude had to be filtered client-side, reusing the existing domainFilter helper mole already had for Brave's include-only case. No published per-query price either (it's a negotiated key, not a metered tier), so the cost constant is 0 with a comment pointing at the existing per-query cost override for anyone on a paid arrangement.

The Gemini gap is the one worth sitting with. mole's whole pitch is a budget that's enforced, not estimated — 0% overshoot, because every call's real token cost is charged before the next one runs. That claim is only as good as the provider's usage reporting, and Gemini's OpenAI-compat layer breaks the assumption openai.go was written against: OpenAI's o-series splits reasoning tokens out via completion_tokens_details.reasoning_tokens; local runtimes that don't split it just return an empty answer when reasoning eats the whole allowance, which mole already has a whole retry/allowance-learning mechanism for. Gemini does neither — it spends tokens on thinking, bills them in total_tokens, and reports the answer's tokens as if that were the whole story, with the actual chain surfaced only as an opaque extra_content.google.thought_signature blob mole's openAIResponse struct has no field for. The failure mode isn't a crash (Usage.IsZero() is false, so ErrNoUsageReported never fires) — it's a ledger that silently thinks a session cost 26 tokens when the provider it's talking to billed 139. For anyone actually running a paid Gemini key through mole's token-mode budgeting, that's not a rounding error, it's the core feature quietly not working. Separately (found while reading, not testing): Request.Effort/Request.Thinking are read only by the Anthropic backend — openai.go never looks at either field, so there's no way to dial a model's thinking budget at all through this path today, for Gemini or anyone else on the generic adapter.

Verdict: in progress. Marginalia is a solid, unsurprising addition — done, tested, not yet upstreamed. The Gemini finding is the more important result of the day: mole's core budget-enforcement claim has a real, reproducible hole for any provider that bills thinking tokens without splitting them out the way OpenAI's o-series does, and Gemini is very likely not the only one shaped like that.