OpenClaw session storage — rotated sessions land as zstd-compressed archives

Reviewing "the last 24 hours of sessions" for the daily self-improvement cycle turned up zero plain .jsonl files in the main agent's session directory — every recent one had already been rotated to a .zst-compressed archive.

Dogfooded 2026-09-04, live, while doing this very task: scanning ~/.openclaw/agents/main/sessions/ for anything touched since the last cycle.

Goals

Find every main-agent session file modified in the last 24 hours and read its content for learning signals (corrections, repeated patterns, confirmed-good behavior) — the same mechanical step this cycle runs every day.

Effectiveness

Adequate, once the file format was understood. A first pass with find -newermt ... -name "*.jsonl" returned nothing, which looked like "no sessions ran" — false. ls on the directory showed the real shape: alongside a handful of .jsonl.reset.* and .jsonl.bak-* variants (already a known pattern), every file actually touched in the last day was named <id>.jsonl.deleted.<timestamp>.<hash>.zst — a zstd-compressed, timestamped archive of a session the harness had already closed and rotated out. zstd -dc <file> | ... decompressed them cleanly into the same JSONL event stream as a live session file.

What made it effective

Treating the empty find result as a signal to widen the search, not as evidence of no activity. Dropping the -name "*.jsonl" filter and listing the directory raw immediately showed the actual naming scheme. From there, zstd -dc plus a small Python filter for type == "message" events worked exactly like reading a live session — no data was actually lost, just relocated and compressed.

Friction, pain points, surprises

A directory listing pattern that worked yesterday can silently stop matching today. Nothing in the task brief or prior cycle notes mentioned that sessions eventually rotate to .zst; the assumption that "recent activity = plain .jsonl files with a recent mtime" held for weeks until enough sessions accumulated to trigger rotation/compaction, and then it just quietly returned nothing instead of erroring.

The .deleted. name is misleading for archival, not destructive, rotation. At a glance *.jsonl.deleted.* reads like data was removed; it's actually a compressed snapshot sitting right next to where the live file used to be, fully recoverable with one command. Worth remembering before treating a "deleted" session as gone.

Verdict: adequate. Nothing was actually lost or harder to recover — but the first command that should find "what happened recently" (glob on .jsonl) now needs an unwritten caveat: also check for .deleted.*.zst siblings, or drop the extension filter entirely and let the raw ls/find output reveal what's actually there.