reflect
MIT · local-first · cross-harness
Selective learnings; markdown is the source of truth. Reuses the agent's own model (claude -p) + a local embedder — no extra key. First-class typed signals.
weighted4.72 / 5
Why build, not adopt — the agent-memory landscape
Eight coding-agent memory systems, four deciding axes, and an honest build-vs-adopt verdict. The short version: the tools that need no second API key tend to hoard noise; the ones that curate well make you run a server and pay a second provider; and none capture corrections, tests, git events or skill upgrades as typed signals. reflect is the row that holds all four.
Interactive version
A richer, standalone version of this page (option cards, scored matrix, the full critique) is published at explainers.stevengonsalvez.com/agent-memory-landscape. For the newcomer framing see Problem & fit.
The problem: context engineering, not a bigger file
Every coding harness gives you one persistent-memory primitive: a static instructions file you
maintain by hand (CLAUDE.md, AGENTS.md, MEMORY.md), front-loaded into the context window every
session. The window is finite and shared with the task — every line you front-load “just in case”
is bloat the moment this session doesn’t need it. The real problem is retrieval: store the
signal-bearing moments and query the right one at the right time. That’s why a class of “agent memory”
products exists. This page compares them and explains why we built rather than adopted.
The four deciding axes
- Store everything vs selectively — whole-session / fire-hose capture is useful but accretes noise and unbounded data over time.
- Separate LLM/embedding key — your coding agent already has a model + subscription; most memory tools make you configure and pay for a second provider just for memory.
- Infrastructure — an always-on server (Postgres, Redis, a Rust/Bun daemon, a vector DB) is operational weight, and often cloud-bound.
- First-class coding-signal capture — corrections, test outcomes, git events, skill upgrades captured structurally, or only as prose an LLM might (or might not) extract?
The eight systems
We score each system on the four facets shown as bars on every card; the weighted total also folds in two more criteria (selective capture / noise-control and license / portability).
- Signals — captures coding signals (corrections, tests, git, skills) as typed events, or hopes an LLM extracts them from prose?
- No key — runs on the agent’s own model, or needs a second LLM/embedding key + subscription?
- Local — runs from local files, or needs an always-on server / cloud?
- Retrieval — hybrid + rerank + graph + temporal quality.
Hindsight
MIT · vectorize-io
LLM-extracted facts + mental models. Best-in-class multi-strategy retrieval (94.6% LongMemEval), ~48 integrations. retain needs an LLM; reuse-your-sub loopback is ToS personal-use-only.
weighted3.03 / 5
claude-mem
Apache-2.0 · thedotmack
Compresses every tool call into observations. Reuses Claude auth + local embeddings (no extra key — ties reflect here). But unbounded growth, no native prune, ChromaDB fragility, probabilistic extraction.
weighted3.08 / 5
ByteRover
Elastic-2.0 (not OSS)
Curate-before-store markdown tree, local-first, importance decay. But curation makes its own LLM calls and is agent-directed, not passive — nothing captured unless the agent calls curate.
weighted3.06 / 5
Mem0
Apache-2.0 + SaaS
LLM-extracted facts; OpenAI by default at ingest (separate key). Full stack is Postgres + Neo4j; graph memory gated to the $249/mo Pro tier. Documented dedup gap.
weighted2.50 / 5
agentmemory
Apache-2.0 · rohitg00
Captures every tool call verbatim, then consolidates via opt-in LLM. Always-on Rust daemon (4 ports, pinned binary). O(N²) forget ceiling ~1k; documented token-burn incidents.
weighted2.28 / 5
Honcho
AGPL-3.0 · Plastic Labs
Theory-of-mind models of users/peers — built for end-user personalization, not coding signals. Self-host needs 1–3 provider keys + Postgres + Redis + a deriver worker; cloud is per-query.
weighted1.99 / 5
OpenViking
AGPL-3.0 · ByteDance
Tiered (L0/L1/L2) LLM-extracted memories/skills with conflict-aware dedup. Strong on noise — but OpenAI/Volcengine by default (separate key) and a Rust+Go+C++ always-on server.
weighted2.56 / 5
¹ Hindsight’s retain needs an LLM; its “reuse your Claude subscription” loopback is documented as
personal-use-only per Anthropic’s terms — not shippable as a default.
The pattern: the only other system that matches reflect on no extra key (claude-mem) collapses on selective/noise; the systems that curate well (Hindsight, OpenViking, ByteRover) lose on no extra key or local-first; and first-class signal capture is a wall of “no.”
Scored matrix
Weighted to a coding-agent self-improvement workflow (capture corrections → behaviour, cheaply, no infra). Darker green = stronger fit, rust = weak; bottom row is the weighted total out of 5. Re-weight toward pure retrieval quality and Hindsight leads — see the sensitivity note below.
| Criterion (weight) | reflect | Hindsight | claude-mem | ByteRover | Mem0 | agentmem | Honcho | OpenViking |
|---|---|---|---|---|---|---|---|---|
| Signal capture (0.25) | 5 | 2 | 2 | 2 | 2 | 2 | 1 | 2 |
| No extra key (0.22) | 5 | 2 | 5 | 3 | 2 | 3 | 2 | 2 |
| Local-first (0.20) | 5 | 3 | 3 | 4 | 2 | 2 | 2 | 2 |
| Selective / noise (0.16) | 4 | 4 | 2 | 4 | 3 | 1 | 2 | 4 |
| Retrieval quality (0.12) | 4 | 5 | 3 | 3 | 4 | 3 | 4 | 4 |
| License / portability (0.05) | 5 | 5 | 4 | 2 | 4 | 4 | 2 | 2 |
| Weighted total | 4.72 | 3.03 | 3.08 | 3.06 | 2.50 | 2.28 | 1.99 | 2.56 |
How reflect works
Benchmark — LOCOMO
reflect 4.1.0 on LOCOMO (long-term conversational memory). Preliminary: a category-stratified pilot graded by an Opus reference LLM-judge. Retrieval runs reflect’s real engine; the dialogue→note extraction is a documented LOCOMO-domain adapter.
| config · Opus judge | single-hop | multi-hop | temporal | open-domain | adversarial | overall |
|---|---|---|---|---|---|---|
| reflect 4.1.0 + retrieval fixes | 0.80 | 0.80 | 0.80 | 0.70 | 0.90 | 0.80 |
The retrieval fixes are two additive, env-gated, zero-new-API-key knobs: a stronger local embedder
(REFLECT_EMBED_MODEL=BAAI/bge-base-en-v1.5) and HyDE query-expansion (REFLECT_RECALL_HYDE=1,
reusing reflect’s own claude -p). Both default off — shipped behaviour is unchanged.
reflect lands mid-field — on par with Memobase / Zep, above Mem0 — while the newest systems (ByteRover,
Honcho, Hindsight) sit higher but are self-reported on their own harnesses. Judges and harnesses differ
across the field, so treat this as directional placement, not a strict ranking. Full methodology:
tests/eval/locomo/REPORT.md.
Token economics — both sides of the ledger
Memory looks cheap if you only price retrieval. The real cost is the write side (LLM extraction/consolidation) plus the read side (context injection).
| System | Write side | Read side | Net extra spend |
|---|---|---|---|
| reflect | reuses harness LLM (claude -p), gated + queued; local embed | local — vector + BM25 + graph, no LLM | $0 extra (rides your existing sub) |
| claude-mem | Haiku on your Claude auth, per tool-call batch | FTS5 + local ONNX vectors | $0 extra, but high write volume |
| Hindsight | retain = LLM extraction; cloud $15/M tokens | vector/graph; synthesis costs | separate key or $15/M retain |
| Mem0 | OpenAI extraction at ingest | vector/BM25; graph = Pro $249/mo | separate OpenAI key + tier |
| OpenViking | VLM extraction + L0/L1 summaries | vector recursive (no LLM) | separate provider key |
| Honcho | deriver LLM ("dreaming") per batch | context() free; dialectic up to $0.50/query | 1–3 keys (self-host) or per-query |
| agentmemory | optional LLM compression (off by default) | local embed; triple-stream RRF | $0 if LLM off — value degrades |
| ByteRover | curation = own LLM calls | BM25 → LLM fallback | own key tokens or Pro $19/mo |
Dangerous default to watch
Auto-retain every turn + large auto-recall is how memory tools quietly burn a subscription — agentmemory has documented incidents of exhausting a Claude Pro quota in a handful of messages. reflect’s capture is gated and queued (not every turn), and recall is OOD-gated + token-budgeted — it injects nothing when nothing fits.
Observability & correction
When the memory is wrong, can you see it, edit it, delete it, and trace where it came from?
- reflect — learnings are plain markdown you open, edit or delete in your editor; volatile signals live in a DB sidecar (clean git diffs); each learning links back to its source transcript + chunk; per-row TTL + contradiction handling expire stale beliefs.
- Most others — facts live in Postgres + vector DBs, inspected via API or SQL, not your editor. claude-mem ships a web viewer but observations are DB rows; ByteRover is the exception (an editable markdown tree). Correcting a wrong memory usually means an API call or a re-curation pass.
Self-improvement — how a correction becomes behaviour
This is the wedge. In every other system a correction is, at best, a sentence in a transcript an
extraction LLM might turn into a fact — Hindsight literally files capturing a skill update as a
bug. reflect routes it as typed signals at hook time (SG1–SG8: contradiction, git
commit/revert, test pass/fail, tool-loop, permission reply, idle sweep, negative-recall gaps) and
auto-refreshes the skills those learnings affect (R13/R14).
Everyone else remembers what was said. reflect captures what changed — and turns it into the next session’s behaviour.
Recommendation & decision rule
Decision rule
Adopt an external memory provider only if it (a) needs no second API key or subscription, (b) runs without a mandatory always-on server, and (c) captures coding signals as typed events. No single external tool clears all three. So: build the thin, differentiated layer (capture + typed signals + local-first), and port — don’t re-invent forever — the commodity retrieval, keeping it behind a seam so a stronger backend can be swapped in later.
Verdict: build + port, with eyes open. reflect ported Hindsight’s retrieval ideas (graph-expansion arm, RRF fusion, cross-encoder rerank, temporal arm) across a 57-port effort — so it has the retrieval brains without the infra/LLM/ToS baggage, at the cost of owning that code.
Honest sensitivity note
These weights favour self-improvement and zero-friction operation — reflect’s strengths. Re-weight toward pure retrieval quality and ecosystem maturity and Hindsight wins: MIT, production-grade, 94.6% LongMemEval, ~48 integrations, 62 releases. If your priority is “best recall, least code I maintain,” adopting Hindsight (as a retrieval backend) is the smarter call. The build case rests on valuing no-extra-key capture and typed signals — which is why retrieval is kept swappable.
If you’d adopt instead — four pilot acceptance tests
- Zero extra credentials — install on a fresh machine with only the coding agent’s existing auth. Does capture + recall work with no new key or account? (reflect / claude-mem: pass · most: fail)
- No always-on server — reboot. Does memory work with nothing running in the background? (reflect: pass · daemon-based tools: fail)
- Correction → behaviour — correct the agent once; next session, does the rule resurface without manual curation? (the typed-signal test — reflect’s wedge)
- Noise over 30 days — run daily for a month. Is recall still sharp, or flooded with stale / duplicate / fire-hosed entries? (fire-hose tools degrade here)