Per-section relevant context retrieval (RAG)

xenarathon commented

2026-08-02 16:35:16 -04:00

Owner

Summary

Replaces the whole-context-blob prompt injection with real retrieval: each section's generation call now gets only the context chunks relevant to that section, selected by embedding similarity (local-first mxbai-embed-large, cloud fallback e.g. Gemini), instead of the full concatenated context-files blob prepended to every request.
New internal/retrieval package: heading-based chunking, an OpenAI-compatible embeddings client, and a Retriever with a fallback chain, cosine-similarity ranking, and per-slot embedding cache.
New [embeddings] config table — entirely opt-in. Unconfigured (the default), behavior is byte-identical to before this branch.
Design spec: docs/superpowers/specs/2026-08-02-rag-context-retrieval-design.md (reviewed via a 4-model salyut consult before finalizing). Implementation plan: docs/superpowers/plans/2026-08-02-rag-context-retrieval.md.
Built via subagent-driven-development: 6 planned tasks, each independently implemented + reviewed (one fix round on Task 2 — a chunk-heading-orphaning bug caught by review). Final whole-branch review found 1 Critical + 3 Important cross-task issues (a zero-context silent-failure path, a panic on malformed embeddings responses, ContextPaths/ContextPrompt divergence, missing diagnostics) — fixed in one pass, re-reviewed clean. That re-review then caught a genuine TUI-corrupting side effect in the diagnostics logging itself (stderr write during the Bubble Tea alt-screen render) — fixed and re-reviewed clean.

Test plan

go build ./... && go vet ./... clean
go test ./... — 319/319 passing
go test -race ./internal/retrieval/... ./internal/tui/... — no races
make license-check clean
Verified live: the configured local embedding endpoint (http://192.168.1.231:11435/v1/embeddings, mxbai-embed-large on a GTX 750 Ti) is real and reachable
Manual smoke test: run redakt new against a post with 2+ context files with [embeddings] uncommented in redakt.toml, confirm candidates generate and retrieved-context framing looks sane

🤖 Generated with Claude Code

## Summary - Replaces the whole-context-blob prompt injection with real retrieval: each section's generation call now gets only the context chunks relevant to that section, selected by embedding similarity (local-first `mxbai-embed-large`, cloud fallback e.g. Gemini), instead of the full concatenated context-files blob prepended to every request. - New `internal/retrieval` package: heading-based chunking, an OpenAI-compatible embeddings client, and a `Retriever` with a fallback chain, cosine-similarity ranking, and per-slot embedding cache. - New `[embeddings]` config table — entirely opt-in. Unconfigured (the default), behavior is byte-identical to before this branch. - Design spec: `docs/superpowers/specs/2026-08-02-rag-context-retrieval-design.md` (reviewed via a 4-model salyut consult before finalizing). Implementation plan: `docs/superpowers/plans/2026-08-02-rag-context-retrieval.md`. - Built via subagent-driven-development: 6 planned tasks, each independently implemented + reviewed (one fix round on Task 2 — a chunk-heading-orphaning bug caught by review). Final whole-branch review found 1 Critical + 3 Important cross-task issues (a zero-context silent-failure path, a panic on malformed embeddings responses, `ContextPaths`/`ContextPrompt` divergence, missing diagnostics) — fixed in one pass, re-reviewed clean. That re-review then caught a genuine TUI-corrupting side effect in the diagnostics logging itself (stderr write during the Bubble Tea alt-screen render) — fixed and re-reviewed clean. ## Test plan - [x] `go build ./... && go vet ./...` clean - [x] `go test ./...` — 319/319 passing - [x] `go test -race ./internal/retrieval/... ./internal/tui/...` — no races - [x] `make license-check` clean - [x] Verified live: the configured local embedding endpoint (`http://192.168.1.231:11435/v1/embeddings`, `mxbai-embed-large` on a GTX 750 Ti) is real and reachable - [ ] Manual smoke test: run `redakt new` against a post with 2+ context files with `[embeddings]` uncommented in `redakt.toml`, confirm candidates generate and retrieved-context framing looks sane 🤖 Generated with [Claude Code](https://claude.com/claude-code)

xenarathon added 13 commits

2026-08-02 16:35:16 -04:00

redakt: candidate-nav scroll fix, partial-failure visibility, wait-for-context 757e94561e

- Review ↑/↓ now scrolls the viewport to the newly selected candidate's
  heading instead of resetting to page-top — long candidates previously
  made a selection change invisible off-screen.
- skippedProviders/lastGenErrors now surface even when some candidates
  succeeded, not just on total generation failure — a dropped provider
  (e.g. an unreachable local model) no longer disappears with no trace.
- Init()/loadSourceFromPicker() no longer auto-fire the first generation
  before context has been decided: they wait for an explicit `r` (or `c`
  then `r`) unless --context already supplied context, so adding context
  mid-flow no longer races a wasted generation call.

redakt: system prompt forbids mimicking injected context's structure 1a8b8ec69e

Global-context files (e.g. a project handoff doc with its own TL;DR/
headings/conclusion) were getting echoed wholesale instead of used as
reference — a model would reproduce the injected file's own document
shape rather than staying scoped to the one outline section being
rewritten. The system prompt now explicitly marks Global context as
reference-only and forbids inventing a title/TL;DR/extra headings/
conclusion beyond what the source section already has.

redakt: spec + plan for per-section RAG context retrieval f83bef70f0

Design doc reviewed via a 4-model salyut consult before finalizing;
implementation plan self-reviewed for spec coverage and signature
consistency across its 6 tasks. Ready for subagent-driven-development
execution.

redakt: gitignore .worktrees/ for isolated feature-branch workspaces 73a2baed2b

config: add [embeddings] table for per-section context retrieval 22c744d113

- Add EmbeddingsConfig struct with primary/fallback embedding service
  configuration including base URL, model, and env token key
- Add TopK, ChunkMaxTokens, and TimeoutSeconds configuration options
  with sensible defaults (5, 400, 30)
- Add Embeddings.Enabled() method to check if retrieval is configured
- Implement expand() for environment variable expansion in embeddings
  configuration fields
- Implement validation to set numeric field defaults when zero
- Implement secret registration for embeddings tokens to prevent
  credential leakage in logs
- Add IsRegisteredSecret() export to internal/log for testing

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

retrieval: chunk context files by heading for per-section retrieval 91e60ff879

retrieval: keep section heading in every chunk of an oversized paragraph f6d4ddda50

splitByBudget previously flushed the heading-only accumulator as its own
near-empty chunk once an oversized first paragraph forced a flush, leaving
the paragraph's truncated sub-chunks with no heading text at all. Split the
heading out of the section body up front in a new splitSection helper and
prepend it to every chunk splitByBudget returns for that section, so
embedding/retrieval never sees a body-only chunk stripped of its section
context.

retrieval: add OpenAI-compatible embeddings client with timeout 61214a0f0c

retrieval: add Retriever with fallback chain, ranking, and caching 5a2fc1b68e

tui: use per-section retrieval in generateCmd when configured bad9881252

tui: wire retriever construction from [embeddings] config c21e07a7b2

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

retrieval: whole-branch review fix pass — 4 findings 9727620952

Post-hoc fixes surfaced by reviewing the completed per-section RAG
context retrieval feature as a whole, after all 6 planned tasks were
individually approved.

1. (Critical) Relevant's two early-degrade paths (empty sectionPrompt,
   every context file unreadable) returned zero chunks instead of
   wrapping contextPromptFallback like the both-clients-failed path
   already did — generateCmd only prepends context when len(chunks)>0,
   so the model silently got NO context while the UI still said "using
   full context". Both paths now route through a shared degradedChunks
   helper. Corrected the plan/spec docs, which encoded the bug verbatim.

2. embedAll indexed into a client's returned vector slice without
   checking its length against the input batch — a non-conformant or
   partially-responding embeddings endpoint (e.g. the cloud fallback)
   panicked the whole TUI session. Added a length check that errors
   instead, and fixed client.go's Embed to size its output from
   len(texts) so a partial response can't silently misalign indices.

3. Post.ContextPaths (what retrieval reads) could drift from
   Post.ContextPrompt (what the assembled blob has): `redakt resume
   --context`, post.ResumeFromFile, and the in-TUI picker's directory
   pick either never touched ContextPaths or recorded the raw directory
   path instead of its expanded files. Every call site now routes the
   same resolved file list into both.

4. Provider errors inside Relevant were silently absorbed with zero
   diagnostics. Added structured debug/warn logging via internal/log
   (never leaking the raw error to the user-facing message).

Adds/strengthens tests across internal/retrieval, internal/tui, and
internal/post for all four findings.

retrieval: re-review fixes — Debug-level degrade log, reject nil embedding entries

CI / build / test / lint (pull_request) Failing after 5m45s

Details

98487f3be5

Finding A: the both-attempts-failed degrade log was at Warn, which
internal/log's default os.Stderr sink actually writes (unlike the Debug
line above it) — a raw stderr write during Bubble Tea's alt-screen
render visibly corrupts the TUI. Demoted to Debug; still fully
diagnosable with debug logging enabled.

Finding B: embedAll only checked vector-slice length, not individual
nil entries. A right-sized-but-nil-containing Embed response (client.go
pads missing entries with nil to preserve index alignment) passed that
check and got cached under r.cache, permanently poisoning that chunk
for the life of the Retriever — every later lookup silently reused the
cached nil and scored -1 via cosineSimilarity's guard, with no error.
embedAll now treats any nil entry the same as a length mismatch: the
whole batch fails and flows through the normal fallback/degrade path
instead of being cached.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

xenarathon merged commit ea2fdf1583 into main

2026-08-03 16:12:26 -04:00

xenarathon referenced this pull request from a commit

2026-08-03 16:12:26 -04:00

Merge pull request 'Per-section relevant context retrieval (RAG)' (#4) from rag-context-retrieval into main

Rows
Columns

Per-section relevant context retrieval (RAG) #4

Summary

Test plan