index

Obsidian RAG Pipeline — Technical Reference

This is the technical companion to the blog post. If you’re here, you want implementation details.

The pipeline gives Claude Code semantic search over my entire Obsidian vault — ~238 markdown files, 742 chunks, all indexed locally with no cloud dependencies. Supports hybrid search (vector + keyword fusion), wikilink graph traversal, and neighbor chunk expansion.

Architecture

┌─────────────────────────────────────────────────────────┐
│                     Obsidian Vault                       │
│   /home/nox/docker/obsidian/vaults/weeslahw_coppermind/ │
│                    ~238 markdown files                   │
└────────────────────────┬────────────────────────────────┘
                         │ bind-mount (:ro)
                         ▼
┌─────────────────────────────────────────────────────────┐
│              Indexer (Docker, profiled)                   │
│                                                          │
│  1. Walk vault, skip .obsidian/.git/media                │
│  2. SHA256 each file → compare against DB                │
│  3. Chunk changed files by heading (6000 char max)       │
│  4. Embed chunks via Ollama (nomic-embed-text, 768-dim)  │
│  5. Upsert into pgvector (DELETE old + INSERT new)       │
│  6. Populate tsvector for full-text search               │
│  7. Extract [[wikilinks]] → note_links table             │
│  8. Resolve wikilink targets to indexed file paths       │
│                                                          │
│  Runs every 30 min via systemd timer                     │
└──────────┬──────────────────────────┬───────────────────┘
           │                          │
           ▼                          ▼
┌──────────────────────┐  ┌──────────────────────────────┐
│   Ollama (cyrion)    │  │   PostgreSQL + pgvector       │
│                      │  │   (phlegethon LXC 302)        │
│  nomic-embed-text    │  │                                │
│  768-dim vectors     │  │  DB: obsidian_rag              │
│  Port 11434          │  │  Tables:                       │
│  ~274MB model        │  │    documents (HNSW + GIN idx)  │
└──────────────────────┘  │    note_links (wikilink graph) │
                          │  742 chunks / 236 files        │
                          │  136 wikilinks / 62 resolved   │
                          └──────────────┬─────────────────┘
                                         │
                                         ▼
                          ┌──────────────────────────────┐
                          │   MCP Server (workstation)    │
                          │                                │
                          │  search_notes(query, limit,    │
                          │    mode=hybrid|vector|keyword)  │
                          │  graph_search(file_path, depth) │
                          │  list_topics()                  │
                          │  get_note(file_path)            │
                          │                                │
                          │  Hybrid: RRF fusion of vector   │
                          │  + keyword, neighbor expansion  │
                          └────────────────────────────────┘

Pages

• Indexer — The chunking, embedding, and wikilink extraction pipeline (indexer.py)
• MCP Server — The Python server that bridges Claude Code to the database (server.py)
• Database Schema — pgvector table structure, note_links, indexes, and query patterns

Key Numbers