Sub-Millisecond Note Retrieval: Why SQLite FTS5 Outperforms Vector Scanning for Local Vaults
For personal knowledge bases under 50,000 notes, SQLite FTS5 in WAL mode delivers sub-0.20ms search latencies under 25MB RAM, outperforming heavy vector embeddings and vector databases that consume 1–4GB of memory and add 50–200ms embedding model latency per query.
The Hidden Cost of Vector-First RAG on Personal Hardware
In recent years, many developer tools default to dense vector embeddings for any knowledge retrieval task. While vector databases excel at fuzzy conceptual matching across millions of unstructured documents, deploying them on personal computers creates severe bottlenecks:
- Memory Bloat: Running local embedding models (e.g., all-MiniLM, BGE) alongside a vector database requires 1GB to 4GB of dedicated RAM.
- Embedding Latency: Every user query must pass through a neural network forward pass before vector search can begin, introducing 50–200ms latency.
- Exact Keyword Failure: Dense embeddings frequently fail on exact identifiers, function names, error codes, and checklist tags (e.g. searching for
PRAGMA busy_timeoutor#todo/critical).
The Sub-Millisecond Relational Solution
Sphene uses an embedded SQLite database configured with native FTS5 full-text indexing, BM25 relevance ranking, and WAL concurrency. Across 10,000 markdown notes, queries execute in <0.20 milliseconds (<180µs benchmarked).
| Architectural Metric | Dense Vector Pipeline | Sphene SQLite FTS5 Engine |
|---|---|---|
| Query Latency | 60 ms – 250 ms (Model forward pass + ANN scan) | < 0.20 ms (Direct inverted index) |
| RAM Footprint | 1,200 MB – 3,800 MB (Weights + vector store) | < 25 MB RAM (Pure Go compiled kernel) |
| Exact Match Precision | Approximate (High cosine drift on codes) | 100% Deterministic (FTS5 phrase & prefix matching) |
| Context Token Preservation | Requires full chunk or raw document | Up to 95% reduction (AST-level section extraction) |
Intra-Document AST Extraction (sphene_find_in_doc)
Rather than streaming an entire 3,000-word design note into an LLM context window, agents invoke sphene_find_in_doc:
// Intra-document search returns matching lines and heading context
{
"name": "sphene_find_in_doc",
"arguments": {
"path": "Workspace/Neural Memory and Vector Indices.md",
"query": "cosine similarity"
}
}The kernel traverses the document AST and extracts only the relevant heading and context block in <0.05ms, saving thousands of tokens per query turn.
Experience Sub-Millisecond Search
Deploy Sphene on your workstation, mini PC, or NAS in 60 seconds.
Get Sphene Free (v2.2)