Local AI File Search: How It Works
Embeddings, vector indexes, and retrieval — all on your device.
The limits of keyword search
Traditional desktop search tools index filenames, paths, and sometimes content — then match your query as a substring or keyword. This works when you remember the exact word in a document, but fails for anything else. Search for "invoice plumber" and you will miss the file titled "Joe's Plumbing — receipt March.pdf" if it does not contain the word "invoice". The fundamental problem is that keyword search operates at the lexical level: it matches characters, not meaning.
What embeddings are
An embedding is a fixed-length numeric vector — typically hundreds or thousands of numbers — that represents the meaning of a piece of text. Embedding models are neural networks trained to place semantically similar content close together in this vector space. The sentences "the cat sat on the mat" and "a feline rested on the rug" produce vectors that are very close to each other. Unrelated content produces vectors that are far apart. Once you have embeddings for both your files and your query, retrieval becomes a nearest-neighbor search: find the files whose vectors are closest to the query vector.
Building a local vector index
When a local file organizer like embly processes a file, it first extracts readable text from it — OCR for images, Whisper transcription for audio, direct parsing for PDFs and documents. The text is then chunked into overlapping segments (typically a few hundred tokens each) and each chunk is passed through an embedding model running locally. The resulting vectors are stored in a vector database on disk. For retrieval, the query is embedded with the same model and the nearest neighbors in the index are returned. The entire pipeline — extraction, embedding, indexing, and search — runs on your machine. Nothing leaves your device.
Why running models locally is now practical
Running embedding models locally was impractical for most hardware until recently. Two things changed. First, smaller, distilled embedding models have become competitive with much larger ones on retrieval benchmarks — models like nomic-embed-text and the MiniLM family produce high-quality vectors at a fraction of the compute cost of their predecessors. Second, modern laptops — particularly those with Apple Silicon — have enough unified memory and neural engine capacity to run these models at interactive speeds. Embedding a document takes milliseconds; searching a 100,000-file index takes under a second.
Approximate nearest-neighbor search
Comparing a query vector against every vector in an index (exact nearest-neighbor search) is O(n) in time — fine for small indexes, slow for large ones. For personal file collections, which can easily reach hundreds of thousands of files, approximate nearest-neighbor (ANN) algorithms like HNSW (Hierarchical Navigable Small Worlds) build a graph structure that allows sub-linear retrieval. HNSW can search millions of vectors in milliseconds with recall rates above 95% compared to the exact result. Libraries like hnswlib and usearch make this accessible in any language with minimal overhead.
What this means in practice
When you search for "the invoice from the plumber last spring" in a tool like embly, the query is embedded locally and the closest vectors in the index are returned — regardless of whether the word "invoice" or "plumber" appears in the file. The match is on meaning: if the file is a receipt from a plumbing company dated March, its embedding will be semantically close to your query. The result is retrieval that works the way human memory works — by association and meaning, not by exact string matching. And because everything runs locally, the response time is bounded by your hardware rather than a network connection, and your queries are never sent to a third party.