ContextOS
The definitive technical specification for the ContextOS retrieval engine.
What is ContextOS?
ContextOS is a specialized semantic search and retrieval engine designed entirely around the constraints of Large Language Models. Unlike general-purpose vector databases which return complete documents based on coordinate proximity, ContextOS parses the source code into Abstract Syntax Trees, maps the dependency graph, and extracts only the relevant tokens required to solve a specific intent.
Design Philosophy
Most retrieval pipelines follow a naive approach: glob("**/*.ts") → Chunk → Embed → Cosine Search. This leads to massive cross-contamination in the LLM's context window because the embeddings capture syntactic similarities rather than logical dependencies.
ContextOS was built on three core tenets:
- Syntactic awareness over semantic guessing. If a function depends on an interface, the system shouldn't guess if they are related. It should read the AST and know for a fact.
- Aggressive token compression. Returning 10 files of 500 lines each instantly destroys context windows. ContextOS isolates only the active symbols (functions, types, classes) and discards the rest of the file.
- Project Isolation. Global vector databases contaminate searches across workspaces. ContextOS uses isolated, project-local SQLite databases (
.contextos) that live directly in the repository.
Mental Model
Think of ContextOS not as a database, but as a compiler.
When you run contextos init, it does not just dump text into an embedding model. It runs a multi-pass compilation step:
- It reads the raw files.
- It parses them into an AST (Abstract Syntax Tree).
- It extracts specific symbols (e.g., the
UserAuthclass). - It builds a graph of imports and dependencies (e.g.,
UserAuthdepends onSessionToken). - It compiles this graph into a localized SQLite file optimized for FTS5 (Full Text Search) and graph traversal.
When you query the engine, you are traversing this compiled graph to package the minimum possible token payload.