Benchmarks & Performance

Performance metrics, latency, and context compression benchmarks for ContextOS.

Implementation Sourcetests/benchmarks

Compression Benchmarks

ContextOS solves this by aggressively stripping out noise: omitting unchanged imports, stripping documentation strings, and extracting only the exact code blocks necessary to satisfy the query.

Standard 100-Query Benchmark

Average Tokens / Query
589

Average token usage per query across the 100-query benchmark.

Retrieval Latency
< 50ms

P99 Latency for retrieving from local SQLite database (including BM25 and Vector matching).

Exact Function Accuracy
98%

Successfully pinpointing the exact AST node (function/class) required to answer the query.

Conceptual Accuracy
96%

Successfully mapping the semantic intent of the query to correct codebase domains.

Throughput & Scalability

The backend Daemon handles indexing asynchronously. Rather than blocking your LLM queries, indexing happens passively in the background using SQLite WAL mode.

  • 1,000,000 files
    Hard limit on total files parsed per workspace to prevent Out-Of-Memory (OOM) errors and sandbox escapes.
  • ~2ms / file
    Average parsing time using the Tree-sitter AST parser, enabling rapid initial indexing.
  • pLimit(5)
    Concurrency cap on file watchers, preventing CPU starvation during massive branch swaps (e.g., git checkout).