Context Compression

The Context Window Problem

If a user queries for the validate() method, standard vector DBs return the entire file. If that file is 3,000 lines long, it consumes 30k+ tokens instantly.

ContextOS compiles results at the AST node level, enforcing a hard maxTokenBudget (default: 1,200 tokens; configurable).

Token Budgeting Algorithm

ContextOS iterates over the ranked results and executes a greedy knapsack-style packing algorithm.

1. Group by File: We group top chunks by their source file. We do not inject chunks out of file order.
2. Calculate Framing Overhead: For each file, we calculate the tokens required for XML framing: <file path="...">...</file>.
3. Exact Token Measurement: We use the gpt-tokenizer package to physically count tokens on the fly.

Token Clamping & Stub Tiers

To prevent large repositories from generating overwhelming context payloads, ContextOS utilizes Token Clamping. The compiler restricts exact token matches via a fast heuristic tokenizer before falling back to exact byte-pair encoding.

Additionally, MCP client requests can specify a tier: "stub" parameter (introduced in v0.9.4). This heavily clamps the maximum token budget (e.g., capping at 250 tokens), optimizing for ultra-low latency when the LLM only needs brief structural overviews rather than deep implementation details.

Fallback Mechanisms

If the budget is exceeded mid-file, ContextOS triggers graceful degradation:

if (currentTokenCount + chunkTokens > budget) {
  // Option 1: Try stripping comments
  const stripped = stripComments(chunk.body);
  if (count(stripped) <= budget) {
    return add(stripped);
  }

  // Option 2: Try truncating the body entirely, leaving just the signature
  const signature = extractSignature(chunk.body);
  if (count(signature) <= budget) {
    return add(signature + '\n  // ... (body truncated)');
  }

  // Option 3: Drop the chunk entirely to preserve budget integrity
  return drop();
}