Distill

Every token earns its place.

Developer Tools

Artificial Intelligence

GitHub

Visit WebsiteSee on Product HuntGithub ⧉

I kept seeing token-compression tools get popular by making the model "talk different" — telegraphic, caveman-style output — to shave tokens. That's a real trick, but it scared me a little: a compression style has no independent check on what it's cutting. At high settings, it's easy to compress away exactly the sentence that mattered — a "this can't be undone" or a dropped assumption — along with the filler. I wanted a tool where safety-relevant content literally can't be compressed away, enforced by code, not by hoping the model remembers an instruction. Two, really. First, every compression tool I looked at only touched output — what the model writes — while ignoring that tool descriptions and tool results usually dominate an agent's context far more than its own prose does. Second, and honestly the bigger one: nobody was reporting net numbers. Every tool quotes gross output reduction and never subtracts what the tool itself costs to run — its own instruction overhead, its own compression calls. A number that only ever looks good isn't a number I trust, so I didn't want to ship one. I started assuming the skill (the output-side compression) was the product, and the middleware was a nice-to-have. Real measurement flipped that: benchmarking showed the skill is close to a no-op inside a modern agent harness that's already terse, while the middleware — compressing MCP tool descriptions and results before they ever reach the model — measured up to 87% on repetitive logs. That reordered the whole story to middleware-first. The rest of the process was mostly finding out I was wrong about smaller things and fixing them once measurement caught them, not before: a single-sample benchmark run that flipped sign on rerun until I forced medians over n=3; a benchmark harness that ran from the project's own repo, so the model could read its own eval fixtures and recognize it was being tested instead of answering naturally; a subtle one-way bias in the skill's own self-check instructions that made it expand output on complex turns instead of shrinking it, invisible until a clean re-measurement exposed it. None of those were things I'd have caught by reasoning about the code — only by actually running it and trusting the number over my assumption. That became the real product principle: measured beats assumed, always, even when — especially when — the honest number is the less flattering one.