What is cache-hunter?

Cache Hunter is a small proxy tool for developers who are building on top of AI models and want to understand how well those models are reusing cached context. Many local AI serving systems, such as vLLM, support a feature called prefix caching, where a model can reuse work it already did for a piece of text it saw before, making later requests faster. The problem is that these systems usually do not tell you directly whether a given request was a cache hit or a cache miss, so Cache Hunter tries to work that out indirectly. It works by sitting between your AI tool, called a harness in the README, and the actual model server. You point your tool at Cache Hunter instead of the real server, and Cache Hunter quietly forwards every request through to the real one and copies down everything that happens along the way, including the full request and response bodies, timing information, and token counts, into a local SQLite database file. Once some traffic has been captured, Cache Hunter includes ready made scripts and example database queries to help make sense of it. These look at how much time each response took per token, since a request that shares a prefix with an earlier one but still takes a long time per token is a sign the cache was likely missed. Other scripts look for requests that share a common beginning, or track how a conversation's context grows over time and check that it stays internally consistent. The tool describes itself as completely transparent, meaning it passes every request through unchanged rather than modifying anything, and it supports the streaming response format used by chat style AI APIs. Logging happens in the background so it does not slow down the requests it is watching, and each request gets a tracking id in its response headers so a specific exchange can be found again later in the database. Cache Hunter is written in TypeScript, installed with npm, and started with a single command. It is meant as a debugging and analysis tool for developers, not an end user product, and its own documentation notes that its findings are only clues about caching behavior rather than a direct confirmation, since the underlying model servers do not expose that information themselves.