This product was not featured by Product Hunt yet. It will not be visible on their landing page and won't be ranked (cannot win product of the day regardless of upvotes).
Product upvotes vs the next 3
Waiting for data. Loading
Product comments vs the next 3
Waiting for data. Loading
Product upvote speed vs the next 3
Waiting for data. Loading
Product upvotes and comments
Waiting for data. Loading
Product vs the next 3
Loading
Weir
Semantic caching and model routing for RAG
Weir is an open-source gateway for RAG that combines semantic caching, cost-aware model routing, and per-request cost and latency accounting. Includes a runnable demo, evaluation suite, and load tests.
Hey Product Hunt! 👋
I built Weir to explore a question: can we make RAG applications cheaper without sacrificing answer quality?
Weir sits in front of an existing RAG service and combines semantic caching, model routing, and per-request cost accounting.
One of the biggest challenges was making semantic caching safer. Questions like "ICU visiting hours" and "general ward visiting hours" can be semantically similar while requiring different answers, so Weir uses an entity guard to reject potentially unsafe cache matches.
In a synthetic, repeat-heavy 300-request replay, full Weir reduced measured cost per 1,000 requests by 94%. A separate 158-question cold pass showed a 20% saving. These are benchmark results, not production guarantees, and the repo documents the limitations and failure modes.
I'd love feedback on the design, evaluation methodology, and what you'd test next.
GitHub: https://github.com/yatinannam/weir
About Weir on Product Hunt
“Semantic caching and model routing for RAG”
Weir was submitted on Product Hunt and earned 2 upvotes and 1 comments, placing #140 on the daily leaderboard. Weir is an open-source gateway for RAG that combines semantic caching, cost-aware model routing, and per-request cost and latency accounting. Includes a runnable demo, evaluation suite, and load tests.
On the analytics side, Weir competes within Open Source, Developer Tools, Artificial Intelligence and GitHub — topics that collectively have 1.1M followers on Product Hunt. The dashboard above tracks how Weir performed against the three products that launched closest to it on the same day.
Who hunted Weir?
Weir was hunted by Yatin Annam. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
For a complete overview of Weir including community comment highlights and product details, visit the product overview.
Yatin Annam