Build industrial-grade LLM applications: one API for every LLM, observability, optimization (prompts, models, etc.), evaluations, and A/B testing — all open source. Turn metrics and human feedback into smarter, faster, and cheaper LLMs. Get started in minutes.
Over time, these components enable you to set up a principled feedback loop for your LLM application. The data you collect is tied to your KPIs, ports across model providers, and compounds into a competitive advantage for your business.
Here are some recent blog posts we wrote that illustrate some of the benefits:
We hope TensorZero will be useful to many of you Hunters!
How is TensorZero different from other tools?
1. TensorZero enables you to optimize complex LLM applications based on production metrics and human feedback.
2. TensorZero supports the needs of industrial-grade LLM applications: low latency (thanks to Rust 🦀), high throughput, type safety, self-hosted, GitOps, customizability, etc.
3. TensorZero unifies the entire LLMOps stack, creating compounding benefits. For example, LLM evaluations can be used for fine-tuning models alongside AI judges.
And it's all open source!
How much does TensorZero cost?
Nothing. TensorZero is 100% self-hosted and open-source (Apache 2.0). There are no paid features.
("But really, how do you plan to make money?" PH sneak peek: next year, we're planning to launch an optional, complementary paid service focused on automated LLM optimization, abstracting away all the GPUs needed to handle that. The developer tool we're working on today will remain open source.)
How can I help?
We'd love to get your feedback: features you like, features that are missing, anything confusing in the docs, etc.
TensorZero is 100% open source, so feedback from the builder community helps us prioritize the roadmap, improve the developer experience, fill any gaps in the docs, and so on.
Thank you! Please let us know if you have any questions or feedback.
We use T0 in production at Vest and it's changed the way that we interact with LLMs: now, instead of having the SDKs of multiple providers tightly coupled to our backend -- or having to maintain our own abstractions -- we just use the T0 gateway. The gateway is just the beginning, however, and what really excites us is how organizing data from our inference calls on T0 unlocks downstream parameter tuning, prompt optimization, and research into the performance of our LLM applications. This is a fantastic project, and I can't wait to see where these guys go!
Congrats on launch! Can I swap model providers per request and get latency/cost dashboards out of the box?
The feedback loop is what will build long term compounding and defensible advantage for AI applications - and separate winners from losers. Congrats team, very exciting.
Really impressive open-source stack — the unified Gateway plus observability and A/B testing feels very production-ready. Curious: how easy is it to plug TensorZero into an existing CI/CD/GitOps pipeline, and do you provide examples for Kubernetes Helm or Argo workflows?
This looks really promising! Having a unified API for different LLM providers with observability, optimization, and A/B testing built-in is super valuable. I especially like the open-source approach — makes it much easier to adopt incrementally. Curious: do you also plan to support fine-tuning workflows or mainly focus on inference optimization?
Congrats on the launch! It's cool how easy TensorZero makes fine tuning - I think a lot of people skip fine tuning today because setting up data collection/curation/evaluation is such a headache.
Unifying all LLM providers into one super-fast API is just genius, tbh—no more hacky integrations or crazy latency. Open-source too? This is wild, team!
Hi Product Hunt - we're the team behind TensorZero, an open-source LLM infrastructure project.
What is TensorZero?
TensorZero is an open-source stack for industrial-grade LLM applications:
Gateway: access every LLM provider through a unified API, built for performance (<1ms p99 latency)
Observability: store inferences and feedback in your database, available programmatically or in the UI
Optimization: collect metrics and human feedback to optimize prompts, models, and inference strategies
Evaluation: benchmark individual inferences or end-to-end workflows using heuristics, LLM judges, etc.
Experimentation: ship with confidence with built-in A/B testing, routing, fallbacks, retries, etc.
Take what you need, adopt incrementally, and complement with other tools.
Why should you use a tool like this?
Over time, these components enable you to set up a principled feedback loop for your LLM application. The data you collect is tied to your KPIs, ports across model providers, and compounds into a competitive advantage for your business.
Here are some recent blog posts we wrote that illustrate some of the benefits:
Reverse Engineering Cursor's LLM Client
Fine-tuned Small LLMs Can Beat Large Ones at 5-30x Lower Cost with Programmatic Data Curation
We hope TensorZero will be useful to many of you Hunters!
How is TensorZero different from other tools?
1. TensorZero enables you to optimize complex LLM applications based on production metrics and human feedback.
2. TensorZero supports the needs of industrial-grade LLM applications: low latency (thanks to Rust 🦀), high throughput, type safety, self-hosted, GitOps, customizability, etc.
3. TensorZero unifies the entire LLMOps stack, creating compounding benefits. For example, LLM evaluations can be used for fine-tuning models alongside AI judges.
And it's all open source!
How much does TensorZero cost?
Nothing. TensorZero is 100% self-hosted and open-source (Apache 2.0). There are no paid features.
("But really, how do you plan to make money?" PH sneak peek: next year, we're planning to launch an optional, complementary paid service focused on automated LLM optimization, abstracting away all the GPUs needed to handle that. The developer tool we're working on today will remain open source.)
How can I help?
We'd love to get your feedback: features you like, features that are missing, anything confusing in the docs, etc.
TensorZero is 100% open source, so feedback from the builder community helps us prioritize the roadmap, improve the developer experience, fill any gaps in the docs, and so on.
Thank you! Please let us know if you have any questions or feedback.