Zro

Private inference for coding agents

API

Developer Tools

Tech

Vercel Day

Visit WebsiteSee on Product HuntTwitter ⧉

Private inference is one of the most underrated problems in the agent space right now. I build in a compliance-heavy vertical, so "your code never leaves your control" is a real buying trigger, not a nice-to-have.

How are you handling the latency tradeoff vs. hosted frontier models?

Nice to see the focus on long-context workloads instead of just benchmark numbers. That's usually where real-world applications start showing their bottlenecks.

Streaming support for token-by-token responses would be huge for chat use cases, even if it means slightly more work on the caching layer. Right now waiting for full completions before anything renders feels dated for an inference API in 2026.

private + coding agents, finally. how much latency do you give up to keep it private?

Congrats on the launch! Keeping it OpenAI-compatible is a massive smart move. A lot of teams want to switch to open-weight models for privacy but dread rewriting their entire agent orchestration layer or dealing with completely different API schemas. How has the drop-in replacement experience been for early testers? Are there any specific edge cases with function calling or streaming where the compatibility layer behaves differently than native OpenAI?

Private inference is the box teams check last, usually right after their proprietary code turns up somewhere it should not. The part I would pressure-test is latency ten tool calls deep in a real agent loop, not on a single prompt. That is where most self-hosted setups fall over.

I am using GLM 5.2 through ZRO in my daily workflow for a while now. The speed and quality have been impressive so far.

Good luck with the launch!

the zero-retention, no-training-on-your-code angle is the whole pitch for me. most devs just assume their prompts end up as training data somewhere and live with it. if you can actually prove retention is zero, that's a real reason to switch, not just a nice-to-have

honestly the no-retention thing is a really thoughtful call, most teams talk about speed but skip the privacy piece entirely. nice to see it baked in from day one

I've been using Zro for about a month, it's great!
Very fast and reliable

Really like the commitment to zero request retention, that's a rare stance these days and shows the team actually thought through the privacy side instead of just slapping it on a landing page.

"Your code shouldn't be someone else's training data" is a strong line. Which open models are you running under the hood, and is EU hosting the default or a paid tier? Congrats on the launch

The API speed of response is really fast. Been used for a while and very satisfied by the quality of the API.

Love that there's no request retention here, that's a real differentiator for anyone dealing with sensitive data. One thing that would help me evaluate it faster though: a small public latency dashboard comparing Zro to other inference providers on common open models. Even just a simple weekly update with p50 and p95 numbers across regions would make it way easier to decide if it's worth migrating workloads over.

The tradeoff between closed APIs and self-hosting open models is very real. most small teams want more privacy and control, but not the operational burden of running inference infrastructure themselves.

The OpenAI-compatible API, zero retention, and optional on-prem path make Zro feel practical for teams that want to start simple and keep more control as they grow. Curious how you handle routing across regions for long-context coding workloads without hurting cache efficiency or latency consistency?

Latency felt noticeably snappier than what I usually get from other inference providers, and I love that nothing gets stored. Refreshing to see a team actually prioritize both speed and privacy at the same time.

"on prem this early feels like a lot for a small team to support properly, are yall going setup manually right now or is there tooling for it".

"multi region's great for latency but some of us need single region only for compliance stuff, is that even an option here".

"openai keeps cutting prices every few months, not sure privacy alone convinces people to switch providers".