Respan Gateway

One AI gateway with built-in observability and evals

Developer Tools

Artificial Intelligence

Tech

Visit WebsiteSee on Product HuntTwitter ⧉

The evals layer baked into the gateway is particularly interesting since most teams still just eyeball logs to check model performance.

the DevOps platform framing alongside 2 lines of code is an interesting positioning tension. DevOps platforms usually require significant setup and ongoing configuration to be useful. 2 lines of code implies you get value immediately. curious which one is more accurate for a new user and at what point the simple integration becomes a platform with enough configuration to actually catch the problems that matter in production

how does Keywords AI decide which model to route a query to when there are multiple options available?

The fallback + spend-limit combo is the part I'd test first. In real LLM apps the annoying bit isn't routing, it's knowing whether a fallback quietly changed latency/cost. Curious if alerts can be tied to a specific customer or workspace?

Really impressed with how Keywords AI makes managing multiple models and routing so seamless. Congrats!

The part I'd want to stress-test is how traces map back to customer and deployment context; that is usually where gateway-only setups stop being enough for debugging production incidents.

Good stuff however I do not think routing is the easy part. It's only easy if it's not done properly. Routing needs to figure out best model. Best model needs to define criteria for 'best'. If it's best output + speed + price, then routing needs to detect intent behind what's flowing through it and adjust accordingly.

How does Keywords AI handle niche or low-volume keywords differently than other tools?

Huge fan of the routing and spend-limiting features so far.
It really bridges the gap between a standard API router and a full-scale LLMops production platform.
Having traces baked in makes managing live traffic so much cleaner.

Interesting take with Respan: Self-driving AI observability and evals for agents. What made you decide to build this now?

Sounds useful. We have a travel AI, and we want to run tests comparing the quality of our model’s responses against other popular models. Do you have any built-in mechanisms for that?

Congrats on the launch! Genuine question from someone running multi-provider LLM calls in production: when a provider degrades mid-request (slow but not erroring), does the gateway support latency-based failover, or only hard-error fallback? And can the cost observability enforce per-provider daily caps, or is it reporting-only? The eval layer baked into the gateway is the part I haven't seen elsewhere — curious how you keep eval prompts from polluting the usage metrics.

Putting evals at the gateway layer instead of bolting them on downstream is a smart place to catch regressions before they reach prod. Does Respan run evals against live traffic samples, or is it more of a pre-deploy gate?

I don't work in AI infra but even from the outside, the "something broke and you don't know why" problem makes total sense. having one place to see what's happening instead of piecing it together sounds like it saves a lot of pain. congrats on the launch.

Having caching and fallbacks baked into one endpoint is a massive win for customer-facing AI features like conversational marketing bots. How does the gateway handle latency during failovers? Is the switch seamless enough that the end-user won't notice a lag?

Connecting to models is rarely the hard part anymore. Figuring out why smth failed three days later is usually where the pain starts. Interesting to see more tools focusing on that side

This is what many dev teams are missing. I’ve seen so many projects stall because they couldn’t effectively trace which model version caused a latency spike.

How does Respan handle 'evals' for non-deterministic outputs? Is it easy to set up automated regression tests for prompt changes?