Most agents pass their evals and fail in production. Prefactor is the evaluation layer that closes the gap. We score every agent run in real time, surface quality regressions and drift as they happen, and show engineering teams exactly how their agents are performing at scale. Built for the teams shipping agents to customers.
Hi Product Hunt - Matt here, co-founder of Prefactor with Simon.
We've been heads-down on this one for a while, so finally getting to show you is a real thrill.
Let me start with the question this whole thing is built around: do you actually know what your agents are doing in production right now?
For most teams we talk to, the honest answer is "…not really." Your evals pass, everything's green, you ship - and then every real run vanishes into a black box. Quality quietly drifts. Risk creeps in. Costs climb. And eventually someone asks which of your agents are still doing their job, and the room goes quiet.
That silence is the entire reason Prefactor exists. Gartner reckons 40%+ of agentic AI projects get scrapped by 2027 - and honestly, this gap is a big part of why.
So we built the thing we kept wishing we had: Prefactor scores every run in production the moment it happens - quality, drift, risk - then wires those scores straight into action. A failing agent gets caught live, not charted three days later.
How it actually works
prefactor init - one command connects your workspace and discovers your agents across your runtimes. First traced run in under 5 minutes.
Drop in the SDK (TypeScript or Python) - native for LangChain, Claude, Vercel AI, OpenClaw and LiveKit. Every call becomes a span, streaming in live with cost and data risk attached.
Run the evals you define on every run - LLM-as-judge, technical checks, qualitative metrics. Custom spans pull context from GitHub, Linear, Jira or your database, so every eval is grounded in what actually happened, not a guess.
Act. Hold, approve or block the second a run crosses a line - automatically at runtime, or routed to a human. Every decision logged and enforced through the SDK or API.
The payoff: you get to ship agents like real software — versioned, staged, promoted through dev → staging → prod only when evals pass, with instant rollback when they don't.
Why it's different
Most tools observe and score, then hand you the problem. Prefactor closes the loop - observe, evaluate, act, all inside the same run. A risky agent gets caught, not just charted.
My favourite bit of feedback so far: a customer with 40 agents in production and, in their words, "no honest way to say which ones were still doing their job." We gave them that answer - and the brake pedal for when one wasn't.
Who it's for
Engineering teams shipping agents to real customers, on any stack. Agent frameworks work natively; everything else plugs in through OpenTelemetry or the core SDK.
A little something for the PH community
Sign up today and you get 1,000,000 free agent steps. Every span counts, so that's a serious amount of live production evaluation on us. Valid until Friday 11:59pm PT, once you've set up your first agent.
Our ask
If you're running agents in production, tell us how you keep tabs on them today - even if the honest answer is "we're mostly hoping." We'll be in the comments all day and we'd genuinely love to hear what's working, what's breaking, and what you'd want a tool like this to do next.
Get started free at prefactor.tech. First 25,000 spans a month free, no card needed.
Prefactor launched on Product Hunt on July 28th, 2026 and earned 528 upvotes and 132 comments, earning #1 Product of the Day. Most agents pass their evals and fail in production. Prefactor is the evaluation layer that closes the gap. We score every agent run in real time, surface quality regressions and drift as they happen, and show engineering teams exactly how their agents are performing at scale. Built for the teams shipping agents to customers.
On the analytics side, Prefactor competes within SaaS, Developer Tools and Artificial Intelligence — topics that collectively have 1M followers on Product Hunt. The dashboard above tracks how Prefactor performed against the three products that launched closest to it on the same day.
Who hunted Prefactor?
Prefactor was hunted by Rohan Chaubey. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
For a complete overview of Prefactor including community comment highlights and product details, visit the product overview.
Rohan Chaubey
Hi Product Hunt - Matt here, co-founder of Prefactor with Simon.
We've been heads-down on this one for a while, so finally getting to show you is a real thrill.
Let me start with the question this whole thing is built around: do you actually know what your agents are doing in production right now?
For most teams we talk to, the honest answer is "…not really." Your evals pass, everything's green, you ship - and then every real run vanishes into a black box. Quality quietly drifts. Risk creeps in. Costs climb. And eventually someone asks which of your agents are still doing their job, and the room goes quiet.
That silence is the entire reason Prefactor exists. Gartner reckons 40%+ of agentic AI projects get scrapped by 2027 - and honestly, this gap is a big part of why.
So we built the thing we kept wishing we had: Prefactor scores every run in production the moment it happens - quality, drift, risk - then wires those scores straight into action. A failing agent gets caught live, not charted three days later.
How it actually works
prefactor init - one command connects your workspace and discovers your agents across your runtimes. First traced run in under 5 minutes.
Drop in the SDK (TypeScript or Python) - native for LangChain, Claude, Vercel AI, OpenClaw and LiveKit. Every call becomes a span, streaming in live with cost and data risk attached.
Run the evals you define on every run - LLM-as-judge, technical checks, qualitative metrics. Custom spans pull context from GitHub, Linear, Jira or your database, so every eval is grounded in what actually happened, not a guess.
Act. Hold, approve or block the second a run crosses a line - automatically at runtime, or routed to a human. Every decision logged and enforced through the SDK or API.
The payoff: you get to ship agents like real software — versioned, staged, promoted through dev → staging → prod only when evals pass, with instant rollback when they don't.
Why it's different
Most tools observe and score, then hand you the problem. Prefactor closes the loop - observe, evaluate, act, all inside the same run. A risky agent gets caught, not just charted.
My favourite bit of feedback so far: a customer with 40 agents in production and, in their words, "no honest way to say which ones were still doing their job." We gave them that answer - and the brake pedal for when one wasn't.
Who it's for
Engineering teams shipping agents to real customers, on any stack. Agent frameworks work natively; everything else plugs in through OpenTelemetry or the core SDK.
A little something for the PH community
Sign up today and you get 1,000,000 free agent steps. Every span counts, so that's a serious amount of live production evaluation on us. Valid until Friday 11:59pm PT, once you've set up your first agent.
Our ask
If you're running agents in production, tell us how you keep tabs on them today - even if the honest answer is "we're mostly hoping." We'll be in the comments all day and we'd genuinely love to hear what's working, what's breaking, and what you'd want a tool like this to do next.
Get started free at prefactor.tech. First 25,000 spans a month free, no card needed.
@simon_russell1 @joshgillies @ethan_lee8 @rheu @joeys and I will be here all day.
Big thanks to @rohanrecommends @rohanchaubey4 hunting us.