Most AI benchmarks are built backwards. Someone sits down, dreams up hard problems, and then measures how well agents solve them. The results are interesting, sure. But they don't always tell you what matters: how agents perform on the actual work that's sitting in your queue. That's why we built cto.bench. Instead of hypothetical tasks, we're building our benchmark from real work. Every data point on cto bench comes directly from how cto.new users are actually using our platform.
I'm excited to share cto bench is live. This is a benchmarking tool that tests against real world usage of the latest and greatest frontier models by cto.new users. Many benchmarking tools run LLMs through custom suites to test viability, but cto bench uses actual usage patterns and PR merge rates to verify how well models are performing on actual tasks. We hope this ads valuable, practical data points to the LLM benchmarking space as it evolves.
About cto bench on Product Hunt
“The ground truth code agent benchmark”
cto bench launched on Product Hunt on December 20th, 2025 and earned 131 upvotes and 10 comments, placing #6 on the daily leaderboard. Most AI benchmarks are built backwards. Someone sits down, dreams up hard problems, and then measures how well agents solve them. The results are interesting, sure. But they don't always tell you what matters: how agents perform on the actual work that's sitting in your queue. That's why we built cto.bench. Instead of hypothetical tasks, we're building our benchmark from real work. Every data point on cto bench comes directly from how cto.new users are actually using our platform.
On the analytics side, cto bench competes within Analytics, Developer Tools and Artificial Intelligence — topics that collectively have 1.2M followers on Product Hunt. The dashboard above tracks how cto bench performed against the three products that launched closest to it on the same day.
Who hunted cto bench?
cto bench was hunted by Michael Ludden. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
Michael Ludden