Cekura Bench publishes voice AI benchmarks you can verify. Our new speech-to-speech benchmark tests 9 realtime voice models, including GPT Realtime 2.1, Gemini Live, Grok and Phonic, as complete phone agents on live calls: 82 scenarios, three runs each. Models are ranked on reliability, data accuracy, stalled calls, response time and cost, and every call transcript is public. Cekura Bench also covers voice agent benchmarks and STT benchmarks, with TTS benchmarks coming soon.
Hi everyone, I am Sidhant from Cekura.
Today we are launching our speech-to-speech benchmark. We ran 9 realtime voice models as the full agent inside the same open-source Pipecat phone agent, with the same prompt, tools and phone line. Each model handled 82 scenarios across a clinic booking flow and a Medicare intake, three times each. A call passes only if the agent reaches the right outcome and saves every field correctly.
Results:
Most reliable: GPT Realtime 2.1, 79.3% of scenarios passed on all three runs
Fastest: Phonic v1, 1.60s median response
Lowest cost: GPT Realtime 2.1 Mini, $0.020 per minute
A cascade baseline (Flux, GPT-4.1, ElevenLabs Flash) scored 82.9%, above every realtime model
Every call links to its transcript, tool calls and scores, and the agent and test cases are on GitHub.
We would value your feedback, especially on which models we should add next.
Did the cascade's lead hold on the Medicare intake flow specifically, where it's mostly reading back dates and ID numbers? Curious whether the realtime models lost more on data capture or on the tool calls.
Really comprehensive benchmark, Voice ai companies should definitely check this out 🚀🚀
Voice AI is moving fast, and having an independent way to compare different AI providers on real phone calls is incredibly useful. Really cool work, team!
I like that they’re publishing the actual call transcripts with the scores. Way easier to see what went wrong instead of just taking the numbers at face value.
Choosing a speech-to-speech model today can be difficult, with new models launching constantly and limited ways to compare them fairly. These results provides transparent, verifiable results to help teams evaluate real-world performance and choose with confidence. 🎯
Publishing the call transcripts alongside the scores is the part I like here. Being able to look at what actually happened on a failed call makes the ranking much easier to judge. Congrats to the team!
Really exciting to see this live! The fact that these models were tested on real phone calls with the same agent, prompts, tools, and scenarios makes the benchmark especially interesting. Looking forward to seeing how the results evolve as more models are added 🚀
Finally, a serious benchmark for speech-to-speech models. No cherry-picked demos. Same agent, same prompt, same tools, and real end-to-end scenarios.
Excited to see what the community does with it. 🚀
Every speech-to-speech model sounds great in a demo, so we put them all through the same real phone calls with real tool use and scored what actually happened. Excited to see how the rankings shift as new models drop 🚀
Really interesting results 👀 The cascade baseline beating all the realtime models was definitely not what I expected.
Excited to see what the community thinks - and what models we should put through this next.
About Cekura Bench on Product Hunt
“Speech-to-speech model benchmarks on live phone calls”
Cekura Bench launched on Product Hunt on October 8th, 2026 and earned 169 upvotes and 18 comments, placing #4 on the daily leaderboard. Cekura Bench publishes voice AI benchmarks you can verify. Our new speech-to-speech benchmark tests 9 realtime voice models, including GPT Realtime 2.1, Gemini Live, Grok and Phonic, as complete phone agents on live calls: 82 scenarios, three runs each. Models are ranked on reliability, data accuracy, stalled calls, response time and cost, and every call transcript is public. Cekura Bench also covers voice agent benchmarks and STT benchmarks, with TTS benchmarks coming soon.
Cekura Bench was featured in SaaS (44.8k followers), Developer Tools (521.1k followers) and Artificial Intelligence (481k followers) on Product Hunt. Together, these topics include over 275.8k products, making this a competitive space to launch in.
Who hunted Cekura Bench?
Cekura Bench was hunted by Garry Tan. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
Want to see how Cekura Bench stacked up against nearby launches in real time? Check out the live launch dashboard for upvote speed charts, proximity comparisons, and more analytics.
Garry Tan