QuickCompare by Trismik

Compare LLMs on your data, measure, and pick the best.

Developer Tools

Artificial Intelligence

Data Science

Visit WebsiteSee on Product Hunt

ive been running gpt-4o-mini in production for sentiment scoring on my crypto site, and the question i actually care about is "would switching to claude haiku save me money without tanking quality on MY prompts". Does quickcompare measure cost per call and tail latency alongside output quality, or just output quality ?

QQ: who's the buyer?

As a solo founder I picked one model and stuck with it because the switching cost (prompt re-tuning, eval

rewriting) usually outweighs the marginal gain.

Comparing 50 LLMs feels like enterprise eval-engineer kit.

What's the indie use case I'm missing?

I have only today adopted Codex, partially into my stack, just because Claude is too slow for some things.

Hey Rebekka! It's awesome. I usually deal with understanding which LLM is the best option for a given case and it's taking a lot of time. I'm sure it's gonna change the game. Wish you all the best here!

Great launch! 🚀 Have to try it out for optimizing our LLM based flows at CatDoes.

Interesting! This can be really useful for our research team. Our pipelines are a mix of different model usually.

Great launch! Btw, can I compare models for specific tasks like marketing, coding, or support?

Hi Product Hunt! I'm Alice, on the Science team at Trismik. Big day for us today, and I've been looking forward to sharing what the whole team has built.

The shape of QuickCompare is a four-step flow: bring a dataset, configure how you want to evaluate (which metrics, which columns, optionally an LLM-as-Judge), pick the models you want to compare, then run them all in parallel against your data. What you get back is a side-by-side view of accuracy, inference cost, and average latency for each model, plus a breakdown of how each one performs on the easier vs harder slices of your dataset rather than just the headline average. The "your dataset" bit is the point: it's an LLM evaluation tool that scores models against your actual task, which we think is what makes it a more useful LLM Arena alternative for teams who have their own data and need a real answer rather than a popularity vote.

Ziggy is the AI assistant inside QuickCompare, and the part I've spent most of my time on. It exists because most LLM evaluation tooling assumes you already know your way around prompt templates, judge prompts, and which metric makes sense for which task. That's a pretty steep tax for someone who just wants to know which model is cheapest at acceptable quality on their data.

So Ziggy looks at your dataset, suggests sensible columns and metrics, writes the Jinja2 input template, and if you need an LLM-as-Judge setup it drafts the judge prompt with a scale that fits the task (binary for classification, 1 to 5 for open-ended generation). You can chat with it the whole way through and it knows where you are in QuickCompare, what you've already filled in, and what's still missing. Once the run finishes, it switches into analysis mode and helps you interpret the cost, latency, and accuracy numbers across the models you ran.

Has anyone had a model you assumed was the right call turn out not to be once you actually tested it? That gap is the bit I find most interesting and would love to hear stories.

Thanks so much for supporting us today, and do give QuickCompare (and Ziggy!) a try 🙂

Hi Product Hunt, I’m excited to share that our Cambridge spinout, Trismik, is launching QuickCompare today.

We built QuickCompare to help AI teams compare LLMs on their own tasks and data, so they can make better decisions before deployment, fine-tuning, or migration.

As both a Cambridge academic and a co-founder, I’ve seen how difficult model selection can be in practice. There’s no shortage of models, but there is still a real need for fast, practical evaluation on real world use cases.

QuickCompare is for teams asking:

• Which model performs best for our workflow?

• Which model is most reliable?

• Which model is worth deeper investment?

Thank you for taking a look, we’d really love your feedback!

Nigel, co-founder of Trismik

Hey Product Hunt, Rebekka here, co-founder at Trismik 👋

We built QuickCompare because we kept seeing the same pattern: teams shipping LLM features were making model decisions with surprisingly little evidence.

Often, they were defaulting to the biggest or most familiar models, relying on public benchmarks, or testing a couple manually and calling it a day. But in practice, that can mean spending far more than necessary on inference without actually getting the best result for your use case.

The reality is that model choice is rarely one-dimensional. It is not just about which model performs best. It is also:

• Which model performs best on your prompts and tasks?
• Where can you cut inference cost without sacrificing output quality?
• When do cheaper models actually match or outperform the expensive default?
• How do cost, speed, and task performance trade off side by side?

For many teams, especially those building AI products at scale, this has real business impact. Huge monthly inference bills, slow experimentation, and too much guesswork in a decision that directly affects margins, product experience, and speed to market.

That is why we built QuickCompare.

QuickCompare helps teams compare models on their own data, side by side, across quality, cost, and speed, so they can make a confident decision based on their actual use case, not generic benchmarks.

And we also built Ziggy, our AI Scientist assistant, to make this much easier. You don't need deep evals expertise to get started. Ziggy helps you set up and run comparisons in a much more intuitive, no-code way.

The goal is simple: help teams find the right model for the job, often cutting cost dramatically while maintaining or even improving performance and speed.

If you're building with LLMs, we'd really love your feedback. In particular, I would love to hear:

• how you are choosing models today
• whether inference cost is a major pain point for you
• what makes model evaluation feel slow, difficult, or inaccessible in practice

🔗 Try QuickCompare free: We’d love to hear what you think 🙏

🎁 Product Hunt bonus: Get an extra $10 in free QuickCompare credits

Thanks so much for checking us out and supporting the launch!