Monitor LLMs and automatically detect hallucinations in prod
Athina helps developers monitor and evaluate LLMs in production. Get complete visibility into RAG pipeline and 40+ preset eval metrics to detect hallucinations and measure performance
Hi PH – I’m Shiv, a co-founder of Athina AI!
We started on this journey about a year ago when we realized first-hand how difficult it is to take LLMs into production.
One of the biggest challenges we faced was dealing with hallucinations, and finding effective ways to measure performance of different models, prompts and retrieval strategies. 😅
After speaking with dozens of builders, we found this to be a universal problem, and we set out to build the product we wished we had.
With Athina, developers can easily monitor their LLM application in production, measure the model performance with our suite of 40+ evaluation metrics, and catch regressions in CI / CD.
After many months of hard work, we’re now processing millions of logs every week from hundreds of users, and we’re excited to finally launch Athina publicly with the PH community! 🚀
Athina takes just a few minutes to set up, and here’s what you get:
🪵 Full visibility into production logs along with usage metadata like cost, token usage, etc. Athina also includes GraphQL access.
📊 Library of 40+ evaluation metrics including retrieval score, answer relevancy, faithfulness, conversation coherence, pii detection, and many more.
📐 Support for custom evaluation metrics: Easily plug in your own evaluation prompt or function.
⏱️ Compare performance across models, prompts, and topics so you can get insights like “gpt-4 has a 4.8% hallucination rate while our custom fine-tuned llama model has a 7.2% hallucination rate”
🛝 Built-In Prompt Playground so you can quickly experiment with different prompt and model combinations
👬 Built for collaboration: Athina supports multiple users in an organization.
🕸️ Enterprise Grade options like on-premise deployment, custom log retention, and more
Thank you all for your support!
----
Website: https://athina.ai
Sign Up: https://app.athina.ai
Demo Video: https://bit.ly/athina-demo-feb-2024
Schedule calls with founders: https://cal.com/shiv-athina/30min
