Langfuse Prompt Experiments

Open Source LLM Engineering Platform

Open Source

Developer Tools

Artificial Intelligence

GitHub

Hi Product Hunt 👋👋👋 I’m Clemens, co-founder of Langfuse. We are so excited to be live on Product Hunt again today, launching the biggest feature yet: Langfuse Prompt Experiments! This new feature is designed to close the loop of developing and improving LLM applications. With Prompt Experiments, you can test and evaluate different prompt versions and LLMs on hundreds of examples simultaneously; Perform live LLM-as-a-judge evaluations and compare results in our new Dataset Comparison View to optimize prompts for your specific use case. You can quickly make prompt changes, see whether they improve metrics without causing new issues, and then deploy to production with confidence. Move faster on your LLM Application development than ever before. A few things you should know about Langfuse: 👣 Tracing: TS & Python & API + OpenAI, Llama Index, LangChain, LiteLLM + many more ✏️ Prompt Management: Collaboratively manage and deploy prompts from within Langfuse ⚖️ Evaluation: Automatically run evals to control for hallucinations or other criteria you care about 💾 Datasets: Collaboratively manage fine-tuning, testing and golden datasets 🧪 Dataset Experiments: Test different prompts and models on hundreds of traces simultaneously 📊 Metrics: Dashboards and analytics on cost, latency and quality 🕹️ LLM Playground: Engineer your prompts and directly see results 🏎️ Export & Fine Tune: Open GET API and csv/JSON exports to build downstream use cases 🚄 Scale: We’ve invested significantly in scaling and resilience as we’ve scaled to thousands of users and handle many many millions of events a day 🧑‍🤝‍🧑Community: Join our thousands of users on GitHub Discussions and Discord See for yourself: ⭐ GitHub: https://github.com/langfuse/lang... ⏯️ Interactive Demo: https://langfuse.com/demo 📒 Docs: https://langfuse.com/docs 💬 Discord: https://langfuse.com/discord Thanks again, PH community. This is where we started, and we are incredibly grateful for all of the community feedback which led to the development of this major new feature. We’re excited and will be in the comments the whole day to hear your thoughts & feedback!