Hey product hunt! We’re excited to open-source Superpipe, a framework for building, evaluating, and optimizing LLM powered pipelines end-to-end.
Aman Dhesi and I built Superpipe after building LLM powered solutions for companies to solve unstructured —> structured data problems like product categorization, entity extraction from PDFs, and email classification. We initially developed Superpipe as an internal tool to bring robustness to our process for optimizing pipelines but realized it could be helpful for other teams as well.
Almost all AI pipelines consist of more than just a single call to and LLM and yet most tools only help you iterate on prompts or fine tune a model. Superpipe is built on the belief that multi-step systems need to be optimized end-to-end. Using Superpipe, you can fine-tune your entire pipeline (embeddings, retrieval, prompt, model), not just one step.
We’ve mostly been focused on unstructured —> structured data use cases but there’s no reason Superpipe couldn’t be used on other AI pipelines as well. We welcome feedback and open source contributions on our Github — https://github.com/villagecomput...
Ben Scharfstein