# AI use cases for Langfuse (langfuse/langfuse) Repo: [langfuse/langfuse](https://github.com/langfuse/langfuse) | Use cases: 3 | Page: https://dedupedai.com/use-cases/langfuse-langfuse/ Langfuse is an open source LLM engineering platform for developing, monitoring, evaluating and debugging AI applications. It traces LLM calls, retrieval and agent steps, manages and versions prompts, runs evaluations with LLM-as-a-judge, code evaluators and user feedback, keeps datasets for testing, and offers a playground and a full API. It can be self-hosted with Docker Compose in minutes or used as a managed cloud. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Self-host LLM tracing in five minutes: https://dedupedai.com/use-cases/langfuse-langfuse/self-host-llm-tracing-in-five-minutes/ - Version and test prompts outside your code: https://dedupedai.com/use-cases/langfuse-langfuse/version-and-test-prompts-outside-your-code/ - Score your app's answers with evals and datasets: https://dedupedai.com/use-cases/langfuse-langfuse/score-llm-outputs-with-evals-and-datasets/ ## How to self-host LLM tracing in five minutes with Langfuse Repo: [langfuse/langfuse](https://github.com/langfuse/langfuse) | Difficulty: Beginner | Time: 30 minutes | Basis: From the README Run Langfuse locally with Docker Compose and start sending traces from your app. **Who it is for:** A developer whose agent gives odd answers and who cannot see which prompt, retrieval or tool call went wrong. **The problem:** Without tracing, debugging an LLM app means print statements and guesswork. **The outcome:** A local Langfuse instance showing each LLM call, retrieval step and agent action in your app. ### Steps 1. Clone the repo with `git clone --depth=1 https://github.com/langfuse/langfuse.git` and `cd langfuse` 2. Start it with `docker compose up` 3. Instrument your app with the Python or JS/TS SDK, or a drop-in integration such as OpenAI, Langchain or LlamaIndex 4. Open the Langfuse UI and inspect the traces for a request that went wrong 5. Configure Docker log rotation, since the default json-file driver does not rotate logs ### Prompt to paste into your agent ```text Self-host langfuse/langfuse locally with Docker Compose, then add tracing to my using the integration. Send a few test requests and show me the trace for the slowest one. ``` ### Caveat The local Docker Compose setup is for trying it out; the README recommends Kubernetes with Helm for production. Risk, disk: Docker's default json-file logging does not rotate, so a long-running instance can fill the disk unless you configure log rotation. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/langfuse-langfuse/self-host-llm-tracing-in-five-minutes/ ## How to version and test prompts outside your code with Langfuse Repo: [langfuse/langfuse](https://github.com/langfuse/langfuse) | Difficulty: Intermediate | Time: 45 minutes | Basis: From the README Move prompts into Langfuse Prompt Management and iterate on them in the LLM Playground. **Who it is for:** A product team where prompt changes need a code deploy and nobody knows which version is live. **The problem:** Prompts hard-coded in the app are slow to change and impossible to compare. **The outcome:** Versioned prompts fetched at runtime, with a playground for trying changes against real traces. ### Steps 1. Run Langfuse self-hosted or on Langfuse Cloud 2. Create your prompts in Prompt Management and fetch them from the SDK; caching on server and client keeps latency low 3. When a trace shows a bad result, open it in the LLM Playground and try a prompt or model change 4. Publish the better version and compare results over time ### Prompt to paste into your agent ```text Using langfuse/langfuse, move the prompts in into Prompt Management, update my code to fetch them through the SDK, and set up the playground so I can test a new version against . ``` ### Caveat The README links to the docs for SDK code; the exact fetch call depends on your language. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/langfuse-langfuse/version-and-test-prompts-outside-your-code/ ## How to score your app's answers with evals and datasets with Langfuse Repo: [langfuse/langfuse](https://github.com/langfuse/langfuse) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Build a dataset of test cases and score outputs with LLM-as-a-judge, code evaluators and user feedback. **Who it is for:** A team about to swap models who needs proof the new one is not worse. **The problem:** Quality is judged by gut feel, so regressions slip into production. **The outcome:** Datasets and evaluation scores you can compare across prompt and model versions before deploying. ### Steps 1. Run Langfuse and trace your app 2. Create a Dataset of representative inputs and expected outputs 3. Run experiments over the dataset for each prompt or model version 4. Score results with LLM-as-a-judge, code evaluators, manual labeling or user feedback collection 5. Use the API to wire these runs into your own LLMOps workflow ### Prompt to paste into your agent ```text Using langfuse/langfuse, create a dataset from , run it against and , and score both with an LLM-as-a-judge evaluator for . Summarise which version wins and on which cases. ``` ### Caveat A judge model has its own biases, so spot-check its scores by hand. Risk, cost: LLM-as-a-judge evaluators call a model for every scored item, which costs money with a paid provider. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/langfuse-langfuse/score-llm-outputs-with-evals-and-datasets/