Large Language Models (LLMs) power modern AI applications, but their unpredictable behavior and complex workflows make it difficult to diagnose issues, optimize performance, and understand how they process data. Without visibility into each step of an LLM chain, troubleshooting and improving efficiency can be challenging.

Datadog’s LLM Observability provides visibility into operational performance, helping you ensure the quality, safety, and security of your LLM applications. End-to-end tracing captures input and output, latency metrics, token usage, and errors. By tracing each step in the LLM chain—including embedding, retrieval, and generation—teams can identify the root causes of unexpected outputs, latency, and errors, helping them troubleshoot performance issues and control costs. 

In this hands-on workshop, you’ll build a chatbot application with a Retrieval-Augmented Generation (RAG) workflow using the OpenAI Python SDK to make calls to local models. You’ll instrument the application for Datadog’s LLM Observability, using auto-instrumentation and manual in-code setup to collect traces. Then, you’ll analyze these traces to connect application behavior with steps in the LLM chain, identify areas for improvement, apply changes, and observe results.

By the end of this workshop, you’ll have practical experience using Datadog’s observability tools to understand LLM application behavior and improve performance.