A hands-on guide to LLM observability using self-hosted Langfuse, covering manual tracing with the low-level API versus decorator-based tracing, attaching custom evaluation scores and quality metrics to traces, and running vLLM health checks before executing any pipeline. The tutorial walks through building explicit trace hierarchies with parent traces and child generation spans, computing lightweight quality scores based on answer length and latency thresholds, and logging structured metadata to the Langfuse dashboard. A dedicated health-check script validates vLLM server availability, model readiness, and generation capability before any tracing begins.
Table of contents
Manual Tracing, Scores, and Evaluation with Langfuse (Self-Hosted)Why Manual Tracing Matters for LLM ObservabilityDecorator vs Manual Tracing: When to Use WhichManual Tracing with the Langfuse Low-Level APILLM Evaluation Metrics and Quality Scoring with LangfusevLLM Diagnostics and Health Checks for LLM ObservabilitySummarySort: