Monitoring AI Agents Locally: Langfuse + OpenTelemetry, No Cloud Account Required

Your agent works fine in a demo. You ship it. Then something goes wrong in production, and you have nothing to go on — no record of which tools it called, what it got back, or where in the chain things actually broke.

Langfuse, an open-source LLM monitoring tool, fixes that — and you can run the whole thing locally with Docker, no cloud account required to get started.

The OpenTelemetry basics you actually need#

Langfuse sits on top of OpenTelemetry, an open standard for instrumenting distributed systems — a set of APIs, SDKs, and tools for collecting metrics, logs, and traces, built to be production-suitable across languages. Three terms matter here:

• Span — one unit of work.
• Trace — the whole request lifecycle, start to finish. For an agent, that's from the moment you send a prompt to the final answer.
• Span processor — collects the spans and lets you inspect or modify them before they're shipped out.

One tradeoff worth knowing before you copy any example code: there are two span processors, simple and batch. Simple processes spans one at a time — great for local debugging, where you want to see each one immediately. Under real production load, you want batch instead, which groups spans together instead of shipping them individually. Use simple in dev, switch to batch before you ship.

The exporter takes the collected spans, wraps them into the standard OpenTelemetry data format, and sends them to your provider — Langfuse, in this case.

In the video#

• 00:35 — What OpenTelemetry actually is: spans, traces, span processors
• 04:33 — Setting up Langfuse locally with Docker and Docker Compose
• 07:27 — Creating an account, an organization, a project, and getting your API keys
• 10:23 — The instrumentation code: importing the instrumentor, wiring up the tracer provider
• 13:00 — Running the agent and watching the first trace land in the dashboard

Setting it up locally#

You need Docker and Docker Compose installed. Clone the Langfuse repo — that's where the docker-compose.yml lives — and run:

docker compose up

That spins up every service Langfuse needs: ClickHouse, Redis, the Langfuse web app, and the worker. Once the containers are up, head to localhost:3000, sign up, and walk through the setup wizard: create an organization, then a project. At the end, you get back a secret key (sk-...) and a public key (pk-...) — the two things you actually need.

Wiring it into your agent#

Base64-encode your secret and public key together, and set two environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:3000/api/public/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <base64-encoded-keys>"

(Swap the endpoint for https://cloud.langfuse.com/... or the US-region equivalent if you're using the managed version instead of self-hosting.)

Then, in code — the example here uses smolagents, but the pattern holds for any framework with OpenTelemetry support:

from smolagents import SmolagentsInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
 
trace_provider = TracerProvider()
trace_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter()))
 
SmolagentsInstrumentor().instrument(tracer_provider=trace_provider)

That's it. No manual logging statements scattered through your agent code — a good framework integration wraps the relevant methods automatically the moment you call instrument().

Watching the first trace#

Run the agent, then check the Langfuse dashboard under Traces. Click into the run, and you see the entire lifecycle: every tool call, every intermediate step, in order, from the prompt going in to the final answer coming back out.

The actual point#

This isn't just a debugging tool for when things are already broken. Set it up before you ship, and you get the same visibility in production that you had while building locally — which tool got called, what it returned, and exactly where a run went sideways, instead of a support ticket and a guess.