Skip to main content

AI Observability for Every TypeScript LLM Stack

· 5 min read
MLflow maintainers
MLflow maintainers
MLflow maintainers
Vercel AI SDK tracing demo in MLflow

TypeScript and JavaScript continue to dominate full-stack AI development, and MLflow 3.6 doubles down on that reality. In addition to the OpenAI SDK hooks we launched earlier this year, we now ship automatic tracing integrations for Vercel AI SDK, LangChain.js, LangGraph.js, Mastra, Anthropic, and Gemini.

In practical terms, any traces emitted from modern JS LLM stacks now land in the same MLflow experiment UI as your Python services, complete with prompt/response payloads, token usage, tool results, and error metadata.

Vercel AI SDK tracing demo in MLflow

Set up MLflow​

Getting MLflow running beside a JS stack no longer requires a Python environment:

  • Docker Compose bundle – For the easiest local or small-team deployment, MLflow's official Docker Compose setup is highly recommended. To get started, simply clone the GitHub repository, then navigate into the docker-compose directory provided in the repository. Run docker compose up -d to launch the entire MLflow Tracking stack: this automatically starts the MLflow Tracking server along with a pre-configured SQL backend (such as PostgreSQL), both pre-wired for seamless ingestion of OpenTelemetry Protocol (OTLP) traces.

  • Managed solutions – MLflow is available on many cloud providers, such as Databricks, AWS SageMaker Experiments, Azure ML, Nebius, etc. You can point MLFLOW_TRACKING_URI at Databricks’ free edition or another managed MLflow service, and start emitting traces without hosting anything yourself.

Vercel AI SDK / Next.js​

Vercel AI SDK tracing is built on top of the OpenTelemetry and designed for minimal instrumentation effort. Since MLflow tracking server is OpenTelemetry-compatible, you can directly send Vercel AI SDK traces to MLflow tracking server with minimal configuration.

The following code snippet shows how to set up tracing for LLM applications using Vercel AI SDK and Next.js.

.env.local
OTEL_EXPORTER_OTLP_ENDPOINT=<your-mlflow-tracking-server-endpoint>
OTEL_EXPORTER_OTLP_TRACES_HEADERS=x-mlflow-experiment-id=<your-experiment-id>
OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
instrumentation.ts
import { registerOTel } from "@vercel/otel";

export async function register() {
  registerOTel({ serviceName: "next-app" });
}
route.ts
import { openai } from "@ai-sdk/openai";
import { generateText } from "ai";

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const { text } = await generateText({
    model: openai("gpt-4o-mini"),
    maxOutputTokens: 100,
    prompt,
    experimental_telemetry: { isEnabled: true },
  });

  return new Response(JSON.stringify({ text }), {
    headers: { "Content-Type": "application/json" },
  });
}

See Vercel AI SDK Tracing for more details.

Mastra​

Mastra includes a built-in OpenTelemetry exporter, making it easy to send tracing data directly to the MLflow tracking server. To enable tracing, simply add an observability configuration block to your Mastra setup. By providing an OpenTelemetry exporter instance that points to your MLflow server, all your Mastra workflows—including the full hierarchy of chains and tools—are automatically traced and visualized in MLflow. This setup preserves the structure and relationships between your workflow components, so you can see detailed traces of end-to-end workflow execution right in the MLflow UI.

import { OtelExporter } from "@mastra/otel-exporter";

export const mastra = new Mastra({
  workflows: { weatherWorkflow },
  ...

  // Add the following observability configuration to enable OpenTelemetry tracing.
  observability: {
    configs: {
      otel: {
        serviceName: "mastra-app",
        exporters: [new OtelExporter({
          provider: {
            custom: {
              // Specify tracking server URI with the `/v1/traces` path.
              endpoint: "http://localhost:5000/v1/traces",
              // Set the MLflow experiment ID in the header.
              headers: { "x-mlflow-experiment-id": "<your-experiment-id>"},
              // MLflow supports HTTP/Protobuf protocol.
              protocol: "http/protobuf"
            }
          }
        })]
      }
    }
  },
});

Launch npm run dev, talk to the playground, and refresh the MLflow UI to see the spans.

See Mastra Tracing for more details.

LangChain.js / LangGraph.js​

LangChain.js instrumentation rides on OpenTelemetry. Configure a tracer once and MLflow ingests every span—chains, agents, retrievers, tool calls, and async flows—through the OTLP /v1/traces endpoint.

import {
  NodeTracerProvider,
  SimpleSpanProcessor,
} from "@opentelemetry/sdk-trace-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { LangChainInstrumentation } from "@arizeai/openinference-instrumentation-langchain";
import * as CallbackManagerModule from "@langchain/core/callbacks/manager";

const exporter = new OTLPTraceExporter({
  url: "http://localhost:5000/v1/traces",
  headers: { "x-mlflow-experiment-id": "123" },
});

const provider = new NodeTracerProvider();
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register();

new LangChainInstrumentation().manuallyInstrument(CallbackManagerModule);

See LangChain Tracing or LangGraph Tracing for more details.

OpenAI SDK​

mlflow-openai wraps the official OpenAI SDK so you can trace every call to OpenAI and capture messages, tool calls, token usage, and errors.

import { OpenAI } from "openai";
import { tracedOpenAI } from "mlflow-openai";
import * as mlflow from "mlflow-tracing";

mlflow.init({ trackingUri: "http://localhost:5000", experimentId: "openai" });

const client = tracedOpenAI(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }));

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "What's the weather like?" }],
});

See OpenAI Tracing for more details.

Anthropic SDK​

mlflow-anthropic wraps the official SDK so you can trace Claude calls (sync or async) in a single line.

import Anthropic from "@anthropic-ai/sdk";
import { tracedAnthropic } from "mlflow-anthropic";
import * as mlflow from "mlflow-tracing";

mlflow.init({
  trackingUri: "http://localhost:5000",
  experimentId: "anthropic",
});

const client = tracedAnthropic(new Anthropic());
const message = await client.messages.create({
  model: "claude-sonnet-4-5-20250929",
  max_tokens: 1024,
  messages: [
    { role: "user", content: "Summarize the new TypeScript release." },
  ],
});

See Anthropic Tracing for more details.

Gemini SDK​

Google's GenAI SDK is supported via mlflow-gemini, covering models.generateContent() plus function-calling metadata.

import { GoogleGenAI } from "@google/genai";
import { tracedGemini } from "mlflow-gemini";

const client = tracedGemini(
  new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY }),
);
const response = await client.models.generateContent({
  model: "gemini-2.5-flash",
  contents: "List three GenAI observability KPIs.",
});

See Gemini Tracing for more details.

Next Steps​

Tell us which JavaScript framework you want next by opening an issue!