Open-source LLM tracing and evals, built on OpenTelemetry. Self-host for free or use the cloud. The anti-LangSmith choice.
Arize (the ML observability company) open-sourced Phoenix in late 2023 as the first true open-source challenger to LangSmith. The bet: tracing and evals shouldn't be proprietary. Phoenix runs on OpenTelemetry — the same standard as the rest of your observability stack — so it plugs into Datadog, Honeycomb, Grafana, and your existing infra. Self-host for free, or use Arize's managed cloud. By 2026, Phoenix is the default choice for teams that don't want LangSmith's vendor lock-in.
Who it's for: LLM teams with existing observability infra who want tracing without a new vendor. Also strong for regulated industries (healthcare, finance) where data residency rules out SaaS.
Built on OpenTelemetry, the CNCF standard. Export traces to Datadog, Honeycomb, Grafana Tempo, or anywhere OTel is supported. Plays nice with the rest of your stack.
Run Phoenix in your own VPC with a single Docker command. No data leaves your infrastructure. Critical for HIPAA, SOC 2, and EU data residency.
Run evals on traces using built-in evaluators (hallucination, relevance, toxicity) or write custom ones. LLM-as-judge via OpenAI/Anthropic or heuristic (exact match, regex).
Unique among LLM tracing tools: Phoenix analyzes embedding drift across trace clusters, surfacing when your retrieval quality degrades. Found nowhere else.
Phoenix is the right choice if you care about open source, data residency, or OpenTelemetry standardization. It's the wrong choice if you want the slickest UX and don't care about vendor lock-in — that's LangSmith's lane. For most enterprise teams shipping production LLM features, the decision comes down to: do you trust LangChain as a vendor? If yes, LangSmith. If no, Phoenix.