TOPIC GUIDE
LLM observability and evaluation
Langfuse, Braintrust, Arize Phoenix, Promptfoo, W&B Weave and Helicone for tracing, evaluating and testing AI applications.
- products
- 40
- things to compare
- 3
- updated
- 2026-10-30
Once an AI feature is in production, you need to see what it did and know whether a change made it better. Observability tools trace each model and tool call; evaluation tools score outputs against test cases; security tools probe for prompt injection and data leaks.
What to compare
- Instrument one real workflow and check that traces show every model and tool step.
- Turn real failures into test cases and run them before each release.
- Compare self-hosting options and licenses if your data cannot leave your infrastructure.
This is a focused selection, not an exhaustive ranking. See each profile for evidence and review scope.
Projects to explore
Langfuse
An open-source platform for tracing, evaluating and improving LLM apps and agents, with prompt management; part of ClickHouse since 2026.
Read the project profileBraintrust
An observability and evaluation platform for AI agents that traces production, runs evals and catches regressions.
Read the project profileArize Phoenix
Arize's source-available platform for tracing, evaluating and debugging LLM applications, which you can run yourself.
Read the project profilePromptfoo
An open-source tool and platform for red teaming, security testing and evaluating LLM applications; being acquired by OpenAI.
Read the project profileW&B Weave
Weights & Biases' toolkit for tracing, evaluating and comparing LLM applications; Weights & Biases is part of CoreWeave.
Read the project profileHelicone
An AI gateway and LLM observability platform for routing and monitoring AI applications.
Read the project profilePortkey
A production stack for generative AI applications across an organization, including a gateway for model access.
Read the project profileLangChain
A Python framework for composing model calls, tools, and agent behavior through a configurable agent harness.
Read the project profileLaminar
An open-source platform to trace, evaluate and debug AI agents, highlighting where an agent failed.
Read the project profileOpik
Comet's open-source platform for tracing, evaluating and testing LLM applications and agents.
Read the project profileDeepEval
An open-source framework for testing LLM applications, with 50+ metrics for agents, RAG and chatbots that run like unit tests.
Read the project profileAgenta
An open-source workspace for building agents through chat, improving them with feedback and evaluations, and sharing them with your team.
Read the project profileOpenLIT
An open-source, OpenTelemetry-native platform for agent observability, evaluations, guardrails, prompt management and cost and GPU monitoring.
Read the project profileRequesty
An AI gateway that routes model requests through one API with caching, provider fallbacks, spend controls and usage analytics.
Read the project profileRespan
Respan combines a model gateway with fallbacks, caching and spend limits, plus tracing and evaluations for AI applications and agents.
Read the project profileGiskard
A testing platform for finding security vulnerabilities and evaluating answer quality in LLM agents and retrieval applications.
Read the project profileLunary
An LLM application platform for tracing model and tool calls, reviewing conversations and managing prompt versions.
Read the project profilePromptLayer
An AI application platform for versioning prompts, running dataset-based evaluations and connecting production traces to prompt and workflow changes.
Read the project profileKiln AI
An AI development workbench combining evaluations, prompt optimization, synthetic datasets, retrieval, agents and fine-tuning with a desktop app and Python library.
Read the project profileAgentOps
An agent observability platform with a Python SDK for tracing model calls, tools and agent interactions, replaying runs and tracking token costs.
Read the project profileTruLens
An open-source Python toolkit for tracing AI agents and evaluating retrieval, responses and tool use with custom metrics and model-based judges.
Read the project profileBifrost
An AI gateway that connects model providers through one API with fallbacks, load balancing, semantic caching and usage controls.
Read the project profileLLM Gateway
An LLM API gateway for routing requests across providers with caching, failover and usage analytics.
Read the project profileInspect AI
An evaluation framework that combines datasets, agents, tools and scorers to assess language models and inspect their execution logs.
Read the project profileEvidently
An open-source Python framework for evaluating LLM outputs, testing AI systems and monitoring data quality and model drift.
Read the project profilegarak
A command-line LLM vulnerability scanner that probes models and chatbots for prompt injection, data leakage, jailbreaks and other unwanted behavior.
Read the project profilePyRIT
A Python framework for automated and human-led AI red teaming with multi-turn attacks, configurable targets, scoring and conversation tracking.
Read the project profileMindgard
An AI security platform for discovering attack surfaces, running adversarial tests against models and agents, and reviewing defensive gaps and remediation guidance.
Read the project profileLangWatch
An AI engineering platform for tracing model and tool calls, testing agents with simulated conversations, evaluating outputs and managing prompts.
Read the project profileLabel Studio
An open-source platform for labeling multimodal datasets and collecting human evaluations of AI outputs.
Read the project profileLatitude
An agent observability platform that traces production sessions, groups recurring failures and sends coding agents the context to investigate fixes.
Read the project profileLangSmith
An observability and evaluation platform for tracing agent execution, inspecting failures and testing AI application outputs.
Read the project profilePydantic Logfire
An observability platform that traces model calls, agent tools, API requests and database queries together and evaluates AI outputs alongside those traces.
Read the project profileLanguage Model Evaluation Harness
A framework for evaluating language models on benchmark tasks with configurable prompts, metrics and local or API-based model backends.
Read the project profileMLflow
An AI engineering platform for tracing and evaluating LLM applications, versioning prompts, managing model access and tracking machine-learning experiments.
Read the project profileOpenLLMetry
Traceloop's open-source set of OpenTelemetry extensions that trace LLM calls, vector databases and agent frameworks and send the data to Traceloop or an existing observability backend.
Read the project profileHoneyHive
An observability and evaluation platform for AI agents that records traces, scores live traffic and runs experiments, built on OpenTelemetry.
Read the project profileClearML
An open-source MLOps and AI infrastructure platform that combines experiment tracking, dataset versioning, pipelines and model serving with GPU cluster scheduling, quotas and fractional GPUs across cloud and on-premises hardware.
Read the project profileCoval
Simulation, evaluation and monitoring platform for voice and chat AI agents, with human review queues, API, CLI and MCP server.
Read the project profileCekura
Automated QA platform for voice and chat AI agents that simulates conversations, red-teams, benchmarks providers and monitors production calls.
Read the project profile