Submit projectSubmit

TOPIC GUIDE

LLM observability and evaluation

Langfuse, Braintrust, Arize Phoenix, Promptfoo, W&B Weave and Helicone for tracing, evaluating and testing AI applications.

products
40
things to compare
3
updated
2026-10-30

Once an AI feature is in production, you need to see what it did and know whether a change made it better. Observability tools trace each model and tool call; evaluation tools score outputs against test cases; security tools probe for prompt injection and data leaks.

What to compare

  1. Instrument one real workflow and check that traces show every model and tool step.
  2. Turn real failures into test cases and run them before each release.
  3. Compare self-hosting options and licenses if your data cannot leave your infrastructure.

This is a focused selection, not an exhaustive ranking. See each profile for evidence and review scope.

Projects to explore

Langfuse

An open-source platform for tracing, evaluating and improving LLM apps and agents, with prompt management; part of ClickHouse since 2026.

Read the project profile

Braintrust

An observability and evaluation platform for AI agents that traces production, runs evals and catches regressions.

Read the project profile

Arize Phoenix

Arize's source-available platform for tracing, evaluating and debugging LLM applications, which you can run yourself.

Read the project profile

Promptfoo

An open-source tool and platform for red teaming, security testing and evaluating LLM applications; being acquired by OpenAI.

Read the project profile

W&B Weave

Weights & Biases' toolkit for tracing, evaluating and comparing LLM applications; Weights & Biases is part of CoreWeave.

Read the project profile

Helicone

An AI gateway and LLM observability platform for routing and monitoring AI applications.

Read the project profile

Portkey

A production stack for generative AI applications across an organization, including a gateway for model access.

Read the project profile

LangChain

A Python framework for composing model calls, tools, and agent behavior through a configurable agent harness.

Read the project profile

Laminar

An open-source platform to trace, evaluate and debug AI agents, highlighting where an agent failed.

Read the project profile

Opik

Comet's open-source platform for tracing, evaluating and testing LLM applications and agents.

Read the project profile

DeepEval

An open-source framework for testing LLM applications, with 50+ metrics for agents, RAG and chatbots that run like unit tests.

Read the project profile

Agenta

An open-source workspace for building agents through chat, improving them with feedback and evaluations, and sharing them with your team.

Read the project profile

OpenLIT

An open-source, OpenTelemetry-native platform for agent observability, evaluations, guardrails, prompt management and cost and GPU monitoring.

Read the project profile

Requesty

An AI gateway that routes model requests through one API with caching, provider fallbacks, spend controls and usage analytics.

Read the project profile

Respan

Respan combines a model gateway with fallbacks, caching and spend limits, plus tracing and evaluations for AI applications and agents.

Read the project profile

Giskard

A testing platform for finding security vulnerabilities and evaluating answer quality in LLM agents and retrieval applications.

Read the project profile

Lunary

An LLM application platform for tracing model and tool calls, reviewing conversations and managing prompt versions.

Read the project profile

PromptLayer

An AI application platform for versioning prompts, running dataset-based evaluations and connecting production traces to prompt and workflow changes.

Read the project profile

Kiln AI

An AI development workbench combining evaluations, prompt optimization, synthetic datasets, retrieval, agents and fine-tuning with a desktop app and Python library.

Read the project profile

AgentOps

An agent observability platform with a Python SDK for tracing model calls, tools and agent interactions, replaying runs and tracking token costs.

Read the project profile

TruLens

An open-source Python toolkit for tracing AI agents and evaluating retrieval, responses and tool use with custom metrics and model-based judges.

Read the project profile

Bifrost

An AI gateway that connects model providers through one API with fallbacks, load balancing, semantic caching and usage controls.

Read the project profile

LLM Gateway

An LLM API gateway for routing requests across providers with caching, failover and usage analytics.

Read the project profile

Inspect AI

An evaluation framework that combines datasets, agents, tools and scorers to assess language models and inspect their execution logs.

Read the project profile

Evidently

An open-source Python framework for evaluating LLM outputs, testing AI systems and monitoring data quality and model drift.

Read the project profile

garak

A command-line LLM vulnerability scanner that probes models and chatbots for prompt injection, data leakage, jailbreaks and other unwanted behavior.

Read the project profile

PyRIT

A Python framework for automated and human-led AI red teaming with multi-turn attacks, configurable targets, scoring and conversation tracking.

Read the project profile

Mindgard

An AI security platform for discovering attack surfaces, running adversarial tests against models and agents, and reviewing defensive gaps and remediation guidance.

Read the project profile

LangWatch

An AI engineering platform for tracing model and tool calls, testing agents with simulated conversations, evaluating outputs and managing prompts.

Read the project profile

Label Studio

An open-source platform for labeling multimodal datasets and collecting human evaluations of AI outputs.

Read the project profile

Latitude

An agent observability platform that traces production sessions, groups recurring failures and sends coding agents the context to investigate fixes.

Read the project profile

LangSmith

An observability and evaluation platform for tracing agent execution, inspecting failures and testing AI application outputs.

Read the project profile

Pydantic Logfire

An observability platform that traces model calls, agent tools, API requests and database queries together and evaluates AI outputs alongside those traces.

Read the project profile

Language Model Evaluation Harness

A framework for evaluating language models on benchmark tasks with configurable prompts, metrics and local or API-based model backends.

Read the project profile

MLflow

An AI engineering platform for tracing and evaluating LLM applications, versioning prompts, managing model access and tracking machine-learning experiments.

Read the project profile

OpenLLMetry

Traceloop's open-source set of OpenTelemetry extensions that trace LLM calls, vector databases and agent frameworks and send the data to Traceloop or an existing observability backend.

Read the project profile

HoneyHive

An observability and evaluation platform for AI agents that records traces, scores live traffic and runs experiments, built on OpenTelemetry.

Read the project profile

ClearML

An open-source MLOps and AI infrastructure platform that combines experiment tracking, dataset versioning, pipelines and model serving with GPU cluster scheduling, quotas and fractional GPUs across cloud and on-premises hardware.

Read the project profile

Coval

Simulation, evaluation and monitoring platform for voice and chat AI agents, with human review queues, API, CLI and MCP server.

Read the project profile

Cekura

Automated QA platform for voice and chat AI agents that simulates conversations, red-teams, benchmarks providers and monitors production calls.

Read the project profile

Go a little deeper