Browse the directory

Open-source observability platform purpose-built for AI agents, with OpenTelemetry-native tracing, plain-English signals, an evals SDK and CLI, SQL dashboards, dataset annotation, and MCP/CLI access, self-hostable with Apache-2.0 SDKs for Python and TypeScript.

Debug, evaluate, and monitor LLM applications from Claude — read traces and spans, score outputs, save prompts, run evaluation experiments, and query project metrics — with the official Opik MCP server by Comet.

Open-source TypeScript agent engineering framework and platform for building AI agents with tools, memory, workflows, RAG, guardrails, evals, MCP, voice, and VoltOps observability.

Open-source framework from OpenAI for evaluating LLM and agent behavior with reusable eval definitions, grading logic, datasets, and regression workflows.

Python agent framework from the Pydantic team for type-safe GenAI apps, tools, structured outputs, MCP, evals, and durable workflows.

Inspect LLM traces and spans, manage prompts, explore datasets, and review evaluation experiments from Claude — with the official Arize Phoenix MCP server, built into the open-source Phoenix AI observability platform.

Open-source evaluation framework for testing RAG systems, prompts, agents, workflows, and other LLM application behavior.

Source-backed agent for designing Inspect AI benchmark tasks, datasets, solver plans, scorer rubrics, model matrices, eval logs, and release-quality prompt evaluation decisions.