Agent Analyzer

Get from raw agent execution data to a diagnostic report in five steps:

Agent Analyzer expects a JSON object representing a single agent run. The top level contains metadata and an array of individual execution steps.

Top-Level Schema

Each element in the steps array describes a distinct action taken by the agent (an LLM inference call, tool invocation, vector retrieval, or custom step).

Before executing analysis, Agent Analyzer validates the input trace against a strict schema in parser.ts.

Validation checks include:

• Valid JSON syntax parsing.
• Top-level value is a JSON object (not primitive or array).
• Required fields (id, name, startTime, endTime, steps) are present and non-empty.
• Timestamps conform to valid ISO 8601 formats parseable by standard JavaScript date engines.
• The steps array contains at least one step.
• Every step object has a valid type ("llm" | "tool" | "retrieval" | "other") and status ("success" | "error").
• Numeric fields (inputTokens, outputTokens, cost) are non-negative numbers if provided.

When validation fails, an explicit error message indicates the exact step index and offending field (e.g. "steps[2] (id: step_03): 'type' must be 'llm' | 'tool' | 'retrieval' | 'other'").

Agent Analyzer executes a deterministic TypeScript pipeline completely in your browser session:

The analyzer does not make external API requests or invoke remote LLMs to evaluate your run. Calculations and issue detection are 100% deterministic algorithms running on the normalized step array.

Agent Analyzer runs 6 deterministic diagnostic detectors over the normalized trace:

The report at /analyze/results is organized hierarchically so you can triage an agent run in seconds:

• Run Overview: Four top-level KPI cards (Total Cost, Latency, Tokens, Issues count) answering how the run performed at a glance.
• Health Status: Shows whether the run is healthy or lists the exact breakdown of optimization findings.
• Fix This First: Automatically isolates the single most critical issue detected, paired with an actionable engineering recommendation.
• Detected Issues: Categorized list of all secondary findings grouped by severity (High, Medium, Low).
• Cost & Model Analysis: Side-by-side breakdown of cost per step and expenditure grouped by LLM model.
• Token & Latency Breakdowns: Visual context growth curve and horizontal step duration bars highlighting anomalies.
• Step Breakdown: Complete chronological log of every step, model, token count, cost, duration, and status.

An individual step with status "error" indicates that a specific tool call or API request failed.

A step error does NOT mean the entire agent run failed. Real-world agents frequently recover from step errors via retries, alternate tool selections, or fallback prompt routines.

Agent Analyzer flags step errors and retries to help you identify flaky tools or intermittent network issues, but preserves the context of the overall run workflow.

The following complete trace represents a realistic 5-step customer support workflow containing LLM inferences, tool calls, a retrieval step, and a transient error recovery:

{
  "id": "run_support_091",
  "name": "customer-support-agent",
  "startTime": "2026-08-23T10:00:00.000Z",
  "endTime": "2026-08-23T10:00:04.850Z",
  "steps": [
    {
      "id": "step_01",
      "name": "classify_intent",
      "type": "llm",
      "model": "gpt-4o-mini",
      "inputTokens": 3200,
      "outputTokens": 85,
      "cost": 0.00053,
      "startTime": "2026-08-23T10:00:00.000Z",
      "endTime": "2026-08-23T10:00:00.680Z",
      "status": "success"
    },
    {
      "id": "step_02",
      "name": "lookup_account",
      "type": "tool",
      "startTime": "2026-08-23T10:00:00.690Z",
      "endTime": "2026-08-23T10:00:00.990Z",
      "status": "error"
    },
    {
      "id": "step_03",
      "name": "lookup_account",
      "type": "tool",
      "startTime": "2026-08-23T10:00:01.000Z",
      "endTime": "2026-08-23T10:00:01.280Z",
      "status": "success"
    },
    {
      "id": "step_04",
      "name": "fetch_kb_articles",
      "type": "retrieval",
      "startTime": "2026-08-23T10:00:01.300Z",
      "endTime": "2026-08-23T10:00:01.650Z",
      "status": "success"
    },
    {
      "id": "step_05",
      "name": "generate_resolution",
      "type": "llm",
      "model": "claude-3-5-sonnet-20241022",
      "inputTokens": 18400,
      "outputTokens": 620,
      "cost": 0.06450,
      "startTime": "2026-08-23T10:00:01.670Z",
      "endTime": "2026-08-23T10:00:04.850Z",
      "status": "success"
    }
  ]
}

You can copy this JSON and paste it directly into /analyze to inspect the resulting diagnostic report.