AI Agent Traces: What They Are and How to Analyze Them

What is an AI agent trace?

When an LLM-based agent processes a request, it typically executes multiple steps: it might call a language model to classify the input, invoke a tool to retrieve data, call the model again to reason over the results, and repeat this loop until it produces an answer.

A trace records each of these steps in order, with metadata attached — when it started, when it ended, how many tokens were used, what it cost, and whether it succeeded or failed. This makes a trace the primary artifact for understanding agent behavior after the fact.

What information does a trace contain?

Agent Analyzer expects a trace JSON with the following structure:

Step types

Each step in a trace has a type field that categorizes what kind of operation it represents:

Why traces are useful for debugging

When an agent produces an unexpected result or behaves inefficiently, logs alone rarely tell the whole story. A trace gives you a structured, step-by-step account of what the agent actually did, which makes it possible to:

• →Identify which step consumed the most tokens or cost
• →Find slow steps that are adding unnecessary latency
• →Detect tools being called more than once when they could be cached
• →Spot retry patterns that indicate unreliable dependencies
• →See how context grows across LLM calls and where costs are inflating

Example trace

This is a minimal valid trace with 3 steps — one LLM call, one tool call, and another LLM call. It shows the required fields and some optional fields like model, inputTokens, and cost:

{
  "id": "run_001",
  "name": "my-agent",
  "startTime": "2024-11-14T10:00:00.000Z",
  "endTime": "2024-11-14T10:00:05.200Z",
  "steps": [
    {
      "id": "step_01",
      "type": "llm",
      "name": "classify_request",
      "model": "gpt-4o-mini",
      "inputTokens": 1200,
      "outputTokens": 80,
      "cost": 0.000192,
      "startTime": "2024-11-14T10:00:00.000Z",
      "endTime": "2024-11-14T10:00:00.720Z",
      "status": "success"
    },
    {
      "id": "step_02",
      "type": "tool",
      "name": "search_database",
      "startTime": "2024-11-14T10:00:00.720Z",
      "endTime": "2024-11-14T10:00:01.180Z",
      "status": "success"
    },
    {
      "id": "step_03",
      "type": "llm",
      "name": "generate_response",
      "model": "gpt-4o-mini",
      "inputTokens": 4800,
      "outputTokens": 340,
      "cost": 0.000816,
      "startTime": "2024-11-14T10:00:01.180Z",
      "endTime": "2024-11-14T10:00:05.200Z",
      "status": "success"
    }
  ]
}

Note: model, inputTokens, outputTokens, and cost are optional. The analyzer works without them, but cost and token analysis will only be available when these fields are present.

How Agent Analyzer analyzes a trace

Once you paste or upload your trace, Agent Analyzer validates the JSON against the schema above, then runs a set of deterministic calculations entirely in your browser. No data leaves your device.

The analysis produces:

• →Total cost — sum of all step cost fields
• →Total tokens — sum of inputTokens and outputTokens across all steps
• →Total duration — endTime minus startTime of the run
• →Cost per step — each step's cost as a percentage of total
• →Cost per model — grouped by the model field on llm steps
• →Token growth — cumulative inputTokens across llm steps in order
• →Latency per step — duration of each step in milliseconds
• →Detected issues — six categories of potential problems

For details on individual analysis areas, see the related articles: cost analysis, token usage, latency, and debugging.