AI Agent Latency: Find Slow Steps in Agent Runs

What latency means in an agent run

Each step in a trace has a startTime and an endTime field, both as ISO 8601 strings. The duration of a step is:

// From analyzer.ts
durationMs = new Date(step.endTime).getTime() - new Date(step.startTime).getTime()

Total run duration is calculated from the run's top-level timestamps, not by summing step durations — which means gaps between steps (e.g. serialization overhead, orchestration delay) are included:

// From analyzer.ts — calculateLatency
durationMs = new Date(run.endTime).getTime() - new Date(run.startTime).getTime()

Why step-level latency matters

A run that takes 8 seconds might have 6 steps averaging under a second and one step taking 4 seconds. Without step-level timing, you cannot distinguish which step is the bottleneck. Common sources of latency in agent runs include:

• →LLM provider latency — time-to-first-token and generation time scale with output length and context size
• →Tool call latency — API calls, database queries, and web requests add variable network overhead
• →Retrieval latency — vector database queries and embedding lookups can be slow under load
• →Orchestration overhead — framework routing, serialization, and state management between steps

Latency breakdown — example

Here is a step-by-step latency breakdown from a 13-step customer support agent run with a total duration of 8.73 seconds:

Mean step duration: 672ms. Steps exceeding 2.5× mean (1,680ms) AND >1s would be flagged. In this run no step crosses both thresholds simultaneously — the slowest step is 1,550ms (2.3× mean).

Slow step detector

The slow step detector flags steps that are both significantly slower than average and absolutely slow (over 1 second):

// From analyzer.ts — detectSlowSteps
const mean      = sum(durations) / durations.length
const threshold = mean * 2.5

// A step is flagged if BOTH conditions are true:
if (duration > threshold && duration > 1000) {
  severity = 'low'
  // detail: "took Xs, which is N× the mean step duration of Ys"
}

The two-condition requirement prevents false positives in fast runs where the mean is very low. A 300ms step in a run where the mean is 50ms would be 6× the mean but is not meaningfully slow, so it would not be flagged (300ms < 1000ms threshold).

Latency visualization

The results page shows a horizontal bar chart with one bar per step, scaled relative to the slowest step in the run. Steps flagged as slow are visually distinguished. This makes it immediately clear which steps are consuming disproportionate time, even before reading the detected issues section.

Diagnosing slow steps

When a slow step is detected, the recommendation is to profile whether the latency comes from the LLM provider, tool latency, or network overhead. In practice:

• →For llm steps: check if output length is unusually high (more output tokens = more generation time)
• →For llm steps: check if inputTokens is much higher than other LLM steps (longer context = slower TTFT on some providers)
• →For tool steps: check if the external API or database call is the bottleneck
• →For retrieval steps: check vector database query performance and embedding model latency