Trace Insights: metric definitions#

This is the normative reference for every derived metric the Insights tab and the run comparison show. The web implementation lives in js/insights-model.js, js/pricing.js, and js/insights-diff.js; the iOS implementation mirrors these definitions exactly and both are locked by the shared golden fixture (test/fixtures/insights-golden.jsonl plus its insights-golden.expected.json).

Insights reads ONLY the optional usage and timing fields of the turn records defined in format.md, plus thinking block content for the estimated-thinking fallback. It introduces no format changes, no new record types, and no new fields. Traces without turn records (classic mcp-only traces) show no Insights tab and render byte-identically to before.

Inputs and scope#

• Only role: "assistant" turns carry usage math. User and system turns

count toward coverage totals only.

• Per-model grouping keys on the turn model field; assistant turns without

a model id group under (no model) and are counted in coverage.

• Every metric is either a finite number or absent (rendered n/a). No

NaN, no Infinity, anywhere. Zero-duration turns yield absent rates.

Usage-field invariants (inherited, not enforced here)#

The shared turn spec (format.md, "Usage invariant") guarantees two invariants that every formula below assumes. Producers normalize provider-reported usage at record time; the recorder's provider adapters (the llm-tape usage-normalization work) enforce them:

1. usage.input_tokens EXCLUDES cache-read and cache-creation tokens.

OpenAI/DeepSeek adapters subtract cached_tokens / prompt_cache_hit_tokens from prompt_tokens before writing the turn. Consequence: cached tokens are billed exactly once by the single cost formula, and the cache hit rate denominator never double counts.

1. usage.thinking_tokens, when present, is a SUBSET of output_tokens.

OpenAI reasoning_tokens sit inside completion_tokens; Anthropic thinking is billed as output; the Gemini adapter folds thoughtsTokenCount into output_tokens and keeps thinking_tokens for display. Consequence: thinking tokens are display-only and are NEVER priced separately. Pricing them again would double count.

Because of these invariants the viewer keeps ONE formula for cost and one for cache hit rate, with no provider branches of any kind.

Cache hit rate#

cache hit rate = cache_read / (input + cache_read + cache_creation)

computed over group totals (per model, and session-wide), where input sums input_tokens across the group's usage-bearing turns and the cache terms sum across the turns that report cache fields.

Reporting semantics, which are not interchangeable:

• A turn reporting NO cache field at all counts as "cache not reported" in

coverage. When no turn in a group reports any cache field, the group's hit rate is absent and renders n/a. It is never an implicit 0%.

• 0% requires reported cache fields with cache_read = 0.
• A turn missing only some cache fields computes over the available fields

(the missing field contributes 0 within that reporting turn).

• A zero denominator yields an absent rate, never NaN.

Tokens per second#

Two-tier timing fallback:

1. Both ttft_ms and duration_ms present and duration_ms > ttft_ms:

output_tokens / ((duration_ms - ttft_ms) / 1000) (generation rate).

1. Only duration_ms present (and positive):

output_tokens / (duration_ms / 1000), flagged approximate and rendered with an approx. suffix.

1. Neither: the metric is absent.

Thinking magnitude#

Fallback chain, per turn:

1. usage.thinking_tokens present: exact.
2. Otherwise, thinking block characters / 4 (minimum 1 when any thinking

text exists): flagged estimated and rendered with an est. label everywhere it appears, including run-comparison callouts.

1. Neither: absent.

Session totals keep the exact and estimated buckets separate; the summary shows est. whenever any estimated bucket contributes. Insights surfaces never display thinking prose, only counts and durations.

Percentiles#

Nearest-rank over the FULL sorted array: for percentile p over n values, rank = ceil(p/100 * n) and the value is sorted[rank - 1]. This is deliberately NOT the capped-at-64 live p95 shortcut used by the Tools view (bumpToolStat in js/trace-model.js), and the identical definition ships in the iOS TraceKit implementation so golden values agree across surfaces.

Estimated cost#

usd = (input_tokens * inPerMtok
     + output_tokens * outPerMtok
     + cache_read_input_tokens * cacheReadPerMtok
     + cache_creation_input_tokens * cacheWritePerMtok) / 1e6

• Rates are USD per million tokens from the bundled table

(js/pricing-data.js) merged with local overrides. Matching is longest-prefix on the model id.

• An unknown model resolves to NO price: the row shows "no price data" and

session totals disclose how many calls were unpriced. There is never a guessed or defaulted rate.

• A usage field with tokens but no rate (for example cache_creation tokens

with no cache-write rate) makes the estimate partial: the remaining fields still price, and the figure is labeled (partial).

• thinking_tokens never enters the formula (invariant 2 above).
• Cache creation is priced at the table's cache-write rate and cache reads

at the cache-read rate; both come from the table, never from hardcoded multipliers.

Pricing honesty posture#

Every cost figure is an estimate from published list prices, labeled est. with the table's newest as-of date visible next to it. Rates are editable (Price settings on the Insights tab), persist locally (localStorage on web, UserDefaults on iOS), and reset to the bundled table in one action. An "Add model" escape hatch covers brand-new models before a viewer release ships updated rates. Nothing here is billing, invoicing, or account truth: token counts come from the provider, prices from a bundled snapshot that goes stale. A shared trace is priced by each viewer's local table and overrides, so two people can see different estimates for the same trace; that variance is accepted under the estimate posture (embedding a price snapshot in the trace would be a format change and is out of scope).

Graceful-absence matrix#

The coverage line ("usage on X of Y assistant turns, ttft on N, duration on M, cache fields on K") makes the density of the underlying data visible next to every aggregate. Claude Code imports typically carry duration_ms at best and no ttft_ms; only live LLM recordings have full timing.

Run-vs-run alignment and callouts#

• Assistant turns align by role-sequence ordinal: the i-th assistant turn

of run A pairs with the i-th of run B. User/system turns are not aligned. This alignment (js/insights-diff.js) is separate from the tool-call diff's name+ordinal alignment (js/diff-model.js), which is unchanged.

• Runs of different lengths produce trailing steps with one side absent.
• If either trace carries no assistant turns the comparison is

incomparable: one-sided traces show a notice ("Run comparison needs turn records in both traces") above the unchanged tool diff; two mcp-only traces render exactly the pre-existing diff.

• Callouts fire when a step's metric ratio (larger side over smaller)

reaches the threshold of 2.0, ordered largest ratio first, capped at 5, for: thinking magnitude, TTFT, duration, output tokens, and est. cost. Example: Run B thought 4.2x longer at step 3.

• When the thinking magnitude on either side is estimated, the callout

appends (est.). When BOTH sides are estimated, chars/4 error compounds, so the callout requires the higher threshold of 3.0 instead of 2.0.

• A metric present on only one side never divides to Infinity: value

metrics (thinking, output tokens, cost) produce "Only run B ..." style callouts; timing metrics simply skip (a missing ttft is a recording gap, not behavior).

• Run comparison costs use the viewer's current price table plus overrides

and carry the same estimate disclaimer as the Insights tab.

Verifying a value by hand#

Take the golden fixture's gpt-5 turn: input_tokens: 600, output_tokens: 300, cache_read_input_tokens: 400, ttft_ms: 200, duration_ms: 2200, priced at in 1.25 / out 10.0 / cache-read 0.125 per Mtok. Then hit rate = 400 / (600 + 400) = 40.0%, tok/s = 300 / ((2200 - 200) / 1000) = 150, and est. cost = (600 x 1.25 + 300 x 10 + 400 x 0.125) / 1e6 = $0.003800. Those are exactly the values the parity tests assert on both surfaces.