Speed Testing Methodology

Standard reports use server timing in hkg1 with versioned cases, exact endpoints and fixed parameters. Three categories × five cases × two windows at least six hours apart give 30 attempts per model. Failures, timeouts and truncations are retained. Community submissions are not official evidence.

No speed ranking is issued below ten completed samples per category. P95 requires 100 completed samples. Initial reports show median and range; Unicode character throughput is separate from provider billing tokens. Reports older than seven days are marked stale.

Costs use verified endpoint-specific prices; missing price or usage stays unknown. Parsing JSON or rendering HTML is not a quality score. Grading requires the exact case ID, version and unchanged prompt. There is no weighted overall recommendation.

View standard reports and raw evidence

All enabled models receive the same prompt and are started at the same time, instead of one after another. This keeps network conditions, time of day and prompt content aligned.

TTFT uses proxy timestamps from the upstream request to its first token (reasoning or content). It includes upstream network, scheduling, processing and buffering, excluding earlier client and proxy preparation. Actual wait is measured separately from client start to first token. Older streams without proxy timestamps fall back to client timing; missing historical diagnostics are not reconstructed. These timings alone cannot prove provider queueing.

For models with thinking output, thinking and final content are timed separately using each stage's active first-to-last-delta window. Gaps between stages and post-output usage waits are excluded.

Token counts prefer the provider's official usage values. When missing, TOKRACE uses a consistent character estimate and calibrates to official totals when available.

Peak speed is the maximum instant tokens/s over a 2-second sliding window, calibrated to official token totals when available.

Leaderboard data comes from voluntary anonymous sharing after arena runs. Only speed metrics are reported, never prompts or API keys. Different endpoints for the same model are tracked separately.

Speed varies with network conditions, time of day and provider load. Leaderboards use medians to reduce one-off jitter, but results are still reference data, not absolute truth. The project is open source and reproducible.