AIBench.cc
Benchmark

LLM API Benchmark | AI API Latency, TTFT and Model Purity Test

Run an LLM API benchmark for OpenAI, Claude, Gemini and AI API relays. Measure P95 latency, TTFT, model purity, prompt cache, rate limits and token cost.

Quick Answer

An LLM API benchmark compares real API behavior across models and providers with the same probes. AIBench.cc measures P95 latency, TTFT, prompt cache, model purity, rate limits and token cost so you can decide whether an AI API channel is ready for production.

How It Works

  1. 1Choose OpenAI-compatible, Anthropic Claude or Google Gemini protocol.
  2. 2Enter the base URL, model id and a temporary test API key.
  3. 3Run connectivity, latency, cache, rate limit, model identity and cost probes.
  4. 4Review a shareable report and compare qualified samples on the leaderboard.

Use Cases

  • Compare LLM API vendors before procurement.
  • Validate whether an AI API relay is stable or silently swapping models.
  • Monitor P95 latency, TTFT and 429 rate limits during rollout.
  • Check whether prompt cache really reduces long-context cost.

Core Metrics

P95 latency

How long 95% of requests take to complete

TTFT

Time to first streamed content chunk

Model purity

Consistency across model field, self-identity and tokenizer signals

Cache hit

Whether cache_read or cached_tokens fields appear

Why average latency is not enough

Average latency hides spikes. Chat apps, agents and batch workflows suffer from tail latency, so AIBench separates TTFT from end-to-end P95/P99 latency.

Why benchmark model purity

The same model id can point to a downgraded, quantized or substituted model. Speed-only tests reward low-quality relays, so AIBench treats model purity as a gate.

Related Search Terms

LLM API benchmarkAI API latency testOpenAI API benchmarkClaude API checkmodel purity testprompt cache

FAQ

Which metrics matter in an LLM API benchmark?

At minimum: P95 latency, TTFT, success rate, 429 rate limits, prompt cache hit, model purity and actual token cost. Average latency alone is not enough.

Can AIBench benchmark AI API relays?

Yes. If the relay exposes an OpenAI-compatible API or native Claude/Gemini protocol, you can run probes with a temporary test key.

Related Guides and Comparisons

Which Metrics Matter in an LLM API Benchmark?Official API vs API Relay: Check Differences

Start a Real API Check

Paste a temporary test key and run the same probes for latency, model purity, cache, rate limits and cost.

Start Check