Model Performance Benchmark

LLM Model Performance

Track throughput, latency, memory fit, and deployment readiness for large language models and instruction-tuned inference stacks.

Benchmark Focus

Prompt throughput, generation latency, context fit, and serving readiness

Comparison Coverage

4 models / 7 profiles

Choose one model and one shared test condition to compare chips side by side

Tracked Models

Published llm models with active benchmark records

Comparable Profiles

Core filters resolve into one shared benchmark profile before table rendering

Chips Covered

Cloud and edge accelerators with published benchmark results

Comparison controls

Pick one model, refine high-frequency conditions in the core area, then use advanced filters only when you need deeper control.

Matching profiles 1

Model

Core Conditions

TaskRequest ModeInput TokensOutput TokensBatch SizePrecisionQuantization

Advanced Conditions

Expose lower-frequency benchmark knobs without crowding the default comparison flow.

SegmentManufacturer

Comparing 4 chip results

Selected model

Gemma 2 9B Instruct

INT4 Chat Serve · Google

Primary metric: latency (ms)

Task text-generationFramework TensorRT-LLMRuntime TensorRT runtimeBatch 4Seq 2048Precision BF16Quant INT4Variant INT4 Chat ServeMode onlineInput Tokens 2048Output Tokens 128Concurrency 4Context 8192Decoding greedy

Chip	Segment	Manufacturer	latency	Throughput	Latency P50	Latency P99	Power	Memory	Source
NVIDIA B200 1,800 TFLOPS	Cloud	NVIDIA	31 ms	1,260 ms	31 ms	40 ms	880 W	40 GB	View Source
NVIDIA H200 SXM 989 TFLOPS	Cloud	NVIDIA	41 ms	1,020 ms	41 ms	53 ms	620 W	38 GB	View Source
AMD Instinct MI300X 1,307 TFLOPS	Cloud	AMD	50 ms	940 ms	50 ms	64 ms	700 W	40 GB	View Source
Intel Gaudi 3 1,835 TFLOPS	Cloud	Intel	55 ms	820 ms	55 ms	69 ms	770 W	36 GB	View Source

Loading benchmark controls...

Browse Cloud Chips Browse Edge Chips