vllm.benchmarks.forward.main

Benchmark the model forward of exact batches as one-shot CUDA graphs.

Each case is a list of requests (query_len, kv_len, num_draft_tokens) per DP rank. The bench builds the batch without the scheduler, runs it through the real execute_model (input prep, attention metadata, DP sync), and times the target-model forward captured as a CUDA graph. Sampling, the speculator and CPU step overhead aren't measured.

Single GPU: vllm bench forward --model M --case decode=32*1@8192. Multi-GPU: torchrun --nproc-per-node=N -m vllm.entrypoints.cli.main bench forward ..., one process per GPU.