vllm bench

Overview

vLLM bench subcommand.

Subcommands

Command	Description
mm-processor	Benchmark multimodal processor latency across different configurations.
forward	Benchmark the model forward of exact batches as CUDA graphs.
latency	Benchmark the latency of a single batch of requests.
serve	Benchmark the online serving throughput.
startup	Benchmark the startup time of vLLM models.
sweep	Benchmark for a parameter sweep.
throughput	Benchmark offline inference throughput.