Skip to content

vllm.benchmarks.forward

Modules:

  • batch –

    Build a SchedulerOutput for an exact batch, without the scheduler.

  • capture –

    Capture the forward of one exact batch as a one-shot CUDA graph and time it.

  • kv_init –

    KV blocks of a benchmark case: initialization and the blocks a step writes.

  • main –

    Benchmark the model forward of exact batches as one-shot CUDA graphs.

  • routing –

    Balanced MoE routing for benchmarks with dummy weights.

  • shapes –

    Exact batch shapes for vllm bench forward.