๐
Performance Benchmark
Publicrerun trt , with --ignore-eos set off

bootstrap
๐ Ready for comparing vllm against alternatives? This will take 4 hours.
A100 trt llama-70B
Collect the results
๐ check the results!
Wait for container to be ready
A100
bootstrapcurl -sSL https://raw.githubusercontent.com/vllm-project/buildkite-ci/main/scripts/kickoff-benchmark.sh | bash
Waited 30s
Ran in 10s

Total Job Run Time: 10s