馃悗

Performance Benchmark

Public

switch to 8B

#8543
/
KuntaiDu:kuntai-update-nightlybench/6e3e6e144(#7412)
Passed in 1h 54m and blocked
bootstrap
馃殌 Ready for comparing vllm against alternatives? This will take 4 hours.
A100 trt llama-8B
Collect the results
馃殌 check the results!
Wait for container to be ready
A100
Kuntai Du
Kuntai Du
Created
Triggered from Webhook

Description

This file contains the downloading link for benchmarking results.

Please download the visualization scripts in the post

Results reproduction

  • Find the docker we use in benchmarking pipeline
  • Deploy the docker, and inside the docker:
    • Download nightly-benchmarks.zip.
    • In the same folder, run the following code
export HF_TOKEN=<your HF token>
apt update
apt install -y git
unzip nightly-benchmarks.zip
VLLM_SOURCE_CODE_LOC=./ bash .buildkite/nightly-benchmarks/scripts/run-nightly-benchmarks.sh

And the results will be inside ./benchmarks/results.

bootstrapcurl -sSL https://raw.githubusercontent.com/vllm-project/buildkite-ci/main/scripts/kickoff-benchmark.sh | bash
Waited 41s
路
Ran in 10s
Agent
Kuntai Du unblocked 馃殌 Ready for comparing vllm against alternatives? This will take 4 hours.
A100 trt llama-8B
Waited 1h 46m
路
Ran in 3m 3s
Agent
Collect the results
Waited 3m 55s
路
Ran in 19s
Agent
Wait for container to be ready
A100
Total Job Run Time: 3m 31s