Monitor performance of target GPU simulations

Try main branch

Failed in 1h 7m
buildkite-agent pipeline upload .buildkite/gpu_...
init :GPU:
gpu_aquaplanet_diagedmf - 1 GPUmkdir -p gpu_aquaplanet_diagedmf && nsys profile --delay 200 --trace=nvtx,mpi,cuda,osrt --output=gpu_aquaplanet_diagedmf/output_active/report julia --threads=3 --color=yes --project=.buildkite .buildkite/ci_driver.jl --config_file config/model_configs/aquaplanet_diagedmf.yml --job_id gpu_aquaplanet_diagedmf
Waited 18m 1s
·
Ran in 16s
gpu_aquaplanet_diagedmf_benchmarkjulia --color=yes --project=.buildkite perf/benchmark.jl --config_file config/model_configs/aquaplanet_diagedmf.yml --job_id gpu_aquaplanet_diagedmf_benchmark
Waited 18m 1s
·
Ran in 14s
gpu_aquaplanet_progedmf - 1 GPUmkdir -p gpu_aquaplanet_progedmf && nsys profile --delay 100 --trace=nvtx,mpi,cuda,osrt --output=gpu_aquaplanet_progedmf/output_active/report julia --threads=3 --color=yes --project=.buildkite .buildkite/ci_driver.jl --config_file config/model_configs/aquaplanet_progedmf.yml --job_id gpu_aquaplanet_progedmf
Waited 18m 32s
·
Ran in 17s
gpu_aquaplanet_progedmf_benchmarkjulia --color=yes --project=.buildkite perf/benchmark.jl --config_file config/model_configs/aquaplanet_progedmf.yml --job_id gpu_aquaplanet_progedmf_benchmark
Waited 18m 31s
·
Ran in 14s
Total Job Run Time: 2h 24m