Nightly

Feature benchmark against merge base or 'latest' 2 (#2) failed, main history:

• Unknown error in Scenario 'ManyKafkaSourcesOnSameCluster':

New regression against v0.140.3

NAME                                | TYPE            |      THIS       |      OTHER      |  UNIT  | THRESHOLD  |  Regression?  | 'THIS' is
--------------------------------------------------------------------------------------------------------------------------------------------------------
ManyKafkaSourcesOnSameCluster       | wallclock       |          32.308 |          32.573 |   s    |    10%     |      no       | better:  0.8% faster
ManyKafkaSourcesOnSameCluster       | memory_mz       |        2549.171 |        2554.893 |   MB   |    20%     |      no       | better:  0.2% less
ManyKafkaSourcesOnSameCluster       | memory_clusterd |          68.855 |          35.114 |   MB   |    50%     |    !!YES!!    | worse:  96.1% more

BUILDKITE_PARALLEL_JOB=1 BUILDKITE_PARALLEL_JOB_COUNT=8 bin/mzcompose --find feature-benchmark run default --other-tag common-ancestor 

Feature benchmark against merge base or 'latest' 2 failed, main history:

• Unknown error in Scenario 'ManyKafkaSourcesOnSameCluster':

New regression against v0.140.3

NAME                                | TYPE            |      THIS       |      OTHER      |  UNIT  | THRESHOLD  |  Regression?  | 'THIS' is
--------------------------------------------------------------------------------------------------------------------------------------------------------
ManyKafkaSourcesOnSameCluster       | wallclock       |          26.139 |          23.370 |   s    |    10%     |    !!YES!!    | worse:  11.8% slower
ManyKafkaSourcesOnSameCluster       | memory_mz       |        2544.403 |        2572.060 |   MB   |    20%     |      no       | better:  1.1% less
ManyKafkaSourcesOnSameCluster       | memory_clusterd |          69.647 |          36.097 |   MB   |    50%     |    !!YES!!    | worse:  92.9% more

BUILDKITE_PARALLEL_JOB=1 BUILDKITE_PARALLEL_JOB_COUNT=8 bin/mzcompose --find feature-benchmark run default --other-tag common-ancestor 

Platform checks upgrade in Cloudtest/K8s failed, main history:

• Potential regression cloudtest upgrade platform-checks fails: Service "cluster-s1-replica-s1-gen-0" is invalid: spec.clusterIPs[0]: Invalid value: []string{"None"}: may not change once set (#8817, closed) in test_upgrade:

materialize.ui.CommandFailureCausedUIError: Running testdrive --materialize-url=postgres://materialize:materialize@environmentd:6875/materialize --materialize-internal-url=postgres://mz_system@environmentd:6877/materialize --kafka-addr=redpanda:9092 --schema-registry-url=http://redpanda:8081 --default-timeout=300s --log-filter=off --var=replicas=1 --var=single-replica-cluster=quickstart --var=default-storage-size=1 --var=default-replica-size=1 --cluster-replica-sizes={"bootstrap": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "2-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 2, "workers": 4}, "free": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "0", "disabled": true, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 0, "workers": 0}, "1cc": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "1C": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "1-no-disk": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "2-no-disk": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4Gi", "scale": 1, "workers": 2}, "1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "1-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 1}, "1-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 1}, "1-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 1}, "1-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 1}, "1-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "mem-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "1 GiB", "scale": 1, "workers": 1}, "2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 2}, "2-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 2}, "2-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 2}, "2-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 2}, "2-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 2}, "2-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 2, "workers": 1}, "2-2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 2, "workers": 2}, "mem-2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "2 GiB", "scale": 1, "workers": 2}, "4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 4}, "4-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 4}, "4-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 4}, "4-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 4}, "4-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 4}, "4-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 4, "workers": 1}, "4-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 4, "workers": 4}, "mem-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 4}, "8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 8}, "8-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 8}, "8-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 8}, "8-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 8}, "8-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 8}, "8-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 8, "workers": 1}, "8-8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "64", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 8, "workers": 8}, "mem-8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 8}, "16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 16}, "16-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 16}, "16-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 16}, "16-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 16}, "16-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 16}, "16-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 16, "workers": 1}, "16-16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "256", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 16, "workers": 16}, "mem-16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 16}, "32": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 32}, "32-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 32}, "32-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 32}, "32-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 32}, "32-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 32}, "32-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 32, "workers":  [...]

Traceback (most recent call last):
  File "/var/lib/buildkite-agent/builds/buildkite-medium-aarch64-f931a0c-i-07528438edb29bd44-1/materialize/nightly/misc/python/materialize/cloudtest/k8s/testdrive.py", line 167, in _run_internal
    self.kubectl(
  File "/var/lib/buildkite-agent/builds/buildkite-medium-aarch64-f931a0c-i-07528438edb29bd44-1/materialize/nightly/misc/python/materialize/cloudtest/k8s/api/k8s_resource.py", line 47, in kubectl
    run_process_with_error_information(
  File "/var/lib/buildkite-agent/builds/buildkite-medium-aarch64-f931a0c-i-07528438edb29bd44-1/materialize/nightly/misc/python/materialize/cloudtest/util/common.py", line 118, in run_process_with_error_information
    raise e
  File "/var/lib/buildkite-agent/builds/buildkite-medium-aarch64-f931a0c-i-07528438edb29bd44-1/materialize/nightly/misc/python/materialize/cloudtest/util/common.py", line 113, in run_process_with_error_information
    subprocess.run(
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['kubectl', '--context', 'kind-mzcloud', '--namespace', 'default', 'exec', '-it', 'testdrive', '--', 'testdrive', '--materialize-url=postgres://materialize:materialize@environmentd:6875/materialize', '--materialize-internal-url=postgres://mz_system@environmentd:6877/materialize', '--kafka-addr=redpanda:9092', '--schema-registry-url=http://redpanda:8081', '--default-timeout=300s', '--log-filter=off', '--var=replicas=1', '--var=single-replica-cluster=quickstart', '--var=default-storage-size=1', '--var=default-replica-size=1', '--cluster-replica-sizes={"bootstrap": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "2-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 2, "workers": 4}, "free": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "0", "disabled": true, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 0, "workers": 0}, "1cc": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "1C": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "1-no-disk": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "2-no-disk": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4Gi", "scale": 1, "workers": 2}, "1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "1-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 1}, "1-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 1}, "1-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 1}, "1-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 1}, "1-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 1}, "mem-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "1 GiB", "scale": 1, "workers": 1}, "2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 2}, "2-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 2}, "2-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 2}, "2-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 2}, "2-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 2}, "2-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 2, "workers": 1}, "2-2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 2, "workers": 2}, "mem-2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "2 GiB", "scale": 1, "workers": 2}, "4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 4}, "4-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 4}, "4-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 4}, "4-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 4}, "4-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 4}, "4-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 4, "workers": 1}, "4-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 4, "workers": 4}, "mem-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 4}, "8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 8}, "8-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 8}, "8-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 8}, "8-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 8}, "8-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 8}, "8-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 8, "workers": 1}, "8-8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "64", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 8, "workers": 8}, "mem-8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 8}, "16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 1, "workers": 16}, "16-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4 GiB", "scale": 1, "workers": 16}, "16-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "8 GiB", "scale": 1, "workers": 16}, "16-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "16 GiB", "scale": 1, "workers": 16}, "16-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "32 GiB", "scale": 1, "workers": 16}, "16-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 16, "workers": 1}, "16-16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "256", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": "4Gi", "scale": 16, "workers": 16}, "mem-16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null [...]

bin/pytest --junitxml=junit_cloudtest_01961ce8-bbdc-41da-b266-10397ff883e9.xml -m=long test/cloudtest/test_upgrade.py --splits=1 --group=1