Nightly

Public

Tests that are too slow or non-deterministic for the regular Test pipeline

Scheduled build

Failed in 7h 47m
:pipeline:
:rust: Miri test (full)
Extended SSH connection tests
CRDB rolling restarts
PubSub disruption
Tests for balancerd
Legacy upgrade tests (last version from git)
CRDB / Persist backup and restore
Postgres / Persist backup and restore
Replica isolation
Zero downtime
SQLsmith explain succeeded with known error logs, main history: :bk-status-passed::bk-status-passed::bk-status-passed::bk-status-passed::bk-status-passed:
sqlsmith-mz_1-1  | thread 'tokio:work-7' panicked at src/transform/src/equivalence_propagation.rs:76:13: ColumnKnowledge performed work after EquivalencePropagation
Test details & reproducer Use SQLsmith to generate random queries (AST/code based) and run them against Materialize: https://github.com/MaterializeInc/sqlsmith The queries can be complex, but we can't verify correctness or performance.
bin/mzcompose --find sqlsmith run default --max-joins=5 --explain-only --runtime=1500 

Platform checks upgrade in Cloudtest/K8s failed, main history: :bk-status-passed::bk-status-passed::bk-status-passed::bk-status-passed::bk-status-passed:

materialize.ui.CommandFailureCausedUIError: Running testdrive --materialize-url=postgres://materialize:materialize@environmentd:6875/materialize --materialize-internal-url=postgres://mz_system@environmentd:6877/materialize --kafka-addr=redpanda:9092 --schema-registry-url=http://redpanda:8081 --default-timeout=300s --log-filter=off --var=replicas=1 --var=leaves-tombstones=false --var=single-replica-cluster=quickstart --var=default-storage-size=1 --var=default-replica-size=1 --cluster-replica-sizes={"bootstrap": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 1}, "2-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 2, "workers": 4}, "free": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "0", "disabled": true, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 0, "workers": 0}, "1cc": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": null, "scale": 1, "workers": 1}, "1C": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": null, "scale": 1, "workers": 1}, "1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 1}, "1-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 1}, "1-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 1}, "1-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 1}, "1-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 1}, "1-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 1}, "mem-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "1 GiB", "scale": 1, "workers": 1}, "2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 2}, "2-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 2}, "2-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 2}, "2-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 2}, "2-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 2}, "2-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 2, "workers": 1}, "2-2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 2, "workers": 2}, "mem-2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "2 GiB", "scale": 1, "workers": 2}, "4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 4}, "4-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 4}, "4-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 4}, "4-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 4}, "4-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 4}, "4-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 4, "workers": 1}, "4-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 4, "workers": 4}, "mem-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 4}, "8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 8}, "8-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 8}, "8-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 8}, "8-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 8}, "8-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 8}, "8-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 8, "workers": 1}, "8-8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "64", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 8, "workers": 8}, "mem-8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 8}, "16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 16}, "16-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 16}, "16-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 16}, "16-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 16}, "16-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 16}, "16-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 16, "workers": 1}, "16-16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "256", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 16, "workers": 16}, "mem-16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 16}, "32": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 32}, "32-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 32}, "32-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 32}, "32-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 32}, "32-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 32}, "32-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 32, "workers": 1}, "32-32": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1024", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 32, "workers": 32}, "mem-32": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc" [...]
Traceback (most recent call last):
  File "/var/lib/buildkite-agent/builds/buildkite-medium-aarch64-f931a0c-i-0e19141e694f475f6-1/materialize/nightly/misc/python/materialize/cloudtest/k8s/testdrive.py", line 176, in _run_internal
    self.kubectl(
  File "/var/lib/buildkite-agent/builds/buildkite-medium-aarch64-f931a0c-i-0e19141e694f475f6-1/materialize/nightly/misc/python/materialize/cloudtest/k8s/api/k8s_resource.py", line 47, in kubectl
    run_process_with_error_information(
  File "/var/lib/buildkite-agent/builds/buildkite-medium-aarch64-f931a0c-i-0e19141e694f475f6-1/materialize/nightly/misc/python/materialize/cloudtest/util/common.py", line 118, in run_process_with_error_information
    raise e
  File "/var/lib/buildkite-agent/builds/buildkite-medium-aarch64-f931a0c-i-0e19141e694f475f6-1/materialize/nightly/misc/python/materialize/cloudtest/util/common.py", line 113, in run_process_with_error_information
    subprocess.run(
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['kubectl', '--context', 'kind-mzcloud', '--namespace', 'default', 'exec', '-it', 'testdrive', '--', 'testdrive', '--materialize-url=postgres://materialize:materialize@environmentd:6875/materialize', '--materialize-internal-url=postgres://mz_system@environmentd:6877/materialize', '--kafka-addr=redpanda:9092', '--schema-registry-url=http://redpanda:8081', '--default-timeout=300s', '--log-filter=off', '--var=replicas=1', '--var=leaves-tombstones=false', '--var=single-replica-cluster=quickstart', '--var=default-storage-size=1', '--var=default-replica-size=1', '--cluster-replica-sizes={"bootstrap": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 1}, "2-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 2, "workers": 4}, "free": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "0", "disabled": true, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 0, "workers": 0}, "1cc": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": null, "scale": 1, "workers": 1}, "1C": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": true, "memory_limit": null, "scale": 1, "workers": 1}, "1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 1}, "1-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 1}, "1-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 1}, "1-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 1}, "1-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 1}, "1-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 1}, "mem-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "1", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "1 GiB", "scale": 1, "workers": 1}, "2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 2}, "2-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 2}, "2-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 2}, "2-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 2}, "2-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 2}, "2-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 2, "workers": 1}, "2-2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 2, "workers": 2}, "mem-2": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "2", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "2 GiB", "scale": 1, "workers": 2}, "4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 4}, "4-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 4}, "4-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 4}, "4-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 4}, "4-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 4}, "4-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 4, "workers": 1}, "4-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 4, "workers": 4}, "mem-4": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "4", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 4}, "8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 8}, "8-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 8}, "8-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 8}, "8-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 8}, "8-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 8}, "8-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 8, "workers": 1}, "8-8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "64", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 8, "workers": 8}, "mem-8": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "8", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 8}, "16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 16}, "16-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "4 GiB", "scale": 1, "workers": 16}, "16-8G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "8 GiB", "scale": 1, "workers": 16}, "16-16G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 16}, "16-32G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "32 GiB", "scale": 1, "workers": 16}, "16-1": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 16, "workers": 1}, "16-16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "256", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 16, "workers": 16}, "mem-16": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "16", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": "16 GiB", "scale": 1, "workers": 16}, "32": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_hour": "32", "disabled": false, "disk_limit": null, "is_cc": false, "memory_limit": null, "scale": 1, "workers": 32}, "32-4G": {"cpu_exclusive": false, "cpu_limit": null, "credits_per_ [...]
Test details & reproducer
bin/pytest --junitxml=junit_cloudtest_01939961-e0cd-4ec2-92d8-6089d64132eb.xml -m=long test/cloudtest/test_upgrade.py --splits=1 --group=1

Terraform + Helm Chart E2E on AWS failed, main history: :bk-status-failed::bk-status-passed::bk-status-passed::bk-status-passed::bk-status-passed:

subprocess.CalledProcessError: Command '['kubectl', 'apply', '-f', '-']' returned non-zero exit status 1.
Test details & reproducer Tests the mz command line tool against a real Cloud instance
bin/mzcompose --find terraform run default 

Parallel Workload (DDL) failed, main history: :bk-status-passed::bk-status-passed::bk-status-passed::bk-status-passed::bk-status-passed:

materialize.parallel_workload.worker_exception.WorkerFailedException: ^^^ +++ Thread worker_2 failed, exiting
('too many values to unpack (expected 1)', 'rollback;')
Test details & reproducer Runs a randomized parallel workload stressing all parts of Materialize, can mostly find panics and unexpected errors. See zippy for a sequential randomized tests which can verify correctness.
bin/mzcompose --find parallel-workload run default --runtime=1500 --threads=8 

Output consistency (feature-flags for DFR) failed, main history: :bk-status-passed::bk-status-failed::bk-status-failed::bk-status-failed::bk-status-failed:

Docker compose failed: docker compose -f/dev/fd/8 --project-directory /var/lib/buildkite-agent/builds/hetzner-aarch64-4cpu-8gb-aa961d52/materialize/nightly/test/feature-flag-consistency up --detach --wait --quiet-pull mz_this mz_other
feature-flag-consistency-mz_this-1  | thread 'tokio:work-1' panicked at src/transform/src/equivalence_propagation.rs:76:13: ColumnKnowledge performed work after EquivalencePropagation
Test details & reproducer Verify that Materialize has the same results with different sets of feature flags.
bin/mzcompose --find feature-flag-consistency run default --seed=01939960-2093-4333-8cb7-6c951de7f112 --max-runtime-in-sec=1200 --evaluation-strategy=dataflow_rendering 
Platform checks upgrade in Cloudtest/K8s
Waited 57s
·
Ran in 30m 41s
Terraform + Helm Chart E2E on AWS
Waited 56s
·
Ran in 28m 24s
Output consistency (feature-flags for DFR)
Waited 3m 3s
·
Ran in 12m 59s
RQG dbt3-joins workload
Timed Out
Waited 2m 36s
·
Ran in 1h 1m
RQG dbt3-joins workload
Timed Out
Waited 2m 5s
·
Ran in 1h 1m
Total Job Run Time: 4d 17h