Nightly
PublicTests that are too slow or non-deterministic for the regular Test pipeline
persist/storage: communicate projection pushdown via shard_source
Passed in 7h 36m
Product limits 1 failed, main history:
- Unknown error in workflow-main:
running docker compose failed (exit status 1)
Traceback (most recent call last):
File "/var/lib/buildkite-agent/builds/hetzner-aarch64-16cpu-32gb-9d80d26f/materialize/nightly/misc/python/materialize/mzcompose/composition.py", line 404, in invoke
raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['docker', 'compose', '-f/dev/fd/3', '--project-directory', PosixPath('/var/lib/buildkite-agent/builds/hetzner-aarch64-16cpu-32gb-9d80d26f/materialize/nightly/test/limits'), 'exec', '-T', 'testdrive', 'testdrive', '--kafka-addr=kafka:9092', '--schema-registry-url=http://schema-registry:8081', '--materialize-url=postgres://u1%40example.com:mzp_83ba159ba2654a6999971d740b4683ea55cd1cfe41c74c2282457a9fa9854e7b@balancerd:6875?sslmode=require', '--materialize-internal-url=postgres://materialize@materialized:6877', '--materialize-use-https', '--aws-endpoint=http://minio:9000', '--var=aws-endpoint=http://minio:9000', '--aws-access-key-id=minioadmin', '--var=aws-access-key-id=minioadmin', '--aws-secret-access-key=minioadmin', '--var=aws-secret-access-key=minioadmin', '--no-reset', '--default-timeout=60s', '--persist-blob-url=file:///mzdata/persist/blob', '--persist-consensus-url=postgres://root@materialized:26257?options=--search_path=consensus', '--source=/var/lib/buildkite-agent/builds/hetzner-aarch64-16cpu-32gb-9d80d26f/materialize/nightly/test/limits/mzcompose.py:1971']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/lib/buildkite-agent/builds/hetzner-aarch64-16cpu-32gb-9d80d26f/materialize/nightly/test/limits/mzcompose.py", line 1971, in workflow_main
c.testdrive(f.getvalue())
File "/var/lib/buildkite-agent/builds/hetzner-aarch64-16cpu-32gb-9d80d26f/materialize/nightly/misc/python/materialize/mzcompose/composition.py", line 1365, in testdrive
return self.exec(
^^^^^^^^^^
File "/var/lib/buildkite-agent/builds/hetzner-aarch64-16cpu-32gb-9d80d26f/materialize/nightly/misc/python/materialize/mzcompose/composition.py", line 844, in exec
return self.invoke(
^^^^^^^^^^^^
File "/var/lib/buildkite-agent/builds/hetzner-aarch64-16cpu-32gb-9d80d26f/materialize/nightly/misc/python/materialize/mzcompose/composition.py", line 433, in invoke
raise CommandFailureCausedUIError(
materialize.ui.CommandFailureCausedUIError: running docker compose failed (exit status 1)
Test details & reproducer
Stresses Materialize with large number of objects, large ingestions, etc. Good to prevent regressions in basic functionality for larger installations.BUILDKITE_PARALLEL_JOB=0 BUILDKITE_PARALLEL_JOB_COUNT=2 bin/mzcompose --find limits run main
aarch64 Docker tags produced in this build
materialize/ubuntu-base:mzbuild-ZDXXLIQ25S3GH73GW5Z2GUWHBSKRJM2H
materialize/prod-base:mzbuild-CHBE2FQ26CD4W4LYZJAVIYFIWDLCN7CV
materialize/balancerd:mzbuild-67QUCITMKAPODYRKBMJJ6ISMLOT2QPKD
materialize/bazel:mzbuild-FZKRDOTGX32G2GYY3KA3XRK3IKSJ7MTC
materialize/chbenchmark:mzbuild-7NPNIYHRXT32FYNDEGMUYTETXLMU7QOE
materialize/ci-java-smoketest:mzbuild-MKZBE2SKCGNNPZG5MUOEEK6TXVTGKFT7
materialize/ci-metabase-smoketest:mzbuild-VAOVP3SIZ2W2DF4SCEBP3EMLSUA3UKUT
materialize/cli:mzbuild-UCRQHCMBLRIBHA6ECCKWN7NNZHXL25AM
materialize/clusterd:mzbuild-65QX4WUQC3G4S2WQQF7KST4NLVVU4IYM
materialize/dbt-materialize:mzbuild-VP456JYHE6ADYWWUQXFOUBG6O2NV3S2W
materialize/debezium:mzbuild-RMYH5YYIXAC4K3VHNK7GUKZQHCAW75CY
materialize/environmentd:mzbuild-VQLTTDRQI7QUEI3FMSPT2UPIY26LMDQF
materialize/fivetran-destination:mzbuild-DMAUBAHHQXJ33LEGAOFR4UENM6CE5GRQ
materialize/fivetran-destination-tester:mzbuild-J23TS6XJNMWPL54XNIRTT2ZRYAP5YHC3
materialize/frontegg-mock:mzbuild-7TJEVZ2UNIJCVVN3RPA2EMC5YJHNZ377
materialize/jobs:mzbuild-PKZYM4VGFNVFT7BVJPD4SBB3TAJZ3ELH
materialize/kgen:mzbuild-2V2G7ON244536ZLH6YTJH2PL5X62J4P2
materialize/maelstrom-persist-base:mzbuild-3CJBQQPQ4FKCQVSFEMGDVDZJNULAZVLX
materialize/maelstrom-persist:mzbuild-XGDJJXOXNVZ2VGTG3BIQ46T3E3S6HM3N
materialize/materialized-base:mzbuild-AZCBTDMB3PWA55MEWSZTOVV5X4VHG45B
materialize/materialized:mzbuild-O6UJ6VGBT6DJHM6ZKMRDTF6BRHNAJ7FH
materialize/test-certs:mzbuild-DMYTEM6GOQFJPGBWRBN64J43QKHEGEDV
materialize/mysql:mzbuild-R4Y33YICQZZBXKY3YVLNFTCLGMUIXZ6L
materialize/mysql-client:mzbuild-SWMKXCOA7WKJWQVS3GXW3P5SA3CFARZ2
materialize/mz:mzbuild-K2QPGGDG77WKDGCGIA5EK3IVW24ML544
materialize/orchestratord:mzbuild-5EN6C3ZUUSXLXS2GGOG3YVD6X7F2LNRB
materialize/postgres:mzbuild-JDGCJUVYM3EY3QFMYDITXTFGXWBMDC2P
materialize/psql:mzbuild-3DKPCLCCR67L6GTXQP7AVLLOFTD2S2RB
materialize/sqlancer:mzbuild-WAVV6X6XEU2XZWMWRVXNHMLKB5GTFOHL
materialize/sqllogictest:mzbuild-WZBI6UKWKV6KON33P4WLWMY62UCWF4GP
materialize/sqlsmith:mzbuild-Q4VF54SVOAXL5XQIPXXA4YYV22APHPW7
materialize/testdrive-base:mzbuild-PHWSUEUATBIJUZCY62U3HCBUELMRUBVA
materialize/testdrive:mzbuild-7Q64HRQIPJ2VNRVPZX6FEFQWNO5XBMHF
x86_64 Docker tags produced in this build
materialize/ubuntu-base:mzbuild-PQQKF2CIMPLOQIF3EKEORR2MSRE45LH7
materialize/prod-base:mzbuild-SXSSD7ZVWJC26HY7DWHI5C2JEAGECCOM
materialize/balancerd:mzbuild-DKJKGP5WABQNJ27BEKY7R6UZSDCQYLOE
materialize/bazel:mzbuild-YY3NO65KI5YODSGNEENXXT7RT2PLDWIL
materialize/chbenchmark:mzbuild-QO5LD7ISGBEKYW4QVD75NK2PVSPFX65M
materialize/ci-java-smoketest:mzbuild-DEETHZ24HV3C4HE2POD2YYXZUG5MMRMI
materialize/ci-metabase-smoketest:mzbuild-QSNPHKNOOKZL2GKCNIYSLG5BRY2Q5VMR
materialize/cli:mzbuild-LCLQETIWXNZQWQEMTUT3TIYGVJPLVMVT
materialize/clusterd:mzbuild-47HXWOND6N6VTCPTM75M7BW74QNU7D6T
materialize/dbt-materialize:mzbuild-BQHB4IE5DGZQG3D47QWOADQ3LGFNIXBB
materialize/debezium:mzbuild-DH3UATRUW63NX4NC45OHBETXJZNLKBQ5
materialize/environmentd:mzbuild-4VDJEMPPP2JW3IRKJPOD5ADI7PLAJSNB
materialize/fivetran-destination:mzbuild-X44GTIS47S4BP62SKUT43PPPVSQCRZI6
materialize/fivetran-destination-tester:mzbuild-GLJTUDVZTGL74MOWQ3FI3ZUNDKCJJKIF
materialize/frontegg-mock:mzbuild-UMUNKJU2NC5MGPHT5JP7JLMVLDQPEAEB
materialize/jobs:mzbuild-MNJBKLM5ZHLU53FHPTBY32Q36Y7UESN2
materialize/kgen:mzbuild-525XB6KRVIG3XURRX443CGHUQSZBIE7Z
materialize/maelstrom-persist-base:mzbuild-NUGWOFCBNZI2NYE5N6D6XMQYPHIZIRDC
materialize/maelstrom-persist:mzbuild-IZCJLXQNHBLBRF7XHYZLC2MZOR7R63X4
materialize/materialized-base:mzbuild-6KUIUKIMFXYYIFIEHCLJEMQTG25XRAHK
materialize/materialized:mzbuild-RIJCRE3J3RBG2S346BIBJZ2PQORJ5OGK
materialize/test-certs:mzbuild-4X7BSRXRRUJQJ2BDOTRBCML4IFZKFT5B
materialize/mysql:mzbuild-3CTOQTRNEZ2WGT3Z6G5AF24W6P54HQBL
materialize/mysql-client:mzbuild-GIRUSXDV2UGNOSJOIOGWHGZDSLFOFD5G
materialize/mz:mzbuild-4G7GXP2ITXLG5HWFQBAI5HDVVSDO523H
materialize/orchestratord:mzbuild-BFB3BYXFGIYYLB6UJQ5MDTQLZ2W66J5L
materialize/postgres:mzbuild-6UIZ6QCFEVDPTZS5MIIIPL5AHVQ2O7TI
materialize/psql:mzbuild-5ENPLRAVYVOA3F77VITI4TXI55SLUEO3
materialize/sqlancer:mzbuild-GMHU6QUASFPVVAZTSW4POUVKE4IXDPGG
materialize/sqllogictest:mzbuild-QARFEXVAH5CASGAMQNZFRFYXBIHX2I7H
materialize/sqlsmith:mzbuild-FATDMKSEQGHTZWSNRHJGMHDJZBFFFP2W
materialize/testdrive-base:mzbuild-ZYPHOO5YEMZIULYBNWTVT662433VFNZ2
materialize/testdrive:mzbuild-UURHAAK7F5KFD6B7OCYP5EKSCPVYINOG
Scalability benchmark (read & write) against merge base or 'latest'
Waited 34m 22s
Ran in 1h 26m
Long multi-node Maelstrom coverage of persist with postgres consensus
Waited 11m 56s
Ran in 7m 5s
Total Job Run Time: 4d 12h