1. Resources
  2. /
  3. Plugins
  4. /
  5. bigquery-buildkite-plugin

Actions Status Actions Status Version Plugin Status

BigQuery Buildkite Plugin

This Buildkite plugin can be used to deploy tables/views schemas to BigQuery

A Buildkite plugin for deploying tables/views schemas to BigQuery.

Using the plugin

If the version number is not provided then the most recent version of the plugin will be used. Do not use version number as master or any branch names.

Simple

steps:
  - plugins:
      - wayfair-incubator/bigquery#v1.2.2:
          gcp_project: gcp-us-project
          dataset_schema_directory: schemas/gcp-us-project/dataset

Configuration

Required

gcp_project (required, string)

The full name of the GCP project you want to deploy.

Example: gcp-us-project

dataset_schema_directory (required, string)

The directory in your repository where are you storing the schemas for your tables and views.

Example: gcp-us-project/dataset_name

prod_build_branch (optional, string)

Default branch for the repo.

Example: main

Default: master

execute_only_changed_files (optional, boolean)

Whether to execute just the files that changed in a directory or not.

Example: true

Default: true

fail_pipeline_on_first_exception (optional, boolean)

Whether to fail pipeline as soon as there is one failure.

Example: true

Default: true

Secret

This plugin expects GCP_SERVICE_ACCOUNT is placed as environment variable. Make sure to store it securely!

env:
  gcp_service_account: '{"email": ""}'

Example

Basic

The following pipeline will deploy all the schemas to the gcp-us-project living under directory/project/

Schemas

This plugin uses GBQ to deploy to Google BigQuery. GBQ now supports specifying partitions with the schema as well.

To leverage this you need to nest your JSON table schema in a dictionary. An example for the same is given below. Library supports Time and Range based partitioning along with Clustering.

All the configuration options can be found here.

{
  "partition": {
    "type": "range",
    "definition": {
      "field": "ID",
      "range": {
        "start": 1,
        "end": 100000,
        "interval": 10
      }
    }
  },
  "clustering": [
    "ID"
  ],
  "schema": [
    {
      "name": "ID",
      "type": "INTEGER",
      "mode": "REQUIRED"
    }
  ]
}

Contributing

See the Contributing Guide for additional information.

To execute tests locally (requires that docker and docker-compose are installed):

docker-compose run test

Credits

This plugin was originally written by Jash Parekh for Wayfair.

The plugins listed on this webpage are provided for informational purposes only. They have not undergone any formal security review or assessment. While we strive to provide useful resources, we cannot guarantee the safety, reliability, or integrity of these plugins. Users are strongly advised to conduct their own security evaluations before downloading, installing, or using any plugin. By using these plugins, you acknowledge and accept any risks associated with their use. We disclaim any liability for any harm or damages arising from the use of the plugins listed.

Start turning complexity into an advantage

Create an account to get started with a 30-day free trial. No credit card required.

Buildkite Pipelines

Platform

  1. Pipelines
  2. Pipeline templates
  3. Public pipelines
  4. Test Engine
  5. Package Registries
  6. Mobile Delivery Cloud
  7. Pricing

Hosting options

  1. Self-hosted agents
  2. Mac hosted agents
  3. Linux hosted agents

Resources

  1. Docs
  2. Blog
  3. Changelog
  4. Webinars
  5. Plugins
  6. Case studies
  7. Events
  8. Migration Services
  9. Comparisons

Company

  1. About
  2. Careers
  3. Press
  4. Brand assets
  5. Contact

Solutions

  1. Replace Jenkins
  2. Workflows for AI/ML
  3. Testing at scale
  4. Monorepo mojo
  5. Bazel orchestration

Legal

  1. Terms of Service
  2. Acceptable Use Policy
  3. Privacy Policy
  4. Subprocessors
  5. Service Level Agreement

Support

  1. System status
  2. Forum
© Buildkite Pty Ltd 2025