GCP Cloud Build — Managed CI/CD

The execution model

A build is a list of steps. Each step names an image, optionally overrides the entrypoint, and passes arguments. The platform pulls the image, starts a container, runs it to completion, and moves on. A non-zero exit fails the build.

The connective tissue is /workspace: a volume mounted into every step at the same path and persisted for the life of the build. Your source is checked out there before the first step, and anything a step writes there is visible to later steps. That is the entire data-passing mechanism -- compile in one container, test in another, package in a third, and the artifacts move between them through the filesystem. State written anywhere else in a step's container is discarded when the step ends, which is a frequent source of confusion when a step installs a tool globally and the next step cannot find it.

Steps run serially by default. Adding waitFor turns the list into a dependency graph: waitFor: ['-'] means 'start immediately, depend on nothing', and naming step IDs means 'start once these have finished'. Two independent test suites both marked to start immediately run concurrently; a packaging step listing both IDs waits for them. This is a genuine DAG, and on builds with several independent branches it is the cheapest available speedup.

Reading and writing a build config

The config is YAML -- cloudbuild.yaml by convention -- and the shape is small enough to hold in your head.

steps:
  - name: 'gcr.io/cloud-builders/npm'
    args: ['ci']
  - id: 'unit'
    name: 'gcr.io/cloud-builders/npm'
    args: ['test']
  - id: 'lint'
    name: 'gcr.io/cloud-builders/npm'
    args: ['run', 'lint']
    waitFor: ['-']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t',
           'us-docker.pkg.dev/$PROJECT_ID/app/api:$SHORT_SHA', '.']
    waitFor: ['unit', 'lint']
images:
  - 'us-docker.pkg.dev/$PROJECT_ID/app/api:$SHORT_SHA'
options:
  machineType: 'E2_HIGHCPU_8'
  logging: CLOUD_LOGGING_ONLY
timeout: '1800s'

Per-step fields worth knowing: dir sets the working directory relative to the workspace, which is how monorepos target a subproject; env and secretEnv inject environment variables; entrypoint replaces the image's default, which is what lets you run bash -c inside an image that normally runs a tool directly; and allowFailure permits a step to fail without failing the build, useful for advisory scans.

The top-level images list is a convenience: images named there are pushed to the registry after all steps succeed, and their digests are recorded in the build result. artifacts does the same for files copied to Cloud Storage. Pushing inside a step works too, but you lose that automatic recording -- and the recorded digest is what downstream provenance and deployment tooling reads.

Triggers — what starts a build

A trigger binds a source event to a build config. The common cases are a push to matching branches, a pull request, and a tag matching a pattern; there are also manual triggers, webhook triggers for arbitrary external systems, and Pub/Sub triggers that let any Google Cloud event -- a new image in Artifact Registry, a message from another system -- start a build.

Two filters on a trigger matter more than their prominence suggests. Included files restricts the trigger to fire only when matching paths changed, and ignored files is the inverse. In a monorepo these are what stop every service rebuilding because someone edited a README, and they are the difference between a five-minute and a fifty-minute feedback loop.

Repository connections come in two generations, and the distinction shows up in documentation and error messages. The older model mirrors a GitHub or Bitbucket repository into Cloud Source Repositories; the newer one connects directly through a host connection with credentials in Secret Manager, supports more providers including self-hosted instances, and is the path forward. Existing mirrored setups keep working, but new connections should use the current model -- particularly if you need pull request triggers from a provider the mirror path does not cover.

Each trigger carries its own configuration: which config file, which service account, and which substitution values. That last one is how a single config file serves staging and production -- same YAML, different _ENVIRONMENT and _REGION per trigger.

Substitutions

Substitutions are the parameterisation mechanism. Built-in values are available automatically: $PROJECT_ID, $BUILD_ID, $COMMIT_SHA, $SHORT_SHA, $BRANCH_NAME, $TAG_NAME, $REPO_NAME, $LOCATION. $SHORT_SHA is the workhorse for image tagging, because it produces an immutable tag tied to a commit -- deploying :latest is the mistake this variable exists to prevent.

User-defined substitutions must begin with an underscore -- _SERVICE_NAME, _REGION -- which keeps them from colliding with future built-ins. Defaults go in the config; triggers override them; a manual build can override them again at submit time. With the dynamic-substitutions option enabled, values can reference other values and use bash-style parameter expansion, which covers most of the cases that otherwise drive people to generate YAML.

One rule with a security edge: substitutions are not secrets. They are stored with the build metadata and visible to anyone who can read build history. API keys, tokens and passwords belong in Secret Manager and reach the build through secretEnv, never through a substitution -- a mistake that is easy to make because passing a token as _API_KEY works perfectly and leaks permanently.

Identity — service accounts and the logging requirement

Builds run as a service account, and that account's permissions are the build's permissions. Historically a legacy Cloud Build service account existed per project with broad roles pre-granted, which made everything work immediately and gave every build in the project the ability to deploy anything. Current practice, and increasingly the default, is to specify a user-managed service account per trigger, granted exactly the roles that trigger needs.

Doing that correctly means granting three categories of permission and it is worth listing them, because a missing one produces an error at a confusing moment: the account needs permission to act as a builder and write logs; it needs whatever the build does -- push to Artifact Registry, deploy to Cloud Run or GKE, read a secret; and if it deploys as another identity, it needs to impersonate that identity.

There is one non-obvious requirement that trips nearly every first migration. When a build runs as a user-managed service account, you must specify a logging destination -- typically options.logging: CLOUD_LOGGING_ONLY, or a Cloud Storage bucket you own. Without it the build fails immediately with a message about logs buckets rather than about permissions, because the default log destination is a Google-managed bucket the custom account cannot write to. The fix is one line and the error message does not suggest it.

Scope accounts per pipeline rather than sharing one. The build service account is a high-value credential -- it can typically deploy to production -- and a compromised dependency in any build that uses it inherits everything it can do.

Secrets

Secret Manager is the supported path. Declare the secrets a build may use in an availableSecrets block, then expose them to individual steps through secretEnv:

availableSecrets:
  secretManager:
    - versionName: projects/$PROJECT_ID/secrets/npm-token/versions/latest
      env: 'NPM_TOKEN'
steps:
  - name: 'gcr.io/cloud-builders/npm'
    entrypoint: 'bash'
    args: ['-c', 'echo "//registry.npmjs.org/:_authToken=$$NPM_TOKEN" > .npmrc \
                  && npm ci']
    secretEnv: ['NPM_TOKEN']

Note the doubled dollar sign. A single $ is consumed by the substitution engine before the shell ever sees it, so a secret referenced as $NPM_TOKEN arrives empty and the failure looks like a Secret Manager problem rather than a quoting one.

Secrets are injected only into steps that list them, so a compromised third-party step image cannot read a token it was not given. Pinning secret versions rather than latest makes rotation deliberate and makes an old build reproducible; pinning to latest is convenient and means a rotation silently changes what a rebuild of an old commit does.

Caching, and why builds are slower than expected

The default execution environment is fresh for every build. There is no persistent layer cache, so a Docker build starts with nothing and a dependency install downloads everything. This is the single largest source of 'why does this take twelve minutes' and it has three standard answers.

Pull the previous image and build with it as a cache source. Fetch the last published tag, tolerate failure on first run, then pass --cache-from. Effective when the Dockerfile is ordered so that dependency installation precedes source copying -- if it is not, no caching strategy will help, and reordering the Dockerfile is the higher-value fix.

Cache dependency directories in Cloud Storage. Restore a tarball of the package cache at the start, save it at the end. Crude, and for ecosystems with large dependency trees it is often the biggest single win available.

Use a builder with its own caching. Kaniko caches layers in a registry; Buildpacks and BuildKit-based tooling maintain their own caches. These generally beat hand-rolled approaches and require less config.

Two further levers: raise machineType for compilation-heavy builds -- the higher-CPU machines cost more per minute and frequently cost less per build -- and parallelise independent steps with waitFor, which is free. Measure before and after; build time intuitions are usually wrong about which step dominates.

Private pools and network access

By default builds execute in a Google-managed environment with no route into your private networks and no stable egress address. That is fine until a build must reach a private GKE control plane, a Cloud SQL instance without a public endpoint, an internal artifact repository, or an on-premises system behind an allow-list.

Private pools solve this. A private pool runs build workers in a Google-managed project peered to your VPC, so builds can reach private addresses and egress through a predictable path. You choose machine type, disk size and concurrency for the pool, which also makes it the answer for builds that need more resources or more determinism than the default environment offers.

The trade is cost and management: a private pool has its own pricing and its own networking to get right -- peering ranges, firewall rules, and DNS resolution for internal names. Use the default pool wherever it suffices, and reach for a private pool when connectivity, not performance, forces it. Regional placement matters too: run the pool in the region holding the resources it talks to, both for latency and because data residency requirements usually apply to build workers as much as to production.

Provenance, scanning and deployment gates

A managed builder can attest to what it built, and Cloud Build does. Builds that produce container images can generate signed build provenance recording the source commit, the build steps and the resulting digest -- the SLSA-style record that lets a downstream system verify an image came from your pipeline rather than from a laptop.

That record becomes enforceable through Binary Authorization: a policy on a Cloud Run service or GKE cluster that refuses to run images lacking a valid attestation. The combination is the practical answer to supply-chain requirements -- images may only reach production if they were built by the pipeline, from the reviewed repository, and passed the gates.

Artifact Registry adds vulnerability scanning on push, and a build step can query the results and fail on severity thresholds. Be deliberate about where that gate sits: failing a build on a newly disclosed critical vulnerability in a base image is correct for a release pipeline and infuriating in a pull-request pipeline, where the developer did not introduce it and cannot fix it. Advisory in one place, blocking in the other.

For anything beyond a single deployment target, hand off to Cloud Deploy rather than scripting promotion in build steps. It models environments, promotion between them, approvals and rollback as first-class objects, and it keeps the build config focused on producing an artifact instead of on orchestrating a release.

Iterating without pushing a commit

The slowest possible way to develop a pipeline is to edit the config, commit, push, wait for the trigger, read the log, and repeat. Two mechanisms remove that loop.

gcloud builds submit uploads the current working directory and runs a build against it immediately, with no commit and no trigger. Add --substitutions to supply the values a trigger would normally provide, and --config to point at a candidate file. This is the right way to develop a build config, and it is also how one-off operational builds should be run.

Debugging a failing step is harder than debugging a failing script because the environment disappears with the container. Three techniques cover most cases. Insert a diagnostic step that prints the workspace contents, the environment and the tool versions -- cheap, and it resolves most 'works locally' failures immediately, which are usually a different image version or a missing file that was gitignored. Run the failing step's image locally with the same arguments and a copy of the source mounted at /workspace; because steps are just containers, this reproduces the environment more faithfully than any CI system that runs scripts on a shared runner. And for a build that only fails under the trigger, compare the substitutions -- the difference between a manual submit and a triggered build is almost always a value that is present in one and empty in the other, with $COMMIT_SHA and $BRANCH_NAME the usual suspects, since a directory upload has no commit attached.

Monorepo patterns

A repository holding a dozen services stresses two things: which builds run, and how much they share. The path filters on a trigger handle the first. One trigger per service, each with an included-files pattern covering that service's directory plus any shared library it depends on, means an edit to one service builds one service. Getting the shared-library patterns right is the fiddly part -- omit them and a change to common code ships nothing, include them too broadly and every change rebuilds everything.

The second question is whether to keep one config per service or one parameterised config for all of them. Per-service configs are clearer and drift apart. A single config driven by _SERVICE and dir keeps the pipeline uniform and makes a change to the build process apply everywhere at once, at the cost of conditionals for the services that are genuinely different. The uniform approach ages better in practice, provided you accept that the one or two unusual services get their own config rather than another branch in the shared one.

For deployment ordering across services, resist encoding a cross-service graph inside one giant build. Independent builds producing independently versioned images, with promotion handled by a delivery tool, degrade far better than a single build whose failure at step nineteen leaves half a system deployed.

Operational details that surprise people

The default timeout is ten minutes. Not the twenty-four-hour maximum -- ten minutes, for the whole build. The first substantial pipeline anyone writes hits it, and the failure reads as a hang rather than as a limit. Set timeout explicitly in every config, and set per-step timeouts on steps that could hang indefinitely.

Concurrency is quota-limited. Projects have a ceiling on concurrent builds; past it, builds queue. On a busy repository with per-pull-request triggers this shows up as mysteriously slow feedback that has nothing to do with build duration. Check queue time as a distinct metric from build time, and request quota before it becomes the constraint.

Steps run privileged enough to matter. A step can access the Docker daemon and the build's service account credentials. Any third-party image you run as a step is code executing with those credentials -- pin images by digest rather than by tag for anything outside your control.

Builds are regional, and logs have a destination. Choose the build region deliberately for data residency and for proximity to the registry and deployment targets. Decide where logs land, since Cloud Logging and a private bucket have different retention, access control and cost profiles.

Know when it is not the right tool. Cloud Build is strongest when the workload is Google Cloud native -- building containers, deploying to Cloud Run or GKE, running Terraform against Google Cloud with a workload identity. It is thinner than GitHub Actions or GitLab CI on ecosystem: fewer prebuilt integrations, a smaller marketplace, no built-in matrix builds, and a less rich pull-request experience. Teams whose repositories, reviews and issue tracking already live elsewhere often keep CI there and use Cloud Build only for the parts that need to run inside the project's trust boundary -- which is a perfectly good architecture, not a compromise.