Cloud Storage Lifecycle Management, in depth: rule semantics, versioned buckets, Custom-Time retention and the traps that cost money

The model: rules, actions and conditions

A lifecycle configuration is a list of rules. Each rule has exactly one action and one or more conditions. There are three actions. Delete removes the object. SetStorageClass changes the object's storage class, for example Standard to Nearline. AbortIncompleteMultipartUpload cancels an XML API multipart upload that was started but never completed, and removes its uploaded parts.

Two combination rules govern everything. Within a rule, conditions are ANDed: an object must satisfy every condition in the rule. Across rules, matches are ORed: if an object matches any rule, that rule's action applies. So to express "delete temporary files after 7 days or logs after 400 days" you write two rules, and to express "logs older than 400 days" you put two conditions in one rule.

When several rules with different actions match the same object at once, Delete takes precedence over SetStorageClass. When several SetStorageClass rules match, the one targeting the class with the lowest at-rest storage price wins. Evaluation is asynchronous: Cloud Storage does not act the instant a condition becomes true, and a change to a bucket's lifecycle configuration can take up to 24 hours to take effect.

Every condition, and what it actually measures

The most common misunderstanding is age. It counts from object creation, and an object is created again when it is overwritten. A file your pipeline rewrites every day will never reach age: 30. A lifecycle SetStorageClass action, by contrast, does not reset the clock that age measures, which is why chains such as Nearline at 30 days then Coldline at 120 days work. AbortIncompleteMultipartUpload accepts only age, matchesPrefix and matchesSuffix.

Storage class transitions and their costs

Lifecycle can only move objects toward colder classes: from Standard (and the legacy Multi-Regional and Regional classes) to Nearline, Coldline or Archive; from Nearline to Coldline or Archive; and from Coldline to Archive. It never moves data back to Standard; that requires rewriting the object yourself or using Autoclass.

Each transition is billed as a Class A operation per object, and the colder classes carry minimum storage durations: 30 days for Nearline, 90 for Coldline and 365 for Archive. Deleting an object before its minimum duration is charged as if it had stayed for the full period. A lifecycle class change is done in place, and Google documents that time spent in the earlier class counts toward the new class's minimum, so the clock effectively runs from creation; a rewrite into a new class, by contrast, replaces the object and can trigger early deletion on the old one. Two consequences follow. A rule that moves objects to Archive at day 30 and another that deletes them at day 100 pays for Archive storage up to day 365 on every object. And transitioning millions of small objects can cost more in operations than it saves in storage, which is exactly what sizeAboveBytes is for. The cost arithmetic is worked through in the classes article; check current prices on the pricing page before committing to a design.

Worked example: a log bucket

An application writes roughly 2 million log objects a day into gs://acme-app-logs/logs/, mostly between 50 KB and 5 MB, and a batch job drops scratch files under tmp/. Logs are queried heavily for a week, occasionally for three months and almost never after that; compliance needs 13 months. Large uploads use the XML multipart API and occasionally die halfway.

{
  "lifecycle": {
    "rule": [
      {
        "action": {"type": "SetStorageClass", "storageClass": "NEARLINE"},
        "condition": {"age": 30, "matchesPrefix": ["logs/"],
                      "matchesStorageClass": ["STANDARD"],
                      "sizeAboveBytes": 1048576}
      },
      {
        "action": {"type": "SetStorageClass", "storageClass": "COLDLINE"},
        "condition": {"age": 120, "matchesPrefix": ["logs/"],
                      "matchesStorageClass": ["NEARLINE"]}
      },
      {
        "action": {"type": "Delete"},
        "condition": {"age": 400, "matchesPrefix": ["logs/"]}
      },
      {
        "action": {"type": "Delete"},
        "condition": {"age": 7, "matchesPrefix": ["tmp/"]}
      },
      {
        "action": {"type": "AbortIncompleteMultipartUpload"},
        "condition": {"age": 3}
      }
    ]
  }
}

Read it rule by rule. Rule one moves logs larger than 1 MiB to Nearline after 30 days, and only if they are still Standard; small objects stay in Standard because the per-object transition cost and Nearline retrieval charges outweigh the saving. Rule two moves Nearline logs to Coldline at 120 days, when reads have become rare. Rule three deletes all logs at 400 days, beyond the 13-month requirement and well past the 90-day Coldline minimum, so no early-deletion charge applies. Rule four clears scratch files after a week. Rule five aborts abandoned multipart uploads whose parts would otherwise be billed indefinitely. Apply and inspect it with gcloud:

# apply, inspect and remove a lifecycle configuration
gcloud storage buckets update gs://acme-app-logs --lifecycle-file=lifecycle.json
gcloud storage buckets describe gs://acme-app-logs --format="default(lifecycle_config)"
gcloud storage buckets update gs://acme-app-logs --clear-lifecycle

Expect nothing to happen for up to a day after applying, and then expect actions to trickle rather than land at once: lifecycle is a background process, not a scheduled job with a fixed run time.

Versioned buckets and soft delete

With Object Versioning enabled, a delete, including a lifecycle Delete on a live object, does not remove data; it makes the live version noncurrent. That means a simple age-based delete rule on a versioned bucket only converts live objects into noncurrent ones, and storage usage does not fall. You need rules that target noncurrent versions explicitly:

{
  "lifecycle": {
    "rule": [
      {"action": {"type": "Delete"},
       "condition": {"isLive": false, "numNewerVersions": 3}},
      {"action": {"type": "Delete"},
       "condition": {"isLive": false, "daysSinceNoncurrentTime": 30}}
    ]
  }
}

The first rule keeps at most three noncurrent versions of each object; the second removes any noncurrent version 30 days after it was replaced. Because rules OR together, a version is deleted when either is true. Note that daysSinceNoncurrentTime counts from when the version was superseded, not from its creation, which is what you want for "keep undo history for 30 days".

Soft delete adds another layer. When it is enabled, which it is by default with a 7-day retention, objects deleted by any means, lifecycle included, remain restorable for the soft delete window and are billed for storage during it. That is a valuable safety net against a bad rule, but it also means a big lifecycle cleanup reduces your bill only after the window passes. On buckets with heavy churn of short-lived objects, weigh the protection against its cost and set the window deliberately.

Retention from business dates with Custom-Time

Creation time is often the wrong clock. Invoices backfilled in a migration were all created last Tuesday, yet each must be retained for seven years from its invoice date. The Custom-Time metadata field solves this: set it to the business date, then use daysSinceCustomTime or customTimeBefore.

from datetime import datetime, timezone
from google.cloud import storage

client = storage.Client()
bucket = client.bucket("acme-invoices")

# Retention should run from the invoice date, not from when the file was uploaded.
blob = bucket.blob("2025/INV-88121.pdf")
blob.reload()
blob.custom_time = datetime(2025, 3, 14, tzinfo=timezone.utc)
blob.patch()

# Rule: delete seven years (2,557 days) after the business date.
bucket.reload()
bucket.add_lifecycle_delete_rule(days_since_custom_time=2557)
bucket.patch()

Custom-Time can be set when the object is written or patched later, and it can only move forward once set, so write it once from an authoritative source. Objects without Custom-Time never match these conditions, which is safe for deletion but means a missing metadata field silently exempts an object. Add a periodic check that counts objects lacking it.

Holds, retention policies and Autoclass

Lifecycle never overrides data protection. An object under an event-based or temporary hold, or one that has not yet satisfied the bucket's retention policy, is not deleted by a Delete rule; the action is effectively skipped until the protection lifts. This is the right way to combine a legal requirement (retention policy, possibly locked) with cleanup (lifecycle): the retention policy sets the floor and lifecycle deletes after it.

Autoclass and lifecycle overlap. On an Autoclass bucket you cannot use SetStorageClass actions or the matchesStorageClass condition, because the service owns class placement. Delete and AbortIncompleteMultipartUpload rules still work, so the usual pattern is Autoclass for tiering plus lifecycle for expiry.

Operating lifecycle policies safely

• Manage configuration as code. Keep the JSON (or Terraform lifecycle_rule blocks) in version control with review; gcloud storage buckets update --lifecycle-file replaces the whole configuration, so a partial file drops the rules you forgot.
• Test on a copy first. Apply the policy to a staging bucket with representative names, sizes and versions, and wait more than 24 hours before judging it.
• Dry-run with listings. Before enabling a delete rule, list and count what would match (by prefix, creation time and size) and compare it with expectations.
• Verify after the fact with bucket inventory or periodic listings: storage bytes by class and prefix should move in the shape the policy predicts.
• Prefer narrow rules. Always scope deletes with matchesPrefix unless the entire bucket is truly disposable.
• Keep soft delete on for buckets where a bad rule would hurt, at least while a new policy beds in.

Failure modes

• Delete rule on a versioned bucket, no noncurrent rule. Usage never drops; noncurrent versions accumulate forever.
• Deleting soon after a cold transition. Archive at 30 days and delete at 100 pays Archive storage to day 365 on every object.
• Small-object transitions. Operation charges exceed storage savings; filter with sizeAboveBytes.
• Rewritten objects never age. Overwrite resets creation time; use Custom-Time or stop overwriting.
• Unscoped delete. A rule without a prefix deletes everything older than N days, including the bucket's reference data.
• Expecting immediacy. Teams apply a rule, see nothing for hours, and apply a second, harsher one. Lifecycle is eventual.
• SetStorageClass on an Autoclass bucket. The configuration is rejected; move tiering decisions to one mechanism.

Lifecycle, Autoclass or your own job

For the internals underneath all three, such as how metadata and strong consistency work, see Google Cloud Storage internals.

What to do next

1. Export the current lifecycle configuration of every bucket and review it for unscoped deletes and missing noncurrent-version rules.
2. For each large bucket, write down the real access pattern by prefix and translate it into age-based transitions whose delete ages clear the 30, 90 and 365 day minimums.
3. Add an AbortIncompleteMultipartUpload rule wherever the XML multipart API is used.
4. Filter transitions with sizeAboveBytes after estimating per-object costs for your size distribution.
5. Move date-driven retention to Custom-Time and monitor objects missing it.
6. Put lifecycle JSON under version control and apply it from CI after a staging test.
7. Confirm the soft delete window on each bucket is a deliberate choice.