Cloud Pub/Sub Subscription Disappeared
Make sure subscription expiration is managed correctly
Did you know Cloud Pub/Sub subscriptions by default can disappear if you have some "quiet" subscriptions?
I came across this again today and was reminded that many are still unaware of the default behaviour of Cloud Pub/Sub subscriptions—they automatically expire after 31 days of inactivity. It seems to catch a lot of users off guard, especially those with less active subscriptions, highlighting the importance of being mindful of this default setting when designing systems.
I have encountered this issue multiple times over the years, where a subscription's expiration is only discovered during a major incident. This makes it extremely challenging and problematic to debug, as it’s rarely intuitive to consider the sudden disappearance of a Pub/Sub subscription as the root cause, especially since one wouldn’t typically expect it to vanish on its own.
I'll outline a few scenarios below to help you determine whether it might be worthwhile to set some or all of your subscriptions to "never expire."
Dead Letter Queue (DLQ)
It's common to have a DLQ subscription that sends messages to a BigQuery table for debugging and analysing why they were routed there.
BigQuery makes it much easier to investigate these issues compared to Pub/Sub. If your pipeline is stable, you might rarely see messages in this queue, but when they do occur, you could suddenly find nothing in BigQuery, leaving you puzzled and trying to figure out what went wrong.
Unpredictable Batch File Drop Event
Sometimes, workloads are triggered by Cloud Storage file landing events, which usually generate messages in a Pub/Sub topic. This approach is often chosen because the timing of upstream systems dropping files can be unpredictable, with some periods potentially seeing no activity for a month or longer.
To avoid data loss, it's crucial to configure these idle subscriptions to never expire. This prevents silent failures that could compromise data integrity, which can be notoriously difficult to debug.
Why the 31 day default?
I have often wondered why the default setting for a Pub/Sub subscription isn’t “never expire”.
I suspect the reasoning is to help tidy things up, particularly as some subscriptions, like those created automatically by Google Cloud when setting up Eventarc triggers, may not need to persist indefinitely. While this makes sense to some extent, it highlights the importance of carefully reviewing and configuring expiration settings.
If a Pub/Sub subscription is not something you consider “disposable without causing problems,” you should always configure it to never expire.
Need to modernise your data stack? I specialise in Google Cloud solutions, including migrating your analytics workloads into BigQuery, optimising performance, and tailoring solutions to fit your business needs. With deep expertise in the Google Cloud ecosystem, I’ll help you unlock the full potential of your data. Curious about my work? Check out My Work to see the impact I’ve made. Let’s chat! Book a call at Calendly or email richardhe@fundamenta.co. 🚀📊