B2B SaaS Teams Running Dozens of Clusters: Taming Messaging and Operations at Scale

The deal closes on a Thursday afternoon. Sales rings the bell in the shared channel: the company has signed its 60th dedicated-instance customer, a large enterprise with strict data-residency rules. Then the platform lead opens the contract appendix and finds the catch. The customer requires Google Cloud. Every one of the other 59 dedicated instances, plus the multi-tenant regions, runs on Azure.

By Friday morning the platform team has an estimate. A dozen services talk to Azure Service Bus through the vendor SDK, with retries and dead-lettering written against it. Swapping in Google Cloud Pub/Sub means touching each one and forking the deployment pipeline. The estimate is three weeks, before anyone asks how the new cluster will be monitored or upgraded.

If you run platform or SRE for a B2B SaaS company with single-tenant or per-region deployments, this probably feels familiar. Each new enterprise customer or region brings revenue, and another cluster to carry.

Why the cluster count turns into an operations problem

Running many clusters is hard because every cluster is a place where small differences accumulate. The root causes repeat across companies:

• Infrastructure leaks into application code. When services import a specific broker or database SDK, the choice of cloud becomes a code-level decision. A customer who needs a different cloud forces a code change, not a configuration change.
• Configuration is copied, not declared. Early clusters were set up with scripts later edited per customer. Nobody can say what the “standard” cluster looks like anymore.
• Runtime versions drift. Upgrades get scheduled cluster by cluster around customer change windows. Soon you are supporting several versions of the same platform at once.
• Certificates expire on their own schedule. Each cluster has its own trust chain and its own expiry dates. One missed rotation can take down service-to-service traffic in a single customer’s environment.
• Observability stops at the cluster boundary. Each cluster has its own dashboards, but there is no fleet-wide view. An incident in cluster 47 is invisible to the on-call engineer until the customer’s support ticket arrives.

Together they explain why upgrades take weeks and each new customer adds more weight than the last.

Why the usual workarounds fall short

Most teams try a few things before rethinking the approach.

Writing an internal messaging wrapper. A thin library hiding the broker SDK sounds sensible. In practice it must be maintained in every language you use, exposes the lowest common denominator of features, and its retry semantics quietly diverge between implementations.

Forking per cloud. An “Azure build” and a “GCP build” work until a third arrives. Every fix must be ported, and testing multiplies.

More runbooks and more people. A runbook that says “repeat on each cluster” scales linearly with cluster count, and people eventually skip a step.

Per-cluster monitoring with shared alerting. Forwarding every cluster’s alerts into one channel creates noise, not visibility.

Each workaround treats a symptom. The real problem is that nothing is standardized across the fleet.

The general approach: standardize the runtime, declare everything, watch the fleet

A more durable pattern rests on a few vendor-neutral principles.

1. Separate application logic from infrastructure choices. Application code should ask for “publish this event” or “save this state,” not “call this cloud’s SDK.” The binding to a specific broker or store belongs in configuration that platform teams own.
2. Declare every cluster’s desired state in Git. Infrastructure components, runtime settings and policy should live in version control and be applied by a GitOps controller. A new cluster becomes a new directory or overlay, not a new set of manual steps.
3. Treat upgrades and certificate rotation as automated, continuous processes. If an upgrade is a project, it will be postponed. If it is a pipeline with staged rollout and health checks, it happens regularly.
4. Build a fleet-wide view on consistent signals. Every cluster should emit the same metrics with the same labels, and traces should carry context across services, so you can compare cluster 12 and cluster 47 side by side.

Implementing it with a portable application runtime

Dapr, the Distributed Application Runtime, is a CNCF graduated, Apache 2.0 project that fits the first principle well. It runs as a sidecar next to each application and exposes building blocks over a consistent API: pub/sub, state management, service invocation, secrets, configuration, workflows and more.

Messaging and state without cloud lock-in

With Dapr pub/sub, a service publishes to a named component. Whether that component is backed by Kafka, RabbitMQ, Redis or a cloud provider’s messaging service is defined in a component file, not in the code. Messages use the CloudEvents format, delivery is at-least-once and dead-letter topics are supported, so the semantics you depend on stay consistent across brokers.

State management works the same way, with pluggable stores such as PostgreSQL, Redis, Cosmos DB, DynamoDB or MongoDB.

With this model, the 60th customer’s Google Cloud requirement becomes a platform task: write component definitions for that environment, validate them, and deploy the same images. That is configuration and testing, not a three-week code change. You should still verify behavior per broker in your own environment, since delivery guarantees and limits differ between messaging systems.

For example, a B2B data-catalog SaaS company processed 25+ million messages a month on 130 production clusters within two months of launching on Dapr.

GitOps for components and control-plane settings

Dapr components are Kubernetes resources, so they fit a GitOps workflow. A practical layout uses overlays:

• A base layer with resiliency policies (timeouts, retries, circuit breakers defined in YAML), access-control policies and common configuration.
• A cloud layer that selects the broker and state store components for Azure, Google Cloud or AWS.
• A thin customer layer for anything truly unique, such as data-residency settings.

The Dapr control plane can run in high-availability mode with three replicas, and its Helm values belong in the same repository. When every cluster is reconciled from Git, drift becomes a diff you can review rather than a mystery you discover during an incident.

Security and certificates across clusters

Dapr’s Sentry service acts as a certificate authority, issuing SPIFFE-based workload identities, and mTLS between sidecars is on by default. You can bring your own root certificates, which matters when an enterprise customer’s security team wants a say in the trust chain. Access-control policies restrict which app IDs can call which operations.

The catch is that certificates still expire on every cluster. Rotation must be tracked and automated fleet-wide, or mTLS causes the outage it was meant to prevent.

Fleet-wide observability and upgrades

Every Dapr sidecar and control-plane component exposes Prometheus metrics, and Dapr propagates W3C trace context across service invocation and pub/sub using OpenTelemetry. That consistency makes a fleet view possible: the same metrics on all clusters, aggregated centrally, so a spike in pub/sub failures on one customer’s cluster stands out before the customer calls.

You can build this layer yourself with a central metrics backend, GitOps-driven upgrades and certificate scripts, or use a managed tool. One option is the free Dapr Ops Dashboard from Diagrid, a company started by the team that created Dapr. It offers multi-cluster management, automated upgrades, certificate rotation and config sync, plus an advisor with 50+ best-practice checks, 150+ metrics and 20+ prebuilt dashboards. As with any vendor tool, evaluate it against your own clusters and compliance requirements before relying on it.

See also: How Is Technology Helping Businesses Reduce Their Carbon Footprint?

A roadmap for your fleet

Most teams cannot stop and re-platform everything. A staged approach works better.

1. Inventory the fleet. List every cluster’s cloud, runtime versions, certificate expiry and broker. The gaps become your first backlog.
2. Pick one messaging path to abstract. Choose a high-traffic but well-understood event flow and move it to a runtime-level pub/sub API. Prove that the same image runs on two clouds.
3. Move components into Git. Create the base, cloud and customer overlays, and let a GitOps controller apply them. Disallow manual changes to those resources.
4. Standardize metrics and tracing. Agree on labels such as cluster, customer and region, then aggregate them centrally.
5. Automate upgrades in rings. Upgrade internal and low-risk clusters first, watch health signals, then promote to the rest of the fleet.
6. Automate certificate rotation. Track expiry fleet-wide and rotate on a schedule, not in response to an alert.

And a short checklist to review each quarter:

• ☐  Every cluster is reconciled from Git, with no unexplained drift.
• ☐  No cluster is more than one runtime version behind.
• ☐  Certificate expiry is visible for all clusters in one view.
• ☐  Application images are identical across clouds; only configuration differs.
• ☐  On-call can see a single customer’s cluster health without logging in to it.
• ☐  A new customer cluster can be created from templates in days, not weeks.

Conclusion

Every new dedicated instance should be growth, not operational debt. The fix is discipline more than any single tool: keep cloud choices out of application code, declare cluster state in Git, automate upgrades and certificates, and watch the fleet as one system. Open-source runtimes such as Dapr give you the portability layer, and Dapr operations tooling can help with the day-two work. When the next customer asks for a different cloud, the answer should be a configuration change and a test run, not a three-week estimate.