Platform & API Product Engineering · Lesson 05

Multi-tenancy & Isolation

Every SaaS API serves multiple customers on the same infrastructure. How you draw the lines between them determines your cost structure, blast radius, compliance posture, and the degree to which one customer can harm another. Getting those lines wrong is the kind of mistake you discover at 2 AM when a noisy tenant crushes a paying customer's throughput — or worse, when a missing WHERE clause returns the wrong company's data.

The three isolation models

Every multi-tenant system makes a fundamental choice about where to draw boundaries between tenants. There are three canonical positions, not a dozen — everything else is a variation or a marketing name for one of these.

The models differ on where the tenant boundary lives:

• SILO — each tenant has its own infrastructure stack: separate application instances, separate database, separate everything. The boundary is a physical network or account boundary. AWS runs this way: each AWS account is a hard isolation unit with separate resource namespaces.
• POOL — all tenants share the same infrastructure. The boundary is enforced by a tenant_id column on every row and a filter in every query. Salesforce pioneered this at scale — OrgId on every object, in every query, for hundreds of thousands of orgs.
• BRIDGE — the application layer is shared, but the data layer is configurable per tenant. Small tenants land in the pool; enterprise tenants with compliance requirements or sheer data volume get their own isolated data store, routed to transparently.

Pros and cons across five dimensions

Data isolation deep dive

The POOL model's entire security guarantee rests on one invariant: every query that touches tenant data includes a WHERE tenant_id = ? filter. Miss it once — in any query path, background job, report, or admin tool — and you have a cross-tenant data leak. In OWASP API Security terms, this is a BOLA (Broken Object-Level Authorization) vulnerability. See sec-05-authn-authz.html for the full BOLA taxonomy.

The unsafe query vs. the correct query

The canonical failure looks like this: a developer adds a "get invoice by ID" endpoint. They implement the authN check (valid JWT required) but forget to scope the DB query to the authenticated tenant.

-- ❌ UNSAFE: no tenant filter — any authenticated user can fetch any invoice
SELECT id, amount, status, customer_name
FROM   invoices
WHERE  id = $1;  -- $1 = invoice ID from request path

-- ✅ CORRECT: tenant_id bound at authentication layer, enforced at query layer
SELECT id, amount, status, customer_name
FROM   invoices
WHERE  id        = $1
AND    tenant_id = $2;  -- $2 = tenant_id extracted from validated JWT claim

The unsafe version authenticates the caller but does not authorize them for the specific object. Tenant B can brute-force or enumerate invoice IDs belonging to Tenant A. The correct version binds the query to the tenant extracted from the verified token — even if the caller guesses Tenant A's invoice ID, the query returns nothing.

Schema and database strategies

Beyond tenant_id on rows, platforms use three structural approaches, each trading isolation for cost:

• Shared schema, tenant_id column. All tenants share the same table. Cheapest, but every query must include the filter. The leak surface is large — any query path that doesn't enforce the filter is an exposure.
• Schema-per-tenant. Each tenant gets their own PostgreSQL schema (namespace) — tenant_acme.invoices vs. tenant_globex.invoices. Queries route to the right schema at connection setup via SET search_path TO tenant_acme. Reduces cross-contamination risk; enables per-tenant schema variations; still shares the underlying database cluster (and its failure domain).
• DB-per-tenant. Full physical isolation — each tenant has their own database instance or cluster. Maximum isolation; linear cost. Appropriate for regulated industries (healthcare, finance) where tenants may demand proof of separation.

PostgreSQL Row-Level Security (RLS)

RLS is a PostgreSQL feature that enforces the tenant filter at the database layer — the application cannot accidentally bypass it, because the DB itself rejects queries that violate the policy. This is defense in depth: even if a developer writes an unsafe query, the DB returns nothing rather than leaking data.

-- 1. Enable RLS on the table
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
ALTER TABLE invoices FORCE ROW LEVEL SECURITY;  -- applies even to the table owner

-- 2. Create the isolation policy
CREATE POLICY tenant_isolation ON invoices
  USING (tenant_id = current_setting('app.current_tenant')::uuid);

-- 3. Application code sets the tenant before any query in the transaction
SET LOCAL app.current_tenant = 'org_7f3a9c12';

-- Now this query returns only Tenant org_7f3a9c12's rows, enforced by the DB:
SELECT * FROM invoices;  -- implicitly filtered; no WHERE needed in application code

-- An unsafe application query — still safe with RLS active:
SELECT * FROM invoices WHERE id = $1;  -- RLS policy ANDs in tenant_id = current_setting(…)

Compute and throughput isolation

Data isolation prevents cross-tenant reads. Compute isolation prevents one tenant's traffic from degrading another's performance. In a pooled system, these are independent problems — you can have perfect data isolation and still have a noisy-neighbor problem that makes enterprise customers call you at midnight.

The noisy-neighbor problem

In a shared thread pool, a tenant running a bulk data export — thousands of sequential requests, each holding a DB connection and a thread — can exhaust the pool and starve every other tenant. The mechanism: each request acquires a thread from the shared pool. With a pool of 100 threads and a tenant firing 150 concurrent requests, the pool is full. All other tenants queue. Their P99 latency goes from 40ms to 4s. No one crossed a tenant boundary — the data is perfectly isolated — but the customer experience is broken.

Per-tenant rate limits and quotas

Rate limiting in a multi-tenant system is not just "protect the API from overload" — it is a fairness contract between tenants. The implementation follows the same algorithms as single-tenant rate limiting (token bucket, sliding window counter), but the key space is tenant:{tenant_id}:{endpoint} rather than ip:{ip}. Each tenant tier gets its own limit: free tier at 10 req/s, growth at 100 req/s, enterprise at 1,000 req/s with burst allowance. See also plat-01 (Rate Limits & Quotas) for the nested rate limit patterns that apply at the org, user, and API-key levels simultaneously.

Fair scheduling and weighted fair queuing

When multiple tenants share a processing queue — a job queue, a request queue, a DB connection pool — pure FIFO scheduling allows one heavy tenant to monopolize the queue. Weighted fair queuing (WFQ) assigns each tenant a weight proportional to their tier, then schedules requests so that each tenant receives approximately their share of processing capacity over any window, regardless of request arrival pattern. In practice: a tenant firing 1,000 requests into the queue receives their fair share (say 10% of throughput if they are a standard-tier tenant among ten equal tenants), but cannot consume more than their weight allows, even if other tenants are idle.

Tenant-scoped resource naming

URL structure is itself a form of access control and a documentation of scope. The /v1/orgs/{org_id}/... pattern makes the tenant scope explicit in the resource identifier — both the client and the server can see at a glance whose data is being addressed. It also makes access control tests obvious: if org_id in the path doesn't match tenant_id in the token, reject the request immediately in middleware, before it ever reaches the query layer.

-- Tenant-scoped URL namespace
GET  /v1/orgs/{org_id}/invoices
GET  /v1/orgs/{org_id}/invoices/{invoice_id}
POST /v1/orgs/{org_id}/invoices
GET  /v1/orgs/{org_id}/customers/{customer_id}/orders

-- Auth middleware: path org_id must match token claim
if params.org_id != token.claims.tenant_id:
    return 403  -- forbidden: path tenant doesn't match token

This pattern also makes test writing easy: security tests can assert that fetching /v1/orgs/org_A/invoices/{invoice_id_from_org_B} returns 403 or 404, not 200.

Per-tenant configuration

Encryption keys — envelope encryption

Storing all tenants' data encrypted with a single key means one key compromise exposes everyone. Per-tenant encryption isolates the blast radius. The standard pattern is envelope encryption:

1. Generate a Data Encryption Key (DEK) per tenant (or per table, per object — the granularity is a policy choice).
2. Encrypt the DEK with a Key Encryption Key (KEK) managed by a KMS (AWS KMS, GCP Cloud KMS, HashiCorp Vault). The wrapped DEK is stored with the data.
3. At query time: fetch the wrapped DEK, call KMS to unwrap it (one KMS API call, cacheable for the session), decrypt the data with the unwrapped DEK.
4. To revoke a tenant's access to their data: destroy their DEK in the KMS. The data becomes permanently inaccessible without needing to re-encrypt or delete rows.

The KEK never leaves the KMS. The DEK is in memory only during the decryption operation. This is how Stripe, AWS, and most regulated SaaS platforms handle tenant-level encryption.

Data residency

EU customers may be contractually or legally required to have their data stored in EU data centers. This adds a routing problem: the same API endpoint must direct EU tenants' reads and writes to the EU cluster and everyone else to the default cluster. The common implementation: a global routing layer holds a tenant-to-region mapping; on every request, after extracting tenant_id, it proxies to the appropriate regional cluster. This mapping itself must be available globally (typically a small, highly-replicated lookup table with aggressive caching). See rel-16-consistency-cap.html for the consistency considerations this introduces — the routing table and the data are in different systems with different replication lag.

Per-tenant feature flags

Feature flags in a multi-tenant system are scoped to tenants, not just code paths. This lets you: roll out a new API behavior to enterprise tenants first; run A/B experiments on a subset of tenants without affecting others; gate features behind plan tier; or disable a specific feature for a tenant who reported a bug. The implementation is a simple lookup: before executing any code path gated by a flag, resolve feature_flag(tenant_id, flag_name) — typically a Redis lookup with a fallback to a configuration database.

The cache cross-tenant leak

Caches are a frequent source of cross-tenant data leaks because the mistake is invisible in normal operation and only surfaces when two tenants happen to share a cache key collision. The mechanism:

1. Tenant A requests GET /v1/invoices/summary. The cache layer stores the response under key invoices:summary.
2. Tenant B requests the same endpoint. The cache hits on invoices:summary and returns Tenant A's data to Tenant B.

The fix is exact and non-negotiable: every cache key must include the tenant ID. The correct key is invoices:summary:{tenant_id} or {tenant_id}:invoices:summary. See rel-07-caching.html for cache invalidation patterns and the full cache key design framework.

Under the hood: the full BOLA walkthrough

Walk through the complete vulnerable-to-fixed progression, including how RLS catches what the application missed.

RLS is not a replacement for application-level enforcement — setting app.current_tenant correctly is itself application work that could be misconfigured. It is a defense-in-depth layer that prevents the worst case: a bug in application code that would otherwise return another tenant's data.

By the numbers

Blast radius in the POOL model. If each cluster serves 500 tenants and one cluster crashes, 500 tenants (5% of your total) are affected. With 20 clusters and no traffic isolation, an outage is bounded to 5% of tenants rather than 100% — but that is still 500 customers. Cluster-level sharding is a blast-radius reduction strategy: the more clusters, the smaller the fraction affected per incident, but the higher the operational overhead.

Break-even: when does SILO become operationally untenable? The rough formula:

-- Break-even point: where POOL ops overhead < SILO infra + ops overhead

SILO_total     = N * (infra_per_tenant + ops_per_tenant_per_month)
POOL_total     = (N / tenants_per_cluster) * (cluster_cost + ops_per_cluster_per_month)

-- Approximate: at N ≈ 50 tenants, SILO is viable if tenants are high-value.
-- At N ≈ 500+, the ops burden (patching, monitoring, incident response per tenant)
-- dominates. At N ≈ 5,000+, only POOL or BRIDGE is operationally sustainable.

-- The inflection point moves with automation level:
-- fully automated provisioning + IaC pushes SILO viability to ~200–500 tenants.

How real platforms do it

Sources & further reading

• AWS Whitepaper — SaaS Tenant Isolation Strategies
• Salesforce Developer Wiki — Multi-Tenant Architecture
• PostgreSQL Documentation — Row Security Policies (RLS)
• OWASP API Security — BOLA (Broken Object-Level Authorization)
• AWS Partner Blog — SaaS Tenant Isolation Models with EKS
• Stripe API Documentation — Authentication & Account Scoping
• Martin Fowler — Writing Resilient Platform Code (bulkhead & isolation patterns)