---
source_url: "https://github.com/ombharatiya/ai-system-design-guide/blob/main/16-case-studies/08-multi-tenant-saas.md"
title: ai-system-design-guide/16-case-studies/08-multi-tenant-saas.md at main · ombharatiya/ai-system-design-guide · GitHub
mirrored_at: 2026-09-02T15:02:28.926Z
host: github.com
cited_in_42a: true
mirror_canonical: "https://index.42a.ai/github.com/ombharatiya/ai-system-design-guide/blob/main/16-case-studies/08-multi-tenant-saas.md"
---

> **Original source:** https://github.com/ombharatiya/ai-system-design-guide/blob/main/16-case-studies/08-multi-tenant-saas.md

## Case Study: Multi-Tenant AI SaaS Platform

## The Problem

A B2B startup is building an **AI-powered document analysis platform** where each customer uploads their own contracts, and the AI answers questions about them. Customers include competitors who must never see each other's data.

**Constraints given in the interview:**

-   500 enterprise customers, each with 10,000-100,000 documents
-   Absolute data isolation: Customer A's data cannot leak to Customer B
-   Shared infrastructure for cost efficiency
-   Compliance: SOC 2 Type II, GDPR
-   Query latency under 2 seconds

* * *

## The Interview Question

> "Design a multi-tenant RAG system where Coca-Cola and Pepsi can both be customers, and there is zero risk of cross-tenant data leakage."

* * *

## Solution Architecture

flowchart TB
    subgraph Gateway\["API Gateway Layer"\]
        REQ\[User Request\] --> AUTH\[Auth Service\]
        AUTH --> TENANT\_ID\[Extract Tenant ID\]
        TENANT\_ID --> VALIDATE\[Validate Tenant Access\]
    end

    subgraph Isolation\["Tenant Isolation Layer"\]
        VALIDATE --> ROUTER{Isolation Strategy}
        ROUTER -->|Small Tenant| SHARED\[(Shared Vector DB<br/>Namespace Isolation)\]
        ROUTER -->|Large Tenant| DEDICATED\[(Dedicated Pod<br/>Physical Isolation)\]
    end

    subgraph Query\["Query Execution"\]
        SHARED --> FILTER\[Tenant Filter<br/>WHERE tenant\_id = X\]
        DEDICATED --> DIRECT\[Direct Query\]
        FILTER --> LLM\[LLM with Tenant Context\]
        DIRECT --> LLM
    end

    subgraph Audit\["Audit Layer"\]
        LLM --> RESPONSE\[Response\]
        LLM --> LOG\[(Audit Log<br/>Immutable)\]
    end

Loading

* * *

## Key Design Decisions

### 1\. Hybrid Isolation: Namespace vs Physical

**Answer:** Pure physical isolation (one database per tenant) is expensive. Pure namespace isolation (shared database with tenant\_id filter) has leakage risk if a filter bug occurs. We use a **tiered approach**:

Tier

Tenant Size

Isolation Method

Why

Standard

<50K docs

Namespace in shared Qdrant

Cost efficient

Premium

50K-500K docs

Dedicated Qdrant collection

Performance isolation

Enterprise

\>500K docs

Dedicated Qdrant pod

Physical + regulatory

### 2\. Defense in Depth for Data Isolation

**Answer:** We never trust a single layer. Our isolation stack:

1.  **API Gateway**: Validates tenant\_id from JWT, rejects cross-tenant requests
2.  **Database Layer**: Row-level security (RLS) enforces tenant\_id filter at DB level
3.  **Application Layer**: ORM wrapper automatically injects tenant filter
4.  **LLM Layer**: System prompt explicitly states "You are answering for Tenant X only"
5.  **Output Layer**: Post-generation filter scans for any document IDs not belonging to tenant

### 3\. Why Not One Vector DB Per Tenant?

**Answer:** 500 tenants × $100/month per managed instance = $50K/month just for databases. By using namespace isolation for 80% of tenants, we reduce this to $8K/month. The remaining 20% on dedicated infrastructure pay a premium tier price.

* * *

## The Data Ingestion Pipeline

flowchart LR
    subgraph Upload\["Document Upload"\]
        DOC\[Customer Document\] --> VALIDATE\[Validate Format\]
        VALIDATE --> TAG\[Tag with Tenant ID\]
    end

    subgraph Process\["Processing Pipeline"\]
        TAG --> PARSE\[Parse Document\]
        PARSE --> CHUNK\[Chunk + Embed\]
        CHUNK --> ENCRYPT\[Encrypt Metadata\]
    end

    subgraph Store\["Storage"\]
        ENCRYPT --> VECTOR\[(Vector DB)\]
        ENCRYPT --> BLOB\[(Blob Storage)\]
        ENCRYPT --> AUDIT\[(Audit Log)\]
    end

Loading

**Critical:** The tenant\_id is attached at the **earliest possible point** (upload validation) and travels with the document through every stage. It is not derived or looked up later.

* * *

## Handling the Compliance Requirements

### SOC 2 Type II

Control

Implementation

Access logging

Every query logged with tenant\_id, user\_id, timestamp

Encryption at rest

AES-256 for blob storage, database-native for vector DB

Encryption in transit

TLS 1.3 everywhere

Access reviews

Automated quarterly reports from audit logs

### GDPR Right to Deletion

async def delete\_tenant\_data(tenant\_id: str):
    \# 1. Delete from vector DB
    await vector\_db.delete(filter\={"tenant\_id": tenant\_id})
    
    \# 2. Delete from blob storage
    await blob\_storage.delete\_prefix(f"tenants/{tenant\_id}/")
    
    \# 3. Anonymize audit logs (cannot delete for compliance)
    await audit\_log.anonymize(tenant\_id\=tenant\_id)
    
    \# 4. Generate deletion certificate
    return generate\_deletion\_certificate(tenant\_id)

* * *

## Cost Analysis (500 Tenants)

Component

Monthly Cost

Shared Vector DB (Qdrant Cloud)

$2,500

Dedicated pods (20 enterprise tenants)

$4,000

LLM costs (pooled, GPT-4o-mini)

$8,000

Blob storage (S3)

$1,500

Audit logging (CloudWatch)

$500

**Total**

**$16,500/month**

**Per tenant average**

**$33/month**

* * *

## Interview Follow-Up Questions

**Q: What if a bug in your ORM bypasses the tenant filter?**

A: Defense in depth. Even if the ORM fails, the database enforces RLS (Row-Level Security). The query `SELECT * FROM documents` internally becomes `SELECT * FROM documents WHERE tenant_id = current_tenant()`. This is enforced at the Postgres level, not the application level.

**Q: How do you handle a tenant who wants to export all their data?**

A: We provide a data portability API that streams all documents with their embeddings and metadata. The export is triggered by an admin, logged in the audit trail, and delivered to a customer-controlled S3 bucket (not our infrastructure).

**Q: What if the LLM hallucinates information from its training data that matches a competitor's confidential info?**

A: This is a real risk. We mitigate by: (1) Using only retrieval-grounded generation (the LLM cannot answer without retrieved docs). (2) Filtering outputs for any content that does not trace back to the tenant's uploaded documents. (3) Offering a "private model" tier where we fine-tune a tenant-specific model on their data only.

* * *

## Key Takeaways for Interviews

1.  **Multi-tenancy is about layers**: never rely on a single isolation mechanism
2.  **Tiered isolation balances cost and security**: not all tenants need dedicated infrastructure
3.  **Tenant ID must be immutable and early**: tag at upload, not at query time
4.  **Compliance is an architecture concern**: design for audit, deletion, and portability from day one

* * *

_Related chapters: [LLM Security](https://github.com/ombharatiya/ai-system-design-guide/blob/main/12-security-and-access/01-llm-security.md), [Access Control & Multi-Tenant Isolation](https://github.com/ombharatiya/ai-system-design-guide/blob/main/12-security-and-access/02-access-control.md)_