Discord Infrastructure Interactive Tour

Overview and High-level Architecture

This section gives a macro view of Discord's infrastructure and how a hybrid cloud strategy with thoughtful technologies supports hundreds of millions of users. The flow below shows a typical user request.

User Request Flow

👤 User

→

Cloudflare (edge) DDoS protection, CDN, Workers

↔

Google Cloud (core) Compute, storage, analytics

→

Discord backend services Elixir, Rust, ScyllaDB

Messages Stored

Trillions

Efficiently stored with ScyllaDB

P99 Read Latency

~15 ms

Improved from Cassandra's 40ms+

Node Efficiency

177 → 72

Nodes after migrating to ScyllaDB

Outbound Traffic Savings

> $100k

Saved monthly via Cloudflare

Hybrid Cloud Strategy: GCP and Cloudflare

Discord blends Google Cloud Platform with Cloudflare in a best-of-breed approach. GCP is the compute and data hub while Cloudflare provides global security and performance.

Google Cloud Platform: Core Hub

Handles most backend services, application logic and core data storage.

✔️
Core compute (GCE/GKE) running backend services, database nodes and containers.
✔️
Custom storage ("Super-Disk") combines local SSDs and persistent disks for ultra-low-latency, reliable DB storage.
✔️
Data analytics (BigQuery/Pub/Sub) powers large-scale analysis and real-time sync.

Cloudflare: Global Shield

Entry point for all traffic, delivering global content distribution, security and edge compute.

✔️
DDoS mitigation Protects WebSocket servers from large attacks—crucial for its gaming community.
✔️
Global CDN Caches and serves static assets from edge nodes, cutting latency and egress cost.
✔️
Edge computing (Workers) Runs code near users for rewrites, routing and authentication before reaching GCP.

Dissecting the Key Tech Stack

Discord embraces a polyglot strategy, choosing the right tool for the job so engineering can focus on differentiating features. Click the cards below to see how each technology fits.

Elixir / BEAM

Handles millions of concurrent WebSocket connections

Rust

High-performance data services and middleware

C++

Extreme performance voice/video processing

ScyllaDB

Primary message store

Elasticsearch

Full-text message search

Python / Data Stack

Data orchestration and AI

Database Evolution: Pursuing Extreme Performance

Discord's critical persistence layer has seen several major migrations—a testament to its fearless innovation in face of explosive user growth. Click the timeline below for details.

2015 MongoDB

2015-2022 Cassandra

2022-present ScyllaDB

Core Architecture Principles

Discord's success stems from a pragmatic engineering philosophy. These principles guide decisions and help them meet hyperscale challenges.

🎯

Build for differentiation, buy for commoditization

Invest engineering effort in unique areas like real-time chat and voice. Use managed services for commoditized parts such as VMs or data warehouses.
🤝

Embrace stateful design for real-time

Not every scaling issue can be solved statelessly. For maximal broadcast efficiency, the Elixir layer keeps state and scales around it.
🔧

Choose the right tool for the job (polyglot)

Be pragmatic: Elixir for concurrency, Rust/C++ for extreme performance, Python for data processing.
🚀

Fearless migration culture

Continuously evolve and refactor core systems to pay technical debt. Willingly invest in high-risk, high-reward data migrations to prepare for the next scale jump.