Sharding is a technique for splitting a large dataset or workload across multiple independent databases or storage nodes, called shards. Each shard contains only a portion of the total data, often divided by user ID, geographic region, or another partition key. Sharding reduces load on individual servers and allows systems to scale horizontally as demand grows. However, it also introduces complexity in query routing, consistency, and operational management.
Why it matters
As applications grow, a single database can become a bottleneck. Sharding enables near-infinite scale by distributing data and queries. It supports high throughput and large data volumes but requires careful planning to avoid hotspots, uneven load, or complicated cross-shard queries.
Examples
Sharding users across multiple PostgreSQL instances, or MongoDB automatically distributing data across a shard cluster. Lessons like Sharding and Horizontal Scaling explain strategies and tradeoffs.