Sharding and Partitioning

Why split data

Replication copies the same data to many machines. Sharding (also called partitioning) splits different data across machines, so each holds only a slice called a shard.

Reach for it when one machine cannot keep up with:

• Storage: the dataset is bigger than one disk.
• Write throughput: one leader cannot absorb the writes.
• Working set: the hot data no longer fits in RAM.

Replication scales reads. Sharding scales writes and size. Real systems use both: each shard is itself a leader with followers.