sharding

When you shard a vector index or route requests across model replicas, naive 'hash mod N' reshuffles everything the moment N changes. Consistent hashing moves only a small fraction. Here is the implementation.

A build-it-yourself check on distributed input pipelines. What matters is partitioning data across workers with no overlap and no gaps, epoch-consistent shuffling with a shared seed, and handling the uneven-last-batch problem. The code follows.

Showing the top 10 is easy. Showing a player their global rank among 50M others, updated live, is the real challenge. The answer turns on the right data structure, sharding, and approximate ranks for the long tail. Here is how leaderboards scale.

FSDP and ZeRO let you train models too large for one GPU without splitting the compute. The signal is the ZeRO stages (what gets sharded at each level) and the communication you pay for it. Here is the answer.

When a model is too large for one GPU you split the model itself, not just the data. The signal is telling tensor parallelism (split within a layer) apart from pipeline parallelism (split across layers) and matching each to the interconnect. Here is the answer.