6 Hard Problems Scaling Vector Search

6 Hard Problems Scaling Vector Search

RelatedHighlights

Vector databases (Part 4): Analyzing the trade-offs

Prashanth Rao thedataquarry.com

Vector Databases Are the Wrong Abstraction. Here’s a Better Way.

Timescale Blog timescale.com

Thumbnail of Vector Databases Are the Wrong Abstraction. Here’s a Better Way.

How Discord Stores Trillions of Messages

Bo Ingram discord.com

Thumbnail of How Discord Stores Trillions of Messages

Scalability is crucial - systems need to be designed with the assumption that query volume, document corpus size, indexing complexity etc. could increase by 10x. What works at one scale may completely break at a higher scale.

Sharding the index, either by document or by word, is important to distribute the indexing and querying load across machines.

... See more

Claude