RIP, vector database
Learn how treating ANN as a secondary index changes performance characteristics of vector stores.
Write amplification in current vector databases has grown so large that tuning indexing throughput now yields diminishing returns, according to recent observations. The core problem is that indexing forces frequent data movement, inflating I/O work. A new design change promises to break that cycle.
Traditional vector stores embed the ANN index directly with the rows, so every insert or update triggers a reshuffle of both the payload and the vector structures. This coupling multiplies writes: each logical write generates multiple physical writes to keep the index in sync. The resulting amplification hurts latency and throughput, especially at scale.
Turbopuffer v3 addresses the issue by refusing to key on the ANN address; instead it stores vectors in separate fragments while the rows remain stationary. LanceDB follows the same principle, treating ANN as a secondary index that never moves rows. The vector index points to data locations but does not rewrite the underlying fragments, dramatically reducing the number of physical writes.
For anyone building a retrieval system, the lesson is to decouple vector indexing from row storage. Choose a backend that isolates ANN structures, or implement a fragment‑based layout yourself, to keep write amplification low and keep indexing performance predictable.
TakeawaySeparate ANN indexes from row storage to avoid costly data movement and improve indexing throughput.