Yasser Alharbi

تخصص الذكاء الاصطناعي · دفعة 2026

Efficient Recall-Guaranteed Spectral Similarity Search on Massive Datasets: 2 Billion Stars from Gaia

Vector search has become a core operation in modern retrieval systems, but many scientific and interactive workloads do not fit the top-k model that most systems are built around. Given a query object, a similarity radius, and optional filters over structured attributes, these workloads require every matching record to be returned, a threshold query, whose answer size varies from one query to the next. In this setting recall cannot be traded away: extra candidates can be discarded by an exact check, but a silently missing result can invalidate the analysis that depends on it. This project develops a similarity search system that guarantees recall by construction and makes that guarantee affordable at massive scale. Compressed vector indexes and provable geometric bounds discard only those records that cannot possibly satisfy the query, so no valid result is ever lost, and the research problem is efficiency: a guarantee is useful only if the set of surviving candidates is small enough to verify quickly. On spectral measurements from the Gaia catalogue the system cuts the fraction of the database that is scanned from 88% to 40%, and the vectors reaching exact-distance verification from 179 per true answer to 1.11, while running roughly twice as fast as the recall-guaranteed baseline and holding recall at exactly 1.0 on every query. Four composable cell bounds, a covering sphere, a bounding box, a Voronoi bisector test and per-cell sub-group boxes, achieve this at zero storage cost per vector, leaving memory spent on the quantizer as the one remaining trade-off, which the work characterises explicitly.

اقرأ التقرير النهائي (PDF)