A Randomized Approximate Nearest Neighbors Algorithm: Theory and Applications

The classical nearest neighbors problem is formulated as follows: given a collection of N points in the Euclidean space R^d, for each point, find its k nearest neighbors (i.e. closest points). Obviously, for each point X, one can compute the distances from X to every other point, and then find k shortest distances in the resulting array. However, the computational cost of this naive approach is at least (d*N^2)/2 operations, which is prohibitively expensive in many applications. For example, "naively" solving the nearest neighbors problem with d=100, N=1,000,000 and k=30 on a modern laptop can take about as long as a day of CPU time. Fortunately, in such areas as data mining, image processing, machine learning etc., it often suffices to find "approximate" nearest neighbors instead of the "true" ones. In this work, a randomized approximate algorithm for the solution of the nearest neighbors problem is described. It has a considerably lower computational cost than the naive algorithm, and is fairly fast in practical applications. We provide a probabilistic analysis of this algorithm, and demonstrate its performance via several numerical experiments.