A Geometric Basis for Accurate 2-bit Quantisation
Modern neural networks generate huge lists of numbers to represent digital objects (images, text etc) which allow us to measure similarity among them using a well-defined mathematical function. The size of the lists causes a serious problem for the cost of measurement. Here we show how some geometric principles known to Archimedes can be generalised into high dimensions to allow these numbers, typically each represented in 8 or even 16 bytes, to be squeezed into a tiny fraction of that space, where their similarity can be measured using a function that is 100 times faster.
Keywords
deep learning, similarity, geometry, convex polytopes
Staff
Richard Connor, Alan Dearle, Ben Claydon