Why MinHash's minimum-hash trick estimates Jaccard similarity
For two sets and (here, shingle sets from two documents), Jaccard similarity is:
Layer 4 · Foundations
Data Sourcing & Curation
The math of Jaccard estimation and MinHash collision probability, the LSH threshold curve, and hardware reality of running this at trillions of tokens.
14 min read180 XP