Detect duplicate images by hash

You send one entry per image — a name and its hash, as a hexadecimal string or a 0/1 bitstring — plus a maximum Hamming distance in bits. Every pair of hashes within that distance is linked, and the linked pairs are merged into connected groups, so a near-duplicate chain collapses into a single cluster instead of overlapping pairs. The result lists each group with its members, the internal pairwise distances and the largest distance inside the group, together with totals: how many images are duplicates and how many remain unique. Nothing is uploaded: the hashes you already have are all the input it needs.

From a list of hashes to duplicate groups

A perceptual hash such as a 64-bit pHash encodes what an image looks like, so two photos that differ only in compression, size or a watermark produce hashes that differ in just a few bits. Comparing those hashes pairwise is easy; the annoying part is what comes next. Image A matches B, and B matches C, but A does not quite match C — are they one duplicate set or two? This capability resolves that question the standard way: it treats every image as a node, draws an edge between any two hashes whose Hamming distance is at most the threshold you choose, and reports the connected components as the duplicate groups. You get one entry per group with the member names, every within-threshold pair and its exact distance, and the largest distance inside the group, so you can audit why two images ended up together. Because grouping is by transitivity, a resave chain that drifts one bit at a time still lands in a single cluster, which is exactly how near-duplicate detection is expected to behave on real photo libraries.

Choosing the threshold and the hash format

The threshold is a maximum Hamming distance in bits and defaults to 5, a common starting point for 64-bit hashes: identical images score 0, and visually indistinguishable re-encodes usually stay under 5. Lower it toward 0 when you only want exact or near-exact matches, raise it cautiously when your pipeline resizes aggressively or crops borders, because each extra bit of tolerance multiplies the chance of false positives on large collections. Hashes are accepted in two formats: hexadecimal strings, decoded at four bits per character, or literal 0/1 bitstrings. The only hard rule is consistency — every hash in one request must use the same format and decode to the same bit length, otherwise a distance between them would be meaningless and the request is rejected with a clear error. A negative threshold is likewise rejected as invalid input. You can compute the hashes themselves with the sibling capability image.phash, which produces exactly the 16-character hexadecimal strings this endpoint expects.

Deterministic output you can diff and store

The response is designed to be stored next to your catalog and compared over time. Groups are sorted by the position of their earliest member in your input list, members keep input order, and the internal pairs are listed with both names and the exact distance, so two runs over the same list produce byte-identical output and regressions show up as clean diffs. Alongside the groups you get the counts that dashboards need: total images processed, the bit length of the hashes, the number of duplicate groups, how many images fall inside any group, and how many remain unique. There is no randomness, no clock and no network involved: the same input always yields the same answer, in the API and in the free browser runner on this page, which executes the very same code. That makes it safe to use in CI pipelines that fail a build when new duplicates slip into an asset repository, and in scheduled clean-up jobs that email a deduplication report.