API reference

Signature generation

class image_match.goldberg.ImageSignature(n: int = 9, crop_percentiles: tuple[int, int] | None = (5, 95), P: int | None = None, diagonal_neighbors: bool = True, identical_tolerance: float = 0.00784313725490196, n_levels: int = 2, fix_ratio: bool = False)[source]

Image signature generator.

Based on the method of Goldberg, et al. Available at http://www.cs.cmu.edu/~hcwong/Pdfs/icip02.ps

__init__(n: int = 9, crop_percentiles: tuple[int, int] | None = (5, 95), P: int | None = None, diagonal_neighbors: bool = True, identical_tolerance: float = 0.00784313725490196, n_levels: int = 2, fix_ratio: bool = False) → None[source]

Initialize the signature generator.

The default parameters match those given in Goldberg’s paper.

Note

Non-default parameters have not been extensively tested. Use carefully.

Parameters:

  • n (Optional[int]) – size of grid imposed on image. Grid is n x n (default 9)

  • crop_percentiles (Optional[Tuple[int]]) – lower and upper bounds when considering how much variance to keep in the image (default (5, 95))

  • P (Optional[int]) – size of sample region, P x P. If none, uses a sample region based on the size of the image (default None)

  • diagonal_neighbors (Optional[boolean]) – whether to include diagonal grid neighbors (default True)

  • identical_tolerance (Optional[float]) – cutoff difference for declaring two adjacent grid points identical (default 2/255)

  • n_levels (Optional[int]) – number of positive and negative groups to stratify neighbor differences into. n = 2 -> [-2, -1, 0, 1, 2] (default 2)

  • fix_ratio (Optional[boolean]) – use the larger crop ratio for both directions. Only use if you can guarantee the incoming image is square (default False)

static compute_differentials(grey_level_matrix: ndarray, diagonal_neighbors: bool = True) → ndarray[source]

Computes differences in greylevels for neighboring grid points.

First part of ‘step 4’ in the paper.

Returns n x n x 8 rank 3 array for an n x n grid (if diagonal_neighbors == True)

The n x nth coordinate corresponds to a grid point. The eight values are the differences between neighboring grid points, in this order:

upper left upper upper right left right lower left lower lower right

Parameters:

  • grey_level_matrix (numpy.ndarray) – grid of values sampled from image

  • diagonal_neighbors (Optional[boolean]) – whether or not to use diagonal neighbors (default True)

Returns:

a n x n x 8 rank 3 numpy array for an n x n grid (if diagonal_neighbors == True)

Examples

>>> img = gis.preprocess_image('https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg')
>>> window = gis.crop_image(img)
>>> grid = gis.compute_grid_points(img, window=window)
>>> grey_levels = gis.compute_mean_level(img, grid[0], grid[1])
>>> gis.compute_differentials(grey_levels)
array([[[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   1.82683143e-03,  -0.00000000e+00,
           2.74085276e-01,   1.24737821e-01],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          -1.82683143e-03,   2.15563930e-03,   2.72258444e-01,
           1.22910990e-01,   3.48522956e-01],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          -2.15563930e-03,   1.16963917e-01,   1.20755351e-01,
           3.46367317e-01,   1.99638513e-01],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          -1.16963917e-01,   1.32128118e-01,   2.29403399e-01,
           8.26745956e-02,   1.16455050e-01],
        ...
static compute_grid_points(image: ndarray, n: int = 9, window: list[tuple[int, int]] | None = None) → tuple[ndarray, ndarray][source]

Computes grid points for image analysis.

Corresponds to the second part of ‘step 2’ in the paper

Parameters:

  • image (numpy.ndarray) – n x m array of floats – the greyscale image. Typically, the output of preprocess_image

  • n (Optional[int]) – number of gridpoints in each direction (default 9)

  • window (Optional[List[Tuple[int]]]) – limiting coordinates [(t, b), (l, r)], typically the output of (default None)

Returns:

tuple of arrays indicating the vertical and horizontal locations of the grid points

Examples

>>> img = gis.preprocess_image('https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg')
>>> window = gis.crop_image(img)
>>> gis.compute_grid_points(img, window=window)
(array([100, 165, 230, 295, 360, 424, 489, 554, 619]),
 array([ 66, 109, 152, 195, 238, 280, 323, 366, 409]))
static compute_mean_level(image: ndarray, x_coords: ndarray, y_coords: ndarray, P: float | None = None) → ndarray[source]

Computes array of greyness means.

Corresponds to ‘step 3’

Parameters:

  • image (numpy.ndarray) – n x m array of floats – the greyscale image. Typically, the output of preprocess_image

  • x_coords (numpy.ndarray) – array of row numbers

  • y_coords (numpy.ndarray) – array of column numbers

  • P (Optional[int]) – size of boxes in pixels (default None)

Returns:

an N x N array of average greyscale around the gridpoint, where N is the

number of grid points

Examples

>>> img = gis.preprocess_image('https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg')
>>> window = gis.crop_image(img)
>>> grid = gis.compute_grid_points(img, window=window)
>>> gis.compute_mean_level(img, grid[0], grid[1])
array([[ 0.62746325,  0.62563642,  0.62348078,  0.50651686,  0.37438874,
         0.0644063 ,  0.55968952,  0.59356148,  0.60473832],
       [ 0.35337797,  0.50272543,  0.27711346,  0.42384226,  0.39006181,
         0.16773968,  0.10471924,  0.33647144,  0.62902124],
       [ 0.20307514,  0.19021892,  0.12435402,  0.44990121,  0.38527996,
         0.08339507,  0.05530059,  0.18469107,  0.21125228],
       [ 0.25727387,  0.1669419 ,  0.08964046,  0.1372754 ,  0.48529236,
         0.39894004,  0.10387907,  0.11282135,  0.30014612],
       [ 0.23447867,  0.15702549,  0.25232943,  0.75172715,  0.79488688,
         0.4943538 ,  0.29645163,  0.10714578,  0.0629376 ],
       [ 0.22167555,  0.04839472,  0.10125833,  0.1550749 ,  0.14346914,
         0.04713144,  0.10095568,  0.15349296,  0.04456733],
       [ 0.09233709,  0.11210942,  0.05361996,  0.07066566,  0.04191625,
         0.03548839,  0.03420656,  0.05025029,  0.03519956],
       [ 0.19226873,  0.20647194,  0.62972106,  0.45514529,  0.05620413,
         0.03383168,  0.03413588,  0.04741828,  0.02987698],
       [ 0.05799523,  0.23310153,  0.43719717,  0.27666873,  0.25106573,
         0.11094163,  0.10180622,  0.04633349,  0.02704855]])
static crop_image(image: ndarray, lower_percentile: int = 5, upper_percentile: int = 95, fix_ratio: bool = False) → list[tuple[int, int]][source]

Crops an image, removing featureless border regions.

Corresponds to the first part of ‘step 2’ in Goldberg’s paper

Parameters:

  • image (numpy.ndarray) – n x m array of floats – the greyscale image. Typically, the output of preprocess_image

  • lower_percentile (Optional[int]) – crop image by percentage of difference (default 5)

  • upper_percentile (Optional[int]) – as lower_percentile (default 95)

  • fix_ratio (Optional[boolean]) – use the larger ratio for both directions. This is useful for using the fast signature transforms on sparse but very similar images (e.g. renderings from fixed directions). Use with care – only use if you can guarantee the incoming image is square (default False).

Returns:

[(top, bottom), (left, right)]

Return type:

A pair of tuples describing the ‘window’ of the image to use in analysis

Examples

>>> img = gis.preprocess_image('https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg')
>>> gis.crop_image(img)
[(36, 684), (24, 452)]
generate_signature(path_or_image: ImageInput, bytestream: bool = False) → ndarray[source]

Generates an image signature.

See section 3 of Goldberg, et al.

Parameters:

  • path_or_image (string or numpy.ndarray) – image path, or image array

  • bytestream (Optional[boolean]) – will the image be passed as raw bytes? That is, is the ‘path_or_image’ argument an in-memory image? (default False)

Returns:

A rank 1 numpy array of length n x n x 8

(or n x n x 4 if diagonal_neighbors == False)

Return type:

The image signature

Examples

>>> from image_match.goldberg import ImageSignature
>>> gis = ImageSignature()
>>> gis.generate_signature('https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg')
array([ 0,  0,  0,  0,  0,  0,  2,  2,  0,  0,  0,  0,  0,  2,  2,  2,  0,
        0,  0,  0,  2,  2,  2,  2,  0,  0,  0, -2,  2,  2,  1,  2,  0,  0,
        0, -2,  2, -1, -1,  2,  0,  0,  0, -2, -2, -2, -2, -1,  0,  0,  0,
        2, -1,  2,  2,  2,  0,  0,  0,  1, -1,  2,  2, -1,  0,  0,  0,  1,
        0,  2, -1,  0,  0, -2, -2,  0, -2,  0,  2,  2, -2, -2, -2,  2,  2,
        2,  2,  2, -2, -2, -2, -2, -2,  1,  2, -2, -2, -1,  1,  2,  1,  2,
       -1,  1, -2,  1,  2, -1,  2, -1,  0,  2, -2,  2, -2, -2,  1, -2,  1,
        2,  1, -2, -2, -1, -2,  1,  1, -1, -2, -2, -2,  2, -2,  2,  2,  2,
        1,  1,  0,  2,  0,  2,  2,  0,  0, -2, -2,  0,  1,  0, -1,  1, -2,
       -2, -1, -1,  1, -1,  1,  1, -2, -2, -2, -1, -2, -1,  1, -1,  2,  1,
        1,  2,  1,  2,  2, -1, -1,  0,  2, -1,  2,  2, -1, -1, -2, -1, -1,
       -2,  1, -2, -2, -1, -2, -1, -2, -1, -2, -2, -1, -1,  1, -2, -2,  2,
       -1,  1,  1, -2, -2, -2,  0,  1,  0,  1, -1,  0,  0,  1,  1,  0,  1,
        0,  1,  1, -1, -1,  1, -1,  1, -1,  1, -1, -1, -1, -2, -1, -1, -1,
       -2, -2,  1, -2, -2,  1, -2, -2, -2, -2,  1,  1,  2,  2,  1, -2, -2,
       -1,  1,  2,  2, -1,  2, -2, -1,  1,  1,  1, -1, -2, -1, -2, -2,  0,
        1, -1, -1,  1, -2, -2,  0,  1,  2,  1,  0,  2,  0,  2,  2,  0,  0,
       -1,  1,  0,  1,  0,  1,  2, -1, -1,  1, -1, -1, -1,  2,  1,  1,  2,
        2,  1, -2,  2,  2,  1,  2,  2,  2,  2, -1,  2,  2,  2,  2,  2,  2,
        1,  2,  2,  2,  2,  1,  1,  2, -2,  2,  2,  2,  2, -1,  2,  2, -2,
        2,  2,  2,  2,  0,  0, -2, -2,  1,  0, -1,  1, -1, -2,  0, -1,  0,
       -1,  1,  0,  0, -1,  1,  0,  2,  0,  2,  2, -2, -2, -2, -2, -1, -1,
       -1,  0, -1, -2, -2,  1, -1, -1,  1,  1, -1, -2, -2,  1,  1,  1,  1,
        2, -2, -2, -2, -1,  1,  1,  1,  2, -2, -2, -2, -1, -1,  0,  1,  1,
       -2, -2,  0,  1, -1,  1,  1,  1, -2,  1,  1,  1,  2,  2,  2,  2, -1,
       -1,  0, -2,  0,  0,  1,  0,  0, -2,  1,  0, -1,  0, -1, -2, -2,  1,
        1,  1,  1, -1, -1, -2,  0, -1, -1, -1, -1, -2, -2, -2, -1, -1, -1,
        1,  1, -2, -2,  1, -2, -1,  0, -1,  0, -2, -1,  1, -2, -1, -1,  0,
        0, -1,  0,  0, -1, -1, -2,  0, -1,  0,  0, -1, -1, -2,  0,  1,  1,
        1,  0,  1, -2, -1,  0, -1,  0, -1,  0,  0,  0,  1,  1,  0, -1,  0,
        2, -1,  2,  1,  2,  1, -2,  2, -1, -2,  2,  2,  2,  2,  2,  2,  2,
        2,  2,  2,  2, -2,  2,  1,  2,  2, -1,  1,  1, -2,  1, -2, -2, -1,
       -1,  0,  0, -1,  0, -2, -1, -1,  0,  0, -1,  0, -1, -1, -1, -1,  1,
        0,  1,  1,  1, -1,  0,  1, -1,  0,  0, -1,  0, -1,  0,  0,  0, -2,
       -2,  0, -2,  0,  0,  0,  1,  1, -2,  2, -2,  0,  0,  0,  2, -2, -1,
        2,  2,  0,  0,  0, -2, -2,  2, -2,  1,  0,  0,  0, -2,  2,  2, -1,
        2,  0,  0,  0,  1,  1,  1, -2,  1,  0,  0,  0,  1,  1,  1, -1,  1,
        0,  0,  0,  1,  0,  1, -1,  1,  0,  0,  0, -1,  0,  0, -1,  0,  0,
        0,  0], dtype=int8)
static normalize_and_threshold(difference_array: ndarray, identical_tolerance: float = 0.00784313725490196, n_levels: int = 2) → None[source]

Normalizes difference matrix in place.

‘Step 4’ of the paper. The flattened version of this array is the image signature.

Parameters:

  • difference_array (numpy.ndarray) – n x n x l array, where l are the differences between the grid point and its neighbors. Typically the output of compute_differentials

  • identical_tolerance (Optional[float]) – maximum amount two gray values can differ and still be considered equivalent (default 2/255)

  • n_levels (Optional[int]) – bin differences into 2 n + 1 bins (e.g. n_levels=2 -> [-2, -1, 0, 1, 2])

Examples

>>> img = gis.preprocess_image('https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg')
>>> window = gis.crop_image(img)
>>> grid = gis.compute_grid_points(img, window=window)
>>> grey_levels = gis.compute_mean_level(img, grid[0], grid[1])
>>> m = gis.compute_differentials(grey_levels)
>>> m
array([[[ 0.,  0.,  0.,  0.,  0.,  0.,  2.,  2.],
        [ 0.,  0.,  0.,  0.,  0.,  2.,  2.,  2.],
        [ 0.,  0.,  0.,  0.,  2.,  2.,  2.,  2.],
        [ 0.,  0.,  0., -2.,  2.,  2.,  1.,  2.],
        [ 0.,  0.,  0., -2.,  2., -1., -1.,  2.],
        [ 0.,  0.,  0., -2., -2., -2., -2., -1.],
        [ 0.,  0.,  0.,  2., -1.,  2.,  2.,  2.],
        [ 0.,  0.,  0.,  1., -1.,  2.,  2., -1.],
        [ 0.,  0.,  0.,  1.,  0.,  2., -1.,  0.]],
[[ 0., -2., -2., 0., -2., 0., 2., 2.],

[-2., -2., -2., 2., 2., 2., 2., 2.], [-2., -2., -2., -2., -2., 1., 2., -2.], [-2., -1., 1., 2., 1., 2., -1., 1.], [-2., 1., 2., -1., 2., -1., 0., 2.], …

static normalized_distance(_a: ndarray, _b: ndarray) → float[source]

Compute normalized distance between two points.

Computes || b - a || / ( ||b|| + ||a||)

Parameters:

  • _a (numpy.ndarray) – array of size m

  • _b (numpy.ndarray) – array of size m

Returns:

normalized distance between signatures (float)

Examples

>>> a = gis.generate_signature('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
>>> b = gis.generate_signature('https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg')
>>> gis.normalized_distance(a, b)
0.22095170140933634
static preprocess_image(image_or_path: ImageInput, bytestream: bool = False, handle_mpo: bool = False) → ndarray[source]

Loads an image and converts to greyscale.

Corresponds to ‘step 1’ in Goldberg’s paper

Parameters:

  • image_or_path (string or numpy.ndarray) – image path, or image array

  • bytestream (Optional[boolean]) – will the image be passed as raw bytes? That is, is the ‘path_or_image’ argument an in-memory image? (default False)

  • handle_mpo (Optional[boolean]) – try to compute a signature for steroscopic images by extracting the first image of the set (default False)

Returns:

Array of floats corresponding to greyscale level at each pixel

Examples

>>> gis.preprocess_image('https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg')
array([[ 0.26344431,  0.32423294,  0.30406745, ...,  0.35069725,
         0.36499961,  0.36361569],
       [ 0.29676627,  0.28640118,  0.34523255, ...,  0.3703051 ,
         0.34931333,  0.31655686],
       [ 0.35305216,  0.31858431,  0.36202   , ...,  0.40588196,
         0.37284275,  0.30871373],
       ...,
       [ 0.05932863,  0.05540706,  0.05540706, ...,  0.01954745,
         0.01954745,  0.01562588],
       [ 0.0632502 ,  0.05540706,  0.05148549, ...,  0.01954745,
         0.02346902,  0.01562588],
       [ 0.06717176,  0.05540706,  0.05148549, ...,  0.02346902,
         0.02739059,  0.01954745]])
exception image_match.goldberg.CorruptImageError[source]

Raised when image data cannot be decoded.

Driver base

class image_match.signature_database_base.SignatureDatabaseBase(k: int = 16, N: int = 63, n_grid: int = 9, crop_percentile: tuple[int, int] | None = (5, 95), distance_cutoff: float = 0.45, *signature_args: Any, **signature_kwargs: Any)[source]

Base class for storing and searching image signatures in a database

Note

You must implement the methods search_single_record and insert_single_record in a derived class

__init__(k: int = 16, N: int = 63, n_grid: int = 9, crop_percentile: tuple[int, int] | None = (5, 95), distance_cutoff: float = 0.45, *signature_args: Any, **signature_kwargs: Any) → None[source]

Set up storage scheme for images

Central to the speed of this approach is the transforming the image signature into something that can be speedily indexed and matched. In our case, that means splitting the image signature into N words of length k, then encoding those words as integers. The idea here is that integer indices are more efficient than array indices.

For example, say your image signature is [0, 1, 2, 0, -1, -2, 0, 1] and k=3 and N=4. That means we want 4 words of length 3. For this signa- ture, that gives us:

[0, 1, 2] [2, 0, -1] [-1, -2, 0] [0, 1]

Note that signature elements can be repeated, and any mismatch in length is chopped off in the last word (which will be padded with zeros). Since these numbers run from -2..2, there 5 possibilites. Adding 2 to each word makes them strictly non-negative, then the quantity, and transforming to base-5 makes unique integers. For the first word:

[0, 1, 2] + 2 = [2, 3, 4] [5**0, 5**1, 5**2] = [1, 5, 25] dot([2, 3, 4], [1, 5, 25]) = 2 + 15 + 100 = 117

So the integer word is 117. Storing all the integer words as different database columns or fields gives us the speedy lookup. In practice, word arrays are ‘squeezed’ to between -1..1 before encoding.

Parameters:

  • k (Optional[int]) – the width of a word (default 16)

  • N (Optional[int]) – the number of words (default 63)

  • n_grid (Optional[int]) – the n_grid x n_grid size to use in determining the image signature (default 9)

  • crop_percentile (Optional[Tuple[int]]) – lower and upper bounds when considering how much variance to keep in the image (default (5, 95))

  • distance_cutoff (Optional [float]) – maximum image signature distance to be considered a match (default 0.45)

  • *signature_args – Variable length argument list to pass to ImageSignature

  • **signature_kwargs – Arbitrary keyword arguments to pass to ImageSignature

add_image(path: str, img: ImageInput | None = None, bytestream: bool = False, metadata: dict[str, Any] | None = None, *args: Any, **kwargs: Any) → None[source]

Add a single image to the database

Parameters:

  • path (string) – path or identifier for image. If img=None, then path is assumed to be a URL or filesystem path

  • img (Optional[string]) – usually raw image data. In this case, path will still be stored, but a signature will be generated from data in img. If bytestream is False, but img is not None, then img is assumed to be the URL or filesystem path. Thus, you can store image records with a different ‘path’ than the actual image location (default None)

  • bytestream (Optional[boolean]) – will the image be passed as raw bytes? That is, is the ‘path_or_image’ argument an in-memory image? If img is None but, this argument will be ignored. If img is not None, and bytestream is False, then the behavior is as described in the explanation for the img argument (default False)

  • metadata (Optional) – any other information you want to include, can be nested (default None)

  • *args – Variable length argument list to pass to insert_single_record

  • **kwargs – Arbitrary keyword arguments to pass to insert_single_record

add_images(paths: list[ImageInput], metadata: dict[str, Any] | list[dict[str, Any] | None] | None = None, bytestream: bool = False, n_threads: int = 1, chunk_size: int | None = None, **kwargs: Any) → int[source]

Generate signatures for many images and bulk-insert the records.

Signature generation is the bottleneck for bulk imports; n_threads > 1 parallelizes it (helps most for URL/bytes inputs since decode and numpy partially release the GIL). For CPU-only workloads, run signature_database_base.make_record in a multiprocessing pool — it is picklable by design — and pass the records to insert_records.

Parameters:

  • paths (list) – paths, URLs, or raw image data

  • metadata (Optional) – a single dict applied to every record, or a list of dicts/None aligned with paths (default None)

  • bytestream (Optional[boolean]) – inputs are raw image bytes (default False)

  • n_threads (Optional[int]) – signature-generation threads (default 1)

  • chunk_size (Optional[int]) – process and insert in batches of this many images instead of buffering all records in memory (default None = one bulk request)

  • **kwargs – passed to insert_records (e.g. refresh_after — honored on the final chunk only)

Returns:

the number of successfully indexed records

insert_records(records: list[dict[str, Any]], **kwargs: Any) → int[source]

Bulk-insert records made by make_record/add_images.

Must be implemented by derived class; returns the number of records successfully indexed.

insert_single_record(rec: dict[str, Any]) → None[source]

Insert an image record.

Must be implemented by derived class.

Parameters:

rec (dict) –

an image record. Will be in the format returned by make_record

For example, rec could have the form:

{'path': 'https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg',
 'signature': [0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 2, 2, 2, 0, 0, 0, 0, 2, 2, 2, 2, 0 ... ]
 'simple_word_0': 42252475,
 'simple_word_1': 23885671,
 'simple_word_10': 9967839,
 'simple_word_11': 4257902,
 'simple_word_12': 28651959,
 'simple_word_13': 33773597,
 'simple_word_14': 39331441,
 'simple_word_15': 39327300,
 'simple_word_16': 11337345,
 'simple_word_17': 9571961,
 'simple_word_18': 28697868,
 'simple_word_19': 14834907,
 'simple_word_2': 7434746,
 'simple_word_20': 37985525,
 'simple_word_21': 10753207,
 'simple_word_22': 9566120,
 ...
 'metadata': {...}
 }

The number of simple words corresponds to the attribute N

search_image(path: ImageInput, all_orientations: bool = False, bytestream: bool = False, pre_filter: PreFilter = None, n_threads: int = 1, **kwargs: Any) → list[dict[str, Any]][source]

Search for matches

Parameters:

  • path (string) – path or image data. If bytestream=False, then path is assumed to be a URL or filesystem path. Otherwise, it’s assumed to be raw image data

  • all_orientations (Optional[boolean]) – if True, search for all combinations of mirror images, rotations, and color inversions (default False)

  • bytestream (Optional[boolean]) – will the image be passed as raw bytes? That is, is the ‘path_or_image’ argument an in-memory image? (default False)

  • pre_filter (Optional[dict]) – filters list before applying the matching algorithm (default None)

  • n_threads (Optional[int]) – run orientation searches concurrently in this many threads; only relevant when all_orientations=True (default 1 = sequential)

  • **kwargs – Arbitrary keyword arguments to pass to search_single_record (e.g. n_parallel_words for the MongoDB driver)

Returns:

a formatted list of dicts representing unique matches, sorted by dist

For example, if three matches are found:

[
 {'dist': 0.069116439263706961,
  'id': 'AVM37oZq0osmmAxpPvx7',
  'path': 'https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg'},
 {'dist': 0.22484320805049718,
  'id': 'AVM37nMg0osmmAxpPvx6',
  'path': 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg'},
 {'dist': 0.42529792112113302,
  'id': 'AVM37p530osmmAxpPvx9',
  'path': 'https://c2.staticflickr.com/8/7158/6814444991_08d82de57e_z.jpg'}
]

search_single_record(rec: dict[str, Any], pre_filter: PreFilter = None) → list[dict[str, Any]][source]

Search for a matching image record.

Must be implemented by derived class.

Parameters:

  • rec (dict) –

    an image record. Will be in the format returned by make_record

    For example, rec could have the form:

    {'path': 'https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg',
 'signature': [0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 2, 2, 2, 0, 0, 0, 0, 2, 2, 2, 2, 0 ... ]
 'simple_word_0': 42252475,
 'simple_word_1': 23885671,
 'simple_word_10': 9967839,
 'simple_word_11': 4257902,
 'simple_word_12': 28651959,
 'simple_word_13': 33773597,
 'simple_word_14': 39331441,
 'simple_word_15': 39327300,
 'simple_word_16': 11337345,
 'simple_word_17': 9571961,
 'simple_word_18': 28697868,
 'simple_word_19': 14834907,
 'simple_word_2': 7434746,
 'simple_word_20': 37985525,
 'simple_word_21': 10753207,
 'simple_word_22': 9566120,
 ...
 'metadata': {'category': 'art'},
 }

    The number of simple words corresponds to the attribute N

  • pre_filter (dict) –

    a filter to be applied by the concrete implementation

    before applying the matching strategy

    For example:

    { “term”: { “image.metadata.category”: “art” } }

Returns:

a formatted list of dicts representing matches.

For example, if three matches are found:

[
 {'dist': 0.069116439263706961,
  'id': 'AVM37oZq0osmmAxpPvx7',
  'path': 'https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg'},
 {'dist': 0.22484320805049718,
  'id': 'AVM37nMg0osmmAxpPvx6',
  'path': 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg'},
 {'dist': 0.42529792112113302,
  'id': 'AVM37p530osmmAxpPvx9',
  'metadata': {...},
  'path': 'https://c2.staticflickr.com/8/7158/6814444991_08d82de57e_z.jpg'}
]

You can return any fields you like, but must include at least dist and id. Duplicate entries are ok, and they do not need to be sorted

Module helpers

image_match.signature_database_base.make_record(path: ImageInput, gis: ImageSignature, k: int, N: int, img: ImageInput | None = None, bytestream: bool = False, metadata: dict[str, Any] | None = None) → dict[str, Any][source]

Makes a record suitable for database insertion.

Note

This non-class version of make_record is provided for CPU pooling. Functions passed to worker processes must be picklable.

Parameters:

  • path (string) – path or image data. If bytestream=False, then path is assumed to be a URL or filesystem path. Otherwise, it’s assumed to be raw image data

  • gis (ImageSignature) – an instance of ImageSignature for generating the signature

  • k (int) – width of words for encoding

  • N (int) – number of words for encoding

  • img (Optional[string]) – usually raw image data. In this case, path will still be stored, but a signature will be generated from data in img. If bytestream is False, but img is not None, then img is assumed to be the URL or filesystem path. Thus, you can store image records with a different ‘path’ than the actual image location (default None)

  • bytestream (Optional[boolean]) – will the image be passed as raw bytes? That is, is the ‘path_or_image’ argument an in-memory image? If img is None but, this argument will be ignored. If img is not None, and bytestream is False, then the behavior is as described in the explanation for the img argument (default False)

  • metadata (Optional) – any other information you want to include, can be nested (default None)

Returns:

An image record.

For example:

{‘path’: ‘https://pixabay.com/static/uploads/photo/2012/11/28/08/56/mona-lisa-67506_960_720.jpg’,

’signature’: [0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 0, 0, 0, 2, 2, 2, 0, 0, 0, 0, 2, 2, 2, 2, 0 … ] ‘simple_word_0’: 42252475, ‘simple_word_1’: 23885671, ‘simple_word_10’: 9967839, ‘simple_word_11’: 4257902, ‘simple_word_12’: 28651959, ‘simple_word_13’: 33773597, ‘simple_word_14’: 39331441, ‘simple_word_15’: 39327300, ‘simple_word_16’: 11337345, ‘simple_word_17’: 9571961, ‘simple_word_18’: 28697868, ‘simple_word_19’: 14834907, ‘simple_word_2’: 7434746, ‘simple_word_20’: 37985525, ‘simple_word_21’: 10753207, ‘simple_word_22’: 9566120, … ‘metadata’: {…} }

image_match.signature_database_base.get_words(array: ndarray, k: int, N: int) → ndarray[source]

Gets N words of length k from an array.

Words may overlap.

For example, say your image signature is [0, 1, 2, 0, -1, -2, 0, 1] and k=3 and N=4. That means we want 4 words of length 3. For this signature, that gives us:

[0, 1, 2] [2, 0, -1] [-1, -2, 0] [0, 1]

Parameters:

  • array (numpy.ndarray) – array to split into words

  • k (int) – word length

  • N (int) – number of words

Returns:

an array with N rows of length k

image_match.signature_database_base.words_to_int(word_array: ndarray) → ndarray[source]

Converts a simplified word to an integer

Encodes a k-byte word to int (as those returned by max_contrast). First digit is least significant.

Returns dot(word + 1, [1, 3, 9, 27 …] ) for each word in word_array

e.g.: [ -1, -1, -1] -> 0 [ 0, 0, 0] -> 13 [ 0, 1, 0] -> 16

Parameters:

word_array (numpy.ndarray) – N x k array

Returns:

an array of integers of length N (the integer word encodings)

image_match.signature_database_base.max_contrast(array: ndarray) → None[source]

Sets all positive values to one and all negative values to -1.

Needed for first pass lookup on word table.

Parameters:

array (numpy.ndarray) – target array

image_match.signature_database_base.normalized_distance(_target_array: ndarray, _vec: ndarray, nan_value: float = 1.0) → ndarray[source]

Compute normalized distance to many points.

Computes || vec - b || / ( ||vec|| + ||b||) for every b in target_array

Parameters:

  • _target_array (numpy.ndarray) – N x m array

  • _vec (numpy.ndarray) – array of size m

  • nan_value (Optional[float]) – value to replace 0.0/0.0 = nan with (default 1.0, to take those featureless images out of contention)

Returns:

the normalized distance (float)

image_match.signature_database_base.dedupe_results(result: list[dict[str, Any]]) → list[dict[str, Any]][source]

Drop duplicate ids and sort by dist.

Elasticsearch

Requires the elasticsearch extra.

class image_match.elasticsearch_driver.SignatureES(es: ClientT, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, minimum_should_match: int | str | None = None, use_filter_context: bool = False, *args: Any, **kwargs: Any)[source]

Bases: SignatureDatabaseBase, Generic

Elasticsearch driver for image-match

__init__(es: ClientT, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, minimum_should_match: int | str | None = None, use_filter_context: bool = False, *args: Any, **kwargs: Any) → None[source]

Extra setup for Elasticsearch

Parameters:

  • es (elasticsearch) – an instance of the elasticsearch python driver

  • index (Optional[string]) – a name for the Elasticsearch index (default ‘images’)

  • timeout (Optional[int]) – how long to wait on an Elasticsearch query, in seconds (default 10)

  • size (Optional[int]) – maximum number of Elasticsearch results (default 100)

  • delete_duplicates_limit (Optional[int]) – maximum number of duplicate candidates scanned per delete_duplicates call (default 10000)

  • minimum_should_match (Optional[int | str]) – require this many word clauses to match — e.g. 2 or “3<75%”; prunes low-overlap candidates on large indexes (default None = match any word)

  • use_filter_context (Optional[bool]) – wrap the word disjunction in a filter context so Elasticsearch skips BM25 scoring entirely; faster on large indexes but the returned ‘score’ is constant (default False)

  • *args (Optional) – Variable length argument list to pass to base constructor

  • **kwargs (Optional) – Arbitrary keyword arguments to pass to base constructor

Examples

>>> from elasticsearch import Elasticsearch
>>> from image_match.elasticsearch_driver import SignatureES
>>> es = Elasticsearch()
>>> ses = SignatureES(es)
>>> ses.add_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
>>> ses.search_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
[
 {'dist': 0.0,
  'id': 'AVM37nMg0osmmAxpPvx6',
  'path': 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg',
  'score': 0.28797293}
]
delete_duplicates(path: str, limit: int | None = None) → None[source]

Delete all but one entries in elasticsearch whose path value is equivalent to that of path.

Parameters:

  • path (string) – path value to compare to those in the elastic search

  • limit (Optional[int]) – maximum number of duplicate candidates to scan; defaults to the instance’s delete_duplicates_limit (default 10000)

delete_image(path: str, limit: int | None = None) → int[source]

Delete all records whose stored path exactly equals path.

The path match query is fuzzy, so hits are filtered to exact matches before deletion — documents with merely similar paths are kept.

Parameters:

  • path (string) – path value to remove entirely from the index

  • limit (Optional[int]) – maximum number of candidates to scan; defaults to the instance’s delete_duplicates_limit (default 10000)

Returns:

the number of documents deleted

insert_records(records: list[dict[str, Any]], refresh_after: bool = False, **kwargs: Any) → int[source]

Bulk-insert pre-made records (see add_images).

Parameters:

  • records (list[dict]) – image records in the format returned by make_record

  • refresh_after (Optional[boolean]) – refresh the index after the bulk request, making records searchable immediately (default False)

  • **kwargs – extra keyword arguments forwarded to helpers.bulk

Returns:

the number of successfully indexed records

insert_single_record(rec: dict[str, Any], refresh_after: bool = False) → None[source]

Insert an image record.

Parameters:

  • rec (dict) – an image record in the format returned by make_record

  • refresh_after (Optional[boolean]) – refresh the index after inserting, making the record searchable immediately (default False)

search_single_record(rec: dict[str, Any], pre_filter: PreFilter = None) → list[dict[str, Any]][source]

Search for a matching image record.

Parameters:

  • rec (dict) – an image record in the format returned by make_record

  • pre_filter (Optional[dict]) – an Elasticsearch filter clause applied before matching (default None)

Returns:

a list of dicts representing matches, filtered by distance_cutoff

image_match.elasticsearch_driver.build_word_query(rec: dict[str, Any], pre_filter: PreFilter = None, minimum_should_match: int | str | None = None, use_filter_context: bool = False) → dict[str, Any][source]

Build the bool/should term query over a record’s simple_word_* fields.

Removes ‘path’ and ‘metadata’ from rec; the caller pops ‘signature’. Stray stored fields (e.g. timestamp) must not leak into the query.

Parameters:

  • rec – an image record in the format returned by make_record

  • pre_filter – an ES filter clause (or list of clauses) applied before matching

  • minimum_should_match – how many word clauses must match (default None = 1)

  • use_filter_context – evaluate the word disjunction in filter context, skipping scoring entirely (the returned _score is then constant)

image_match.elasticsearch_driver.format_hits(hits: list[dict[str, Any]], signature: ndarray, distance_cutoff: float) → list[dict[str, Any]][source]

Compute distances to a signature and return cutoff-filtered hit dicts.

Async

Requires the elasticsearch-async extra.

class image_match.elasticsearch_async_driver.AsyncSignatureES(es: ClientT, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, minimum_should_match: int | str | None = None, use_filter_context: bool = False, *args: Any, **kwargs: Any)[source]

Bases: SignatureDatabaseBase, Generic

Asynchronous Elasticsearch driver for image-match.

API-identical to SignatureES, but every database-touching method is a coroutine and image decoding/signature generation is offloaded to a worker thread so the event loop is never blocked. Suited for embedding in async services such as FastAPI.

Requires the ‘elasticsearch-async’ extra:

pip install image-match[elasticsearch-async]

Examples

>>> from elasticsearch import AsyncElasticsearch
>>> from image_match.elasticsearch_async_driver import AsyncSignatureES
>>> es = AsyncElasticsearch()
>>> ses = AsyncSignatureES(es)
>>> await ses.add_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
>>> await ses.search_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
[
 {'dist': 0.0,
  'id': 'AVM37nMg0osmmAxpPvx6',
  'path': 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg',
  'score': 0.28797293}
]
__init__(es: ClientT, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, minimum_should_match: int | str | None = None, use_filter_context: bool = False, *args: Any, **kwargs: Any) → None[source]

Extra setup for Elasticsearch

Parameters:

  • es (elasticsearch.AsyncElasticsearch) – an async elasticsearch client instance

  • index (Optional[string]) – a name for the Elasticsearch index (default ‘images’)

  • timeout (Optional[int]) – how long to wait on an Elasticsearch query, in seconds (default 10)

  • size (Optional[int]) – maximum number of Elasticsearch results (default 100)

  • delete_duplicates_limit (Optional[int]) – maximum number of duplicate candidates scanned per delete_duplicates call (default 10000)

  • minimum_should_match (Optional[int | str]) – require this many word clauses to match — e.g. 2 or “3<75%”; prunes low-overlap candidates on large indexes (default None = match any word)

  • use_filter_context (Optional[bool]) – wrap the word disjunction in a filter context so Elasticsearch skips BM25 scoring entirely; faster on large indexes but the returned ‘score’ is constant (default False)

  • *args (Optional) – Variable length argument list to pass to base constructor

  • **kwargs (Optional) – Arbitrary keyword arguments to pass to base constructor

async add_image(path: str, img: ImageInput | None = None, bytestream: bool = False, metadata: dict[str, Any] | None = None, *args: Any, **kwargs: Any) → None[source]

Add a single image to the database.

Signature generation runs in a worker thread; only the index request is awaited.

Parameters:

  • path (string) – path or identifier for image. If img=None, then path is assumed to be a URL or filesystem path

  • img (Optional[string]) – usually raw image data. In this case, path will still be stored, but a signature will be generated from data in img (default None)

  • bytestream (Optional[boolean]) – will the image be passed as raw bytes? (default False)

  • metadata (Optional) – any other information you want to include, can be nested (default None)

  • *args – Variable length argument list to pass to insert_single_record

  • **kwargs – Arbitrary keyword arguments to pass to insert_single_record

async add_images(paths: list[ImageInput], metadata: dict[str, Any] | list[dict[str, Any] | None] | None = None, bytestream: bool = False, n_threads: int = 1, chunk_size: int | None = None, **kwargs: Any) → int[source]

Generate signatures for many images and bulk-insert the records.

Signature generation runs in a worker thread (with its own thread pool when n_threads > 1); only the bulk request is awaited.

Parameters:

  • paths (list) – paths, URLs, or raw image data

  • metadata (Optional) – a single dict applied to every record, or a list of dicts/None aligned with paths (default None)

  • bytestream (Optional[boolean]) – inputs are raw image bytes (default False)

  • n_threads (Optional[int]) – signature-generation threads (default 1)

  • chunk_size (Optional[int]) – process and insert in batches of this many images instead of buffering all records in memory (default None = one bulk request)

  • **kwargs – passed to insert_records (e.g. refresh_after — honored on the final chunk only)

Returns:

the number of successfully indexed records

async delete_duplicates(path: str, limit: int | None = None) → None[source]

Delete all but one entries in elasticsearch whose path value is equivalent to that of path.

Parameters:

  • path (string) – path value to compare to those in the elastic search

  • limit (Optional[int]) – maximum number of duplicate candidates to scan; defaults to the instance’s delete_duplicates_limit (default 10000)

async delete_image(path: str, limit: int | None = None) → int[source]

Delete all records whose stored path exactly equals path.

The path match query is fuzzy, so hits are filtered to exact matches before deletion — documents with merely similar paths are kept.

Parameters:

  • path (string) – path value to remove entirely from the index

  • limit (Optional[int]) – maximum number of candidates to scan; defaults to the instance’s delete_duplicates_limit (default 10000)

Returns:

the number of documents deleted

async insert_records(records: list[dict[str, Any]], refresh_after: bool = False, **kwargs: Any) → int[source]

Bulk-insert pre-made records (see add_images).

Parameters:

  • records (list[dict]) – image records in the format returned by make_record

  • refresh_after (Optional[boolean]) – refresh the index after the bulk request, making records searchable immediately (default False)

  • **kwargs – extra keyword arguments forwarded to helpers.async_bulk

Returns:

the number of successfully indexed records

async insert_single_record(rec: dict[str, Any], refresh_after: bool = False) → None[source]

Insert an image record.

Parameters:

  • rec (dict) – an image record in the format returned by make_record

  • refresh_after (Optional[boolean]) – refresh the index after inserting, making the record searchable immediately (default False)

async search_image(path: ImageInput, all_orientations: bool = False, bytestream: bool = False, pre_filter: PreFilter = None, **kwargs: Any) → list[dict[str, Any]][source]

Search for matches.

Image preprocessing and signature generation run in a worker thread; the search requests are awaited.

Parameters:

  • path (string) – path or image data. If bytestream=False, then path is assumed to be a URL or filesystem path. Otherwise, it’s assumed to be raw image data

  • all_orientations (Optional[boolean]) – if True, search for all combinations of mirror images, rotations, and color inversions (default False)

  • bytestream (Optional[boolean]) – will the image be passed as raw bytes? (default False)

  • pre_filter (Optional[dict]) – filters list before applying the matching algorithm (default None)

  • **kwargs – Arbitrary keyword arguments to pass to search_single_record

Returns:

a formatted list of dicts representing unique matches, sorted by dist

async search_single_record(rec: dict[str, Any], pre_filter: PreFilter = None) → list[dict[str, Any]][source]

Search for a matching image record.

Parameters:

  • rec (dict) – an image record in the format returned by make_record

  • pre_filter (Optional[dict]) – an Elasticsearch filter clause applied before matching (default None)

Returns:

a list of dicts representing matches, filtered by distance_cutoff

OpenSearch

Requires the opensearch extra.

class image_match.opensearch_driver.SignatureOpenSearch(es: OpenSearch, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, minimum_should_match: int | str | None = None, use_filter_context: bool = False, *args: Any, **kwargs: Any)[source]

Bases: SignatureES[OpenSearch]

OpenSearch driver for image-match.

Shares the record format and query DSL with SignatureES – OpenSearch is API-compatible with Elasticsearch 7.x for the operations used here. The only differences are in the opensearch-py client API: index() still takes a ‘body’ parameter, refresh is a params string, and ‘timeout’/’size’ are passed through params because opensearch-py reserves those kwargs.

Install the client with the ‘opensearch’ extra:

pip install image-match[opensearch]

Examples

>>> from opensearchpy import OpenSearch
>>> from image_match.opensearch_driver import SignatureOpenSearch
>>> os_client = OpenSearch()
>>> ses = SignatureOpenSearch(os_client)
>>> ses.add_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
>>> ses.search_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
[
 {'dist': 0.0,
  'id': 'AVM37nMg0osmmAxpPvx6',
  'path': 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg',
  'score': 0.28797293}
]
__init__(es: OpenSearch, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, minimum_should_match: int | str | None = None, use_filter_context: bool = False, *args: Any, **kwargs: Any) → None[source]

Extra setup for OpenSearch

Parameters:

  • es (opensearchpy.OpenSearch) – an instance of the opensearch-py client

  • index (Optional[string]) – a name for the OpenSearch index (default ‘images’)

  • timeout (Optional[int]) – how long to wait on an OpenSearch query, in seconds (default 10)

  • size (Optional[int]) – maximum number of OpenSearch results (default 100)

  • delete_duplicates_limit (Optional[int]) – maximum number of duplicate candidates scanned per delete_duplicates call (default 10000)

  • minimum_should_match (Optional[int | str]) – require this many word clauses to match — e.g. 2 or “3<75%”; prunes low-overlap candidates on large indexes (default None = match any word)

  • use_filter_context (Optional[bool]) – wrap the word disjunction in a filter context so OpenSearch skips BM25 scoring entirely; faster on large indexes but the returned ‘score’ is constant (default False)

  • *args (Optional) – Variable length argument list to pass to base constructor

  • **kwargs (Optional) – Arbitrary keyword arguments to pass to base constructor

insert_single_record(rec: dict[str, Any], refresh_after: bool = False) → None[source]

Insert an image record.

Parameters:

  • rec (dict) – an image record in the format returned by make_record

  • refresh_after (Optional[boolean]) – refresh the index after inserting, making the record searchable immediately (default False)

Async

Requires the opensearch-async extra.

class image_match.opensearch_async_driver.AsyncSignatureOpenSearch(es: AsyncOpenSearch, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, minimum_should_match: int | str | None = None, use_filter_context: bool = False, *args: Any, **kwargs: Any)[source]

Bases: AsyncSignatureES[AsyncOpenSearch]

Asynchronous OpenSearch driver for image-match.

Shares the record format and query DSL with AsyncSignatureES. Differs only in the opensearch-py client API: index() takes a ‘body’ parameter, refresh is a params string, and ‘timeout’/’size’ go through params because opensearch-py reserves those kwargs.

Requires the ‘opensearch-async’ extra:

pip install image-match[opensearch-async]

Examples

>>> from opensearchpy import AsyncOpenSearch
>>> from image_match.opensearch_async_driver import AsyncSignatureOpenSearch
>>> os_client = AsyncOpenSearch()
>>> ses = AsyncSignatureOpenSearch(os_client)
>>> await ses.add_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
>>> await ses.search_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
__init__(es: AsyncOpenSearch, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, minimum_should_match: int | str | None = None, use_filter_context: bool = False, *args: Any, **kwargs: Any) → None[source]

Extra setup for OpenSearch

Parameters:

  • es (opensearchpy.AsyncOpenSearch) – an async opensearch-py client instance

  • index (Optional[string]) – a name for the OpenSearch index (default ‘images’)

  • timeout (Optional[int]) – how long to wait on an OpenSearch query, in seconds (default 10)

  • size (Optional[int]) – maximum number of OpenSearch results (default 100)

  • delete_duplicates_limit (Optional[int]) – maximum number of duplicate candidates scanned per delete_duplicates call (default 10000)

  • minimum_should_match (Optional[int | str]) – require this many word clauses to match — e.g. 2 or “3<75%”; prunes low-overlap candidates on large indexes (default None = match any word)

  • use_filter_context (Optional[bool]) – wrap the word disjunction in a filter context so OpenSearch skips BM25 scoring entirely; faster on large indexes but the returned ‘score’ is constant (default False)

  • *args (Optional) – Variable length argument list to pass to base constructor

  • **kwargs (Optional) – Arbitrary keyword arguments to pass to base constructor

async insert_single_record(rec: dict[str, Any], refresh_after: bool = False) → None[source]

Insert an image record.

Parameters:

  • rec (dict) – an image record in the format returned by make_record

  • refresh_after (Optional[boolean]) – refresh the index after inserting, making the record searchable immediately (default False)

k-NN drivers

Search the signature vector directly with approximate nearest neighbours instead of word pre-filtering. See Migrating to the OpenSearch k-NN index.

class image_match.opensearch_knn_driver.SignatureOpenSearchKNN(es: OpenSearch, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, *args: Any, **kwargs: Any)[source]

Bases: SignatureOpenSearch

OpenSearch k-NN driver for image-match.

Stores the same record format as the word drivers, but searches with an approximate nearest-neighbour query (HNSW) directly on the ‘signature’ vector instead of the simple_word_* term disjunction. The k-NN hit list is rescored client-side with the usual normalized distance, so output format and distance_cutoff semantics are identical to SignatureES.

Trade-off vs. the word drivers: HNSW is approximate — it may miss a candidate the word query would have found, and vice versa. ‘size’ acts as the candidate count (k); raise it for better recall.

The index must be created with knn enabled — see knn_index_body() or tools/migrate_to_knn.py for converting an existing index.

Requires the ‘opensearch’ extra:

pip install image-match[opensearch]

Examples

>>> from opensearchpy import OpenSearch
>>> from image_match.opensearch_knn_driver import SignatureOpenSearchKNN, knn_index_body
>>> os_client = OpenSearch()
>>> os_client.indices.create(index="images_knn", body=knn_index_body(648))
>>> ses = SignatureOpenSearchKNN(os_client, index="images_knn")
>>> ses.add_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
>>> ses.search_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
[
 {'dist': 0.0,
  'id': 'AVM37nMg0osmmAxpPvx6',
  'path': 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg',
  'score': 0.28797293}
]
__init__(es: OpenSearch, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, *args: Any, **kwargs: Any) → None[source]

Extra setup for OpenSearch k-NN

Parameters:

  • es (opensearchpy.OpenSearch) – an instance of the opensearch-py client

  • index (Optional[string]) – a name for the OpenSearch index (default ‘images’)

  • timeout (Optional[int]) – how long to wait on an OpenSearch query, in seconds (default 10)

  • size (Optional[int]) – number of ANN candidates (k) fetched per search before distance rescoring (default 100)

  • delete_duplicates_limit (Optional[int]) – maximum number of duplicate candidates scanned per delete_duplicates call (default 10000)

  • *args (Optional) – Variable length argument list to pass to base constructor

  • **kwargs (Optional) – Arbitrary keyword arguments to pass to base constructor

search_single_record(rec: dict[str, Any], pre_filter: PreFilter = None) → list[dict[str, Any]][source]

Search for a matching image record.

Parameters:

  • rec (dict) – an image record in the format returned by make_record

  • pre_filter (Optional[dict]) – an OpenSearch filter clause applied inside the knn query (default None)

Returns:

a list of dicts representing matches, filtered by distance_cutoff

class image_match.opensearch_knn_async_driver.AsyncSignatureOpenSearchKNN(es: AsyncOpenSearch, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, *args: Any, **kwargs: Any)[source]

Bases: AsyncSignatureOpenSearch

Asynchronous OpenSearch k-NN driver for image-match.

Async counterpart of SignatureOpenSearchKNN: every database-touching method is a coroutine, image decoding/signature generation is offloaded to a worker thread, and all_orientations searches run concurrently. Suited for embedding in async services such as FastAPI.

The index must be created with knn enabled — see opensearch_knn_driver.knn_index_body() or tools/migrate_to_knn.py.

Requires the ‘opensearch-async’ extra:

pip install image-match[opensearch-async]

Examples

>>> from opensearchpy import AsyncOpenSearch
>>> from image_match.opensearch_knn_async_driver import AsyncSignatureOpenSearchKNN
>>> os_client = AsyncOpenSearch()
>>> ses = AsyncSignatureOpenSearchKNN(os_client, index="images_knn")
>>> await ses.add_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
>>> await ses.search_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
__init__(es: AsyncOpenSearch, index: str = 'images', timeout: str = '10s', size: int = 100, delete_duplicates_limit: int = 10000, *args: Any, **kwargs: Any) → None[source]

Extra setup for OpenSearch k-NN

Parameters:

  • es (opensearchpy.AsyncOpenSearch) – an async opensearch-py client instance

  • index (Optional[string]) – a name for the OpenSearch index (default ‘images’)

  • timeout (Optional[int]) – how long to wait on an OpenSearch query, in seconds (default 10)

  • size (Optional[int]) – number of ANN candidates (k) fetched per search before distance rescoring (default 100)

  • delete_duplicates_limit (Optional[int]) – maximum number of duplicate candidates scanned per delete_duplicates call (default 10000)

  • *args (Optional) – Variable length argument list to pass to base constructor

  • **kwargs (Optional) – Arbitrary keyword arguments to pass to base constructor

async search_single_record(rec: dict[str, Any], pre_filter: PreFilter = None) → list[dict[str, Any]][source]

Search for a matching image record.

Parameters:

  • rec (dict) – an image record in the format returned by make_record

  • pre_filter (Optional[dict]) – an OpenSearch filter clause applied inside the knn query (default None)

Returns:

a list of dicts representing matches, filtered by distance_cutoff

image_match.opensearch_knn_driver.knn_index_body(dimension: int, engine: str = 'lucene', space_type: str = 'l2', data_type: str = 'float') → dict[str, Any][source]

Build the settings/mappings body for a hybrid k-NN index.

Only ‘signature’ is mapped explicitly (as knn_vector); all other fields rely on dynamic mapping, so ‘simple_word_*’ fields still index as long and the classic word-overlap drivers keep working on the same index.

Parameters:

  • dimension (int) – signature vector length (648 for default n_grid=9)

  • engine (Optional[str]) – knn engine — ‘lucene’ (default, no native deps) or ‘faiss’; ‘nmslib’ is OS2-only and rejected for new indexes on OpenSearch 3

  • space_type (Optional[str]) – vector space — ‘l2’ (default) or ‘cosinesimil’/’innerproduct’/’hamming’ depending on engine

  • data_type (Optional[str]) – knn_vector data type — ‘float’ (default) or ‘byte’ (4x smaller; signatures are int8 so ‘byte’ fits losslessly; requires OpenSearch >= 2.9)

Returns:

a dict suitable for client.indices.create(index=…, body=…)

image_match.opensearch_knn_driver.build_knn_query(signature: Any, k: int, pre_filter: PreFilter = None) → dict[str, Any][source]

Build the knn query over the record’s signature vector.

Parameters:

  • signature – the image signature (list or ndarray)

  • k – number of approximate nearest neighbours to fetch

  • pre_filter – an OpenSearch filter clause (dict) or list of clauses applied inside the knn query

image_match.opensearch_knn_driver.format_knn_hits(hits: list[dict[str, Any]], signature: ndarray, distance_cutoff: float) → list[dict[str, Any]][source]

Rescore knn candidates with normalized_distance and filter by cutoff.

k-NN hits arrive in the same {_id, _score, _source} shape as word-query hits, so rescoring is shared with elasticsearch_driver.format_hits.

MongoDB

Requires the mongo extra.

class image_match.mongodb_driver.SignatureMongo(collection: Collection[dict[str, Any]], *args: Any, **kwargs: Any)[source]

Bases: SignatureDatabaseBase

MongoDB driver for image-match

__init__(collection: Collection[dict[str, Any]], *args: Any, **kwargs: Any) → None[source]

Additional MongoDB setup

Parameters:

  • collection (collection) – a MongoDB collection instance

  • args (Optional) – Variable length argument list to pass to base constructor

  • kwargs (Optional) – Arbitrary keyword arguments to pass to base constructor

Examples

>>> from image_match.mongodb_driver import SignatureMongo
>>> from pymongo import MongoClient
>>> client = MongoClient(connect=False)
>>> c = client.images.images
>>> ses = SignatureMongo(c)
>>> ses.add_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
>>> ses.search_image('https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg')
[
 {'dist': 0.0,
  'id': 'AVM37nMg0osmmAxpPvx7',
  'path': 'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg/687px-Mona_Lisa,_by_Leonardo_da_Vinci,_from_C2RMF_retouched.jpg',
  'score': 0.28797293}
]
delete_image(path: str) → int[source]

Delete all records whose path equals path.

Parameters:

path (string) – path value to remove entirely from the collection

Returns:

the number of documents deleted

index_collection() → None[source]

Index a collection on words.

insert_records(records: list[dict[str, Any]], **kwargs: Any) → int[source]

Bulk-insert pre-made records via insert_many (see add_images).

Creates the word indexes on first insert, like insert_single_record.

Parameters:

  • records (list[dict]) – image records in the format returned by make_record

  • **kwargs – accepted for API parity, ignored

Returns:

the number of inserted records

insert_single_record(rec: dict[str, Any]) → None[source]

Insert an image record, creating the word indexes if needed.

Parameters:

rec (dict) – an image record in the format returned by make_record

search_single_record(rec: dict[str, Any], pre_filter: PreFilter = None, *, n_parallel_words: int | None = 4, word_limit: int | None = None, process_timeout: float | None = None, maximum_matches: int = 1000) → list[dict[str, Any]][source]

Search for a matching image record.

Parameters:

  • rec (dict) – an image record in the format returned by make_record

  • pre_filter (Optional[dict]) – additional query conditions merged into each find query (default None)

  • n_parallel_words (Optional[int]) – number of words to scan in parallel; if None, uses cpu_count() (default 4)

  • word_limit (Optional[int]) – only scan this many words; if None, all N words are used (default None)

  • process_timeout (Optional[float]) – seconds to wait for a worker result before giving up; None waits indefinitely (default None)

  • maximum_matches (Optional[int]) – if a word matches more records than this, the word is considered non-discriminatory and skipped (default 1000)

Returns:

a list of dicts representing unique matches

image_match.mongodb_driver.get_next_match(result_q: Queue[dict[str, Any] | str], word: dict[str, Any], collection: Collection[dict[str, Any]], signature: np.ndarray, cutoff: float = 0.5, max_in_cursor: int = 100, pre_filter: dict[str, Any] | None = None) → None[source]

Given a cursor, iterate through matches

Scans a cursor for word matches below a distance threshold. Exhausts a cursor, possibly enqueuing many matches Note that placing this function outside the SignatureCollection class breaks encapsulation. This is done for compatibility with multiprocessing.

Parameters:

  • result_q (queue.Queue) – a queue in which to queue results

  • word (dict) – {word_name: word_value} dict to scan against

  • collection (collection) – a pymongo collection

  • signature (numpy.ndarray) – signature array to match against

  • cutoff (Optional[float]) – normalized distance limit (default 0.5)

  • max_in_cursor (Optional[int]) – if more than max_in_cursor matches are in the cursor, ignore this cursor; this column is not discriminatory (default 100)

  • pre_filter (Optional[dict]) – additional query conditions merged into the find query (default None)