# US 2021/0232215 A1 · Stereo correspondence search

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0232215 A1
- Document ID: US20210232215A1
- Status: Published application
- Kind: A1
- Application number: 16/769,148
- Filed: 2018-12-03
- Published: 2021-07-29
- Earliest claimed: 2015-03-21
- Assignees: Mine One GmbH
- Applicants: MINE ONE GmbH
- Inventors: JAMES A. MCCOMBE; CHRISTOPH BIRKHOLD
- Web page: https://patentgrep.com/patent/US20210232215A1

## CPC

- H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06T 7/50: Depth or shape recovery
- G06T 7/593: from stereo images
- G06T 11/00: Two-dimensional [2D] image generation
- G06V 10/40: Extraction of image or video features
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- H04N 13/243: using three or more two-dimensional [2D] image sensors
- H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects
- H04N 13/344: with head-mounted left-right displays
- H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio
- G06T 2207/10048: Infrared image
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Methods, systems, devices and computer software/program code products enable efficiently finding stereo correspondence between a feature or set of features in a first image or signal, and a search domain in a second image or signal.

## Claims

### Claim 1

A method of searching for correspondence between a first selected feature in a first image and an element within a selected search domain in a second image, the method comprising:
in a digital processing resource comprising a digital processor:
A) utilizing a multi-level aggregate digital data structure representative of the second image, the multi-level aggregate digital data structure comprising digital data representations at different resolutions;
B) comparing a relatively lower resolution digital data representation of the first selected feature against a relatively lower resolution digital data representation of a first selected subset of the selected search domain in the multi-level aggregate data structure, the comparing comprising computation of a minimum difference value and a maximum difference value:
C) iteratively executing the comparing (B) against other subsets of the selected search domain;
D) identifying a threshold difference value based on maximum difference values from the comparisons against subsets of the selected search domain; and
E) rejecting subsets for which the comparison results in minimum difference values that exceed the identified threshold difference value.

### Claim 2 (depends on 1)

The method of claim 1 wherein the computation of a minimum difference value and a maximum difference value comprises computation of an estimated minimum difference value and an estimated maximum difference value.

### Claim 3 (depends on 1)

The method of claim 1 further comprising:
iteratively executing the comparing (B), the comparing (C), the identifying (D), and the rejecting (E), using higher resolution digital data representations of the first selected feature and higher resolution digital data representations in the multi-level aggregate structure.

### Claim 4 (depends on 1)

The method of claim 1 wherein the first and second images are images captured by at least one camera from different viewpoints of substantially the same scene.

### Claim 5 (depends on 4)

The method of claim 4 in which the selected search domain is selected based on the relative positions of the respective viewpoints of the first and second images.

### Claim 6 (depends on 4)

The method of claim 4 in which the first and second images are rectified to a common projection such that selected search domain within the second image is oriented along a selected axis of the second image.

### Claim 7

A method for determining disparity between a first image and a second image, the method comprising:
in a digital processing resource comprising a digital processor:
A) generating a multi-level aggregate digital data structure representative of the second image, the multi-level aggregate digital data structure comprising digital data representations at different resolutions;
B) selecting a first feature from the first image;
C) searching for a corresponding element within the second image, utilizing a selected search method, the corresponding element having correspondence with the selected feature from the first image;
D) based on results of the search, recording correspondence information relating to correspondence between the selected feature and the corresponding element;
E) iteratively executing the selecting (B), the searching (C) and the recording (D), using other selected features from the first image, to construct a digital map of correspondences for the selected features from the first image.

### Claim 8 (depends on 7)

The method of claim 7 wherein the selected search method comprises:
(i) comparing a relatively lower resolution digital data representation of the first selected feature against a relatively lower resolution digital data representation of a first selected subset of a selected search domain in the multi-level aggregate data structure, the comparing comprising computation of a minimum difference value and a maximum difference value;
(ii) iteratively executing the comparing (i) against other subsets of the selected search domain;
(iii) identifying a threshold difference value based on maximum difference values from the comparisons against subsets of the selected search domain; and
(iv) rejecting subsets for which the comparison results in minimum difference values that exceed the identified threshold difference value.

### Claim 9 (depends on 8)

The method of claim 8 wherein the selected search method further comprises:
iteratively executing the comparing (i), the comparing (ii), the identifying (iii), and the rejecting (iv), using higher resolution digital data representations of the first selected feature and higher resolution digital data representations in the multi-level aggregate structure; and
outputting correspondence information associated with the subset of the search domain for which the difference value is lowest.

### Claim 10 (depends on 9)

The method of claim 9 wherein the correspondence information is a transform.

### Claim 11 (depends on 9)

The method of claim 9 wherein the correspondence information is an offset representing a position within the search domain.

### Claim 11 (depends on 9)

The method of claim 9 wherein the correspondence information is computed based on a known difference in viewpoints between first and second cameras that captured the first and the second image, respectively.

### Claim 12 (depends on 9)

The method of claim 9 wherein the correspondence information is a 2-dimensional value representing a position within an image.

### Claim 14 (depends on 9)

The method of claim 9 wherein the correspondence information is a disparity value representing the distance between the location of the first selected feature in the first image and the location of the corresponding element in the second image.

### Claim 15 (depends on 8)

The method of claim 8 where the feature is a pixel.

### Claim 16 (depends on 8)

The method of claim 8 where the feature is a matrix of pixels.

### Claim 17 (depends on 16)

The method of claim 16 wherein the matrix of pixels is a structure of adjacent pixels.

### Claim 18 (depends on 1, 3, 7, 8, 9)

The method of claim 1, 3, 7, 8 or 9 wherein the multi-level aggregate data structure contains elements; each lower resolution element comprising the minimum maximum values for one or more higher resolution elements.

### Claim 19 (depends on 7)

The method of claim 7 further comprising:
in a digital processing resource comprising a digital processor:
A) comparing a digital data representation of the selected feature from the first image, at a given resolution, against a digital data representation of a first selected subset of a selected search domain in the second image, at a given resolution, in the multi-level aggregate digital data structure,
B) determining, based on the comparing, a variance value for the first selected subset; and
C) utilizing the variance value to determine whether to execute another comparing operation in a higher resolution digital data representation in the multi-level aggregate digital data structure, wherein, if the determined variance value is less than a selected threshold value, then not executing another comparing operation for the first selected subset in a higher resolution digital data representation in the multi-level aggregate digital data structure.

### Claim 20 (depends on 1, 3, 7, 8, 9)

The method of claim 1, 3, 7, 8 or 9 wherein the multi-level aggregate digital data structure has a configuration substantially similar to the configuration of a MIP map.

### Claim 21 (depends on 20)

The method of claim 20 wherein the comparing utilizes a selected error function.

### Claim 22 (depends on 20)

The method of claim 20 wherein the comparing utilizes a selected comparison function.

### Claim 23 (depends on 22)

The method of claim 22 wherein the comparison function comprises a kernel.

### Claim 24 (depends on 21)

The method of claim 21 wherein applying the error function in executing the comparing produces a scalar error value for a given transform.

### Claim 25 (depends on 21)

The method of claim 21 wherein the error function comprises a comparison function and selected logic applicable to weight results of individual comparisons.

### Claim 26 (depends on 20)

The method of claim 20 wherein the second image is pre-processed to generate a multi-level aggregate digital data structure, in which each element in a given resolution level subsumes the minimum and maximum values from the resolution level above the given resolution level.

### Claim 27 (depends on 26)

The method of claim 26 wherein the multi-level aggregate digital data structure is characterized by a selected reduction ratio.

### Claim 28 (depends on 1, 3, 7, 8, 9)

The method of claim 1, 3, 7, 8 or 9 wherein only selected resolution levels of the multi-level aggregate digital data structure are created in an initial structure generating operation.

### Claim 29 (depends on 28)

The method of claim 28 wherein only selected lowest resolution levels of the multi-level aggregate digital data structure are generated in an initial structure generating operation.

### Claim 30 (depends on 27)

The method of claim 27 wherein all levels of the multi-level aggregate digital data structure projected to be required in executing the comparing are generated in an initial structure generating operation.

### Claim 31

A method of searching for correspondence between a first selected feature in an input signal and an element within a selected search domain in a search signal, the method comprising:
in a digital processing resource comprising a digital processor:
A) utilizing a multi-level aggregate digital data structure representative of the search signal, the multi-level aggregate digital data structure comprising digital data representations at different resolutions;
B) comparing a relatively lower resolution digital data representation of the first selected feature against a relatively lower resolution digital data representation of a first selected subset of the selected search domain in the multi-level aggregate data structure, the comparison comprising computation of a minimum difference value and a maximum difference value;
C) iteratively executing the comparing (B) against other subsets of the selected search domain;
D) identifying a threshold difference value based on maximum difference values from the comparisons against subsets of the selected search domain; and
E) rejecting subsets for which the comparison results in minimum difference values that exceed the identified threshold difference value.

### Claim 32 (depends on 31)

The method of claim 31 further comprising:
iteratively executing the comparing (B), the comparing (C), the identifying (D), and the rejecting (E), using higher resolution digital data representations of the first selected feature and higher resolution digital data representations in the multi-level aggregate structure, and rejecting comparison results having difference values above the threshold difference value.

### Claim 33 (depends on 31)

The method of claim 31 wherein the input signal and the search signal are 1-dimensional signals.

### Claim 34 (depends on 31)

The method of claim 31 wherein the input signal and the search signal are of different formats.

### Claim 35 (depends on 31)

The method of claim 31 wherein either the input signal or the search signal comprises any of: image data, pixel depth data, 3-dimensional point data, 3-dimensional vertex data, LIDAR data, neural network weights, computer readable instructions, abstract signal descriptions, compressed representations, audio data, market data, price signal data, or analytics.

### Claim 36 (depends on 34)

The method of claim 34 wherein either the input signal or the search signal comprises a combination of any of: image data, pixel depth data, 3-dimensional point data, 3-dimensional vertex data, LIDAR data, neural network weights, computer readable instructions, abstract signal descriptions, compressed representations, audio data, market data, price signal data, or analytics.

### Claim 37 (depends on 34)

The method of claim 34 wherein one of the input signal or the search signal comprises an abstract representation describing features of the given signal, and the other signal comprises image data.

### Claim 38

A variance-aware method of searching for correspondence between a first selected feature in a first image and an element within a selected search domain in a second image, the method comprising:
in a digital processing resource comprising a digital processor:
A) utilizing a multi-level aggregate digital data structure representative of the second image, the multi-level aggregate digital data structure comprising digital data representations at different resolutions;
B) comparing a digital data representation of the first selected feature, at a given resolution, against a digital data representation of a first selected subset of the selected search domain, at a given resolution, in the multi-level aggregate data structure:
C) determining, based on the comparing, a variance value for the first selected subset and
D) utilizing the variance value to determine whether to execute another comparing operation in a higher resolution digital data representation in the multi-level aggregate structure, wherein, if the determined variance value is less than a selected threshold value, then not executing another comparing operation for the first selected subset in a higher resolution digital data representation in the multi-level aggregate structure.

### Claim 39 (depends on 38)

The method of claim 38 in which determining a variance value comprises: utilizing a deviation calculation.

### Claim 40 (depends on 38)

The method of claim 38 in which determining a variance value comprises: utilizing a squared deviation calculation.

### Claim 41 (depends on 38)

The method of claim 38 in which determining a variance value comprises: calculating the variance based on a running standard deviation.

### Claim 42 (depends on 38)

The method of claim 38 in which determining a variance value comprises: utilizing a heuristic.

### Claim 43 (depends on 38)

The method of claim 38 in which the comparing is executed for a plurality of subsets of the selected search domain, and the determining comprises utilizing the results of the comparing.

### Claim 44

A variance-aware method of searching for correspondence between a first selected feature in an input signal and an element within a selected search domain in a search signal, the method comprising:
in a digital processing resource comprising a digital processor:
A) utilizing a multi-level aggregate digital data structure representative of the search signal, the multi-level aggregate digital data structure comprising digital data representations at different resolutions;
B) comparing digital data representation of the first selected feature, at a given resolution against a digital data representation of a first selected subset of the selected search domain, at a given resolution, in the multi-level aggregate data structure;
C) determining, based on the comparing, a variance value for the first selected subset; and
D) utilizing the variance value to determine whether to execute another comparing operation in a higher resolution digital data representation in the multi-level aggregate structure, wherein, if the determined variance value is less than a selected threshold value, then not executing another comparing operation for the first selected subset in a higher resolution digital data representation in the multi-level aggregate structure.

### Claim 45 (depends on 44)

The method of claim 44 wherein the variance is the absolute value of the difference between the minimum and maximum values contained in a lower resolution dement of the multi-level aggregate digital data structure.

### Claim 46 (depends on 45)

The method of claim 45 further comprising: selecting a threshold for variance below which corresponding image detail is treated as irrelevant.

### Claim 47 (depends on 45)

The method of claim 45 wherein the variance threshold is a selected fixed value.

### Claim 48 (depends on 46, 47)

The method of claim 46 or 47 wherein the variance threshold is determined, in the processing structure, based on parameters of the images or of the devices or systems from which the images originate.

### Claim 49 (depends on 48)

The method of claim 48 wherein the variance threshold is determined based on physical properties of cameras and camera processing elements associated with a stereo image disparity search.

### Claim 50 (depends on 46)

The method of claim 46 wherein the variance threshold is determined based on a selected model.

### Claim 51 (depends on 50)

The method of claim 50 wherein the model alters the variance threshold based on selected criteria.

### Claim 52 (depends on 51)

The method of claim 51 wherein the selected criteria comprise any of absolute signal values contained within a currently selected element of a can selected multi-level aggregate digital data structure; overall image-representative values across larger sections of an image being searched; available processing power; human perceptual factors: display technology considerations; relative and absolute image positions; inputs from additional connected components; or user-supplied inputs or hints.

### Claim 53 (depends on 45)

The method of claim 45 further comprising: adjusting the variance threshold based on availability of computational or processing power or resources.

### Claim 54 (depends on 45)

The method of claim 45 further comprising: adjusting the variance threshold to selectively prioritize execution of computational operations.

### Claim 55 (depends on 54)

The method of claim 54 wherein multi-level aggregate digital data structure elements with highest variance are designated as candidates for further refinement based on higher resolution versions of a selected image.

### Claim 56 (depends on 55)

The method of claim 55 further comprising: iteratively repeating the designating, as variances associated with all transforms across a given image being processed become smaller, such that correlations between images are refined as available computational or processing power permits.

### Claim 57

A method of executing a lossless search for correspondence between a first selected feature in a first image and an element within a selected search domain in a second image, the method comprising:
in a digital processing resource comprising a digital processor:
A) utilizing a first multi-level aggregate digital data structure representative of the selected feature, the first multi-level aggregate digital data structure comprising, digital data representations at different resolutions;
B) utilizing a second multi-level aggregate digital data structure representative of a first selected subset of the selected search domain, the second multi-level aggregate digital data structure comprising digital data representations at different resolutions;
C) comparing a digital data representation of the selected feature, at a given resolution, to a digital data representation of the first selected subset of the selected search domain, at a given resolution, and computing, based on the comparing, minimum and maximum error for each eligible data element of the digital data representation at the given resolution in the multi-level aggregate digital data structure, wherein an data element is considered to be ineligible if a subsuming data dement in a corresponding lower resolution was indicated to be ineligible;
D) and if the currently executed comparison was at a highest possible resolution, then determining the lowest value for the error, wherein (1) the error determination can incorporate error values from multiple resolutions of the multi-level aggregate digital data structure, and (2) the result of the search is a transform that produces the lowest error.

### Claim 58 (depends on 57)

The method of claim 57 wherein features are selected from a multi-level aggregate digital data structure representative of the first image; and if the currently executed comparison was not at the highest possible resolution, then accessing a next higher resolution digital data representation in the multi-level aggregate digital data structure representative of the first image to execute a subsequent comparison.

### Claim 59 (depends on 58)

The method of claim 58 wherein, in executing a lossless search, aggregate downsampled versions of the first image, represented by a currently selected multi-level aggregate digital data structure, represent the bounds of the higher resolution first image elements.

### Claim 60 (depends on 59)

The method of claim 59 wherein the aggregate downsampled versions of the first image represent the hounds of the higher resolution first image elements in a manner analogous to the respective multi-level aggregate digital data structure representing the bounds of the second image.

### Claim 61 (depends on 59)

The method of claim 59 wherein the executed comparison function considers two ranges, wherein the first range is the range of subsumed values from each element in the first image and the second range is the range of subsumed values from each element in the second image.

### Claim 62 (depends on 61)

The method of claim 61 wherein, if the first and second ranges are disjoint, the minimum value for the error is based on the distance between the nearest points in each range.

### Claim 63 (depends on 61)

The method of claim 61 wherein, if the first and second ranges overlap, the minimum error can be zero, and the maximum error can be based on the distance between the two points in each range that are furthest from each other.

### Claim 64

A method of searching for a correspondence between a first selected feature in a first image and an element within a selected search domain in a second image, the method comprising:
in a digital processing resource comprising a digital processor:
comparing a digital data representation of the selected feature from the first image against a digital data representation of a selected subset of the selected search domain in the second image;
determining, for a given comparison, an error value corresponding to the given comparison, the error value being representative of the degree to which the selected feature from the first image corresponds with the selected subset of the selected search domain in the second image when the comparing is executed in accordance with selected processing parameters; and
iteratively executing the comparing and determining, while applying selected changes to the processing parameters across the selected subset of the selected search domain, until processing parameters resulting in a smallest error value are found; and
estimating minimum and maximum possible errors for correspondence for the selected feature in the first image at a full resolution, by executing the comparing and determining at reduced resolution on multi-level aggregate digital data structures representative of the first image and the selected subset of the selected search domain in the second image, respectively.

### Claim 65 (depends on 64)

The method of claim 64 executed to determine a transform applicable to the selected feature in the first image such that the selected feature in the first image is matchable to the selected subset of the selected search domain of the second image with a level of error lower than a selected error threshold, the determined transform corresponding to the processing parameters resulting in the smallest error value.

### Claim 66

A method of searching for a correspondence between a first selected feature in a first image and an element within a selected search domain in a second image, the method comprising;
in a digital processing resource comprising a digital processor:
comparing a digital data representation of the selected feature from the first image against a digital data representation of a selected subset of the selected search domain in the second image;
determining, for a given comparison, an error value corresponding to the given comparison, the error value being representative of the degree to which the selected feature from the first image corresponds with the selected subset of the selected search domain in the second image when the comparing is executed in accordance with selected processing parameters; and
iteratively executing the comparing and determining, while applying selected changes to the processing parameters across the selected subset of the selected search domain, until processing parameters resulting in a smallest error value are found; and
utilizing, in executing the comparing, a multi-level aggregate digital data structure containing digital data representations at different resolution levels.

### Claim 67 (depends on 66)

The method of claim 66 executed to determine a transform applicable to the selected feature in the first image such that the selected feature in the first image is matchable to the selected subset of the selected search domain of the second image with a level of error lower than a selected error threshold; wherein the determined transform corresponding to the processing parameters resulting in the smallest error value.

### Claim 68 (depends on 66)

The method of claim 66 wherein each resolution level contains at least one element, and in which a given element at a given resolution level can reference both minimum and maximum values of other demerits subsumed within the given element at higher resolution levels of the multi-level aggregate digital data structure.

### Claim 69 (depends on 1, 3, 7, 8, 9, 68)

The method of claim 1, 3, 7, 8, 9 or 68, further comprising:
in a digital processing structure:
applying correspondence searching, operations to multi-dimensional data.

### Claim 70 (depends on 69)

The method of claim 69 wherein the image data has n dimensions, and n is greater than or equal to 2.

### Claim 71 (depends on 70)

The method of claim 70 wherein applying correspondence searching operations to multi-dimensional image data comprises:
in a digital processing structure:
searching a multi-dimensional image data field using a selected comparison kernel.

### Claim 72 (depends on 71)

The method of claim 71 wherein using a selected comparison kernel comprises:
evaluating the selected comparison kernel; then, based on the evaluation, associating, with a representative element of the kernel, a designation of eligibility and then utilizing the designation of eligibility to determine whether the representative element, and any contained higher resolution elements, should be rejected from further consideration.

### Claim 73 (depends on 72)

The method of claim 72 further comprising, if a selected relatively lower resolution element is designated ineligible, rejecting from further consideration the selected lower resolution element and rejecting any corresponding subsumed higher resolution elements.

### Claim 74 (depends on 72)

The method of claim 72 further comprising: if a selected relatively lower resolution element is designated eligible, executing comparison operations on contained higher resolution elements.

### Claim 75 (depends on 74)

The method of claim 74 further comprising:
in a digital processing structure;
expressing an n-dimensional image data field in a form comprising fewer than n dimensions, by applying a dimension collapsing operation.

### Claim 76 (depends on 75)

The method of claim 75 wherein applying a dimension collapsing operation comprises executing a search characterized by a selected search pattern.

### Claim 77 (depends on 76)

The method of claim 76 wherein the search pattern comprises a scan line search.

### Claim 78 (depends on 38, 44)

The method of any of claim 38 or 44 further comprising: executing a search characterized by a selected search pattern, wherein the search pattern comprises a scan line search.

### Claim 79 (depends on 1, 7, 31, 57)

The method of any of claim 1, 7, 31 or 57 further comprising: executing a search characterized by a selected search pattern, wherein the search pattern comprises a scan line search.

### Claim 80 (depends on 76)

The method of claim 76 wherein the search pattern comprises a Hilbert curve.

### Claim 81 (depends on 76)

The method of claim 76 wherein the search pattern comprises a Morton order.

### Claim 82 (depends on 1, 81)

The method of claims 1 or 81 wherein successive search operations are executed in multiple passes through data to be searched.

### Claim 83 (depends on 82)

The method of claim 82 wherein, in one pass, evaluation of a comparison kernel finds a minimum and maximum error for the kernel.

### Claim 84 (depends on 83)

The method of claim 83 wherein a subsequent pass comprises determining a minimum value of the maximum error for the kernel, to be used as a threshold when determining which elements are eligible for further consideration.

### Claim 85 (depends on 84)

The method of claim 84 wherein a further subsequent pass comprises comparing all eligible elements at a next resolution level.

### Claim 86 (depends on 1, 3, 7, 8, 9, 31, 38)

The method of claims 1, 3, 7, 8, 9, 31, 38 44, 57, 64 wherein:
in a digital processing structure:
the first image, representing an input image, is processed to create a first multi-level aggregate digital data structure;
the second image, representing, a search image, is processed to create a second multi-level aggregate digital data structure; and
features from the input image and associated first multi-level aggregate digital data structure are searched within the search image and associated second multi-level aggregate digital data structure by evaluating a selected comparison kernel in parallel against as much of the search image as possible.

### Claim 87 (depends on 86)

The method of claim 86 further comprising: optimizing execution of processing operations to substantially fully utilize all available digital processing resources.

### Claim 88 (depends on 86)

The method of claim 86 further comprising:
testing a selected feature of the input image against a selected search domain in the second image, and generating, based on the testing, resultant error values;
and if the resultant error is value is within a selected threshold designated for further processing, then continuing processing at a next resolution level of the associated multi-level aggregate digital data structure.

### Claim 89 (depends on 88)

The method of claim 88 further comprising optimizing execution of processing operations to minimize computational work and memory traffic.

### Claim 90 (depends on 89)

The method of claim 89 wherein multiple features of the input image are tested in parallel against the selected search domain.

### Claim 91 (depends on 90)

The method of claim 90 wherein processing operations are executed in accordance with digital instructions executing on a computing device.

### Claim 92 (depends on 91)

The method of claim 91 wherein the computing device comprises a GPU (General Processing Unit).

### Claim 93 (depends on 91)

The method of claim 91 wherein the computing device comprises a CPU (Central Processing Unit).

### Claim 94 (depends on 91)

The method of claim 91 wherein the computing device comprises a TPU (Tensor Processing Unit).

### Claim 95 (depends on 90)

The method of claim 90 wherein processing operations are executed by dedicated hardware circuitry.

### Claim 96 (depends on 90)

The method of claim 90 wherein data generated for use in a given processing operation are retained for future use.

### Claim 97 (depends on 90)

The method of claim 90 wherein data for use in a given processing operation are generated on an as-needed basis.

### Claim 98 (depends on 86)

The method of claim 86 further comprising:
generating a first multi-level aggregate digital data structure representative of the input image and a second multi-level aggregate digital data structure representative of the search image;
selecting a first feature from the input image to locate within a selected search domain in the search image and executing a search operation for the selected first feature; and
selecting an additional feature from the generated multi-level aggregate digital data structures and executing a search operation for the additional feature.

### Claim 99 (depends on 98)

The method of claim 98 wherein the search operation for the additional feature is executed after execution of the search operation for the selected first feature.

### Claim 100 (depends on 98)

The method of claim 98 wherein the search operation for the additional feature is executed in parallel with the search operation for the selected first feature.

### Claim 101 (depends on 86)

The method of claim 86 wherein the result of a given search operation is a transform indicating a best correspondence between a feature from the input image and a best match for the input image feature in the search image.

### Claim 102 (depends on 101)

The method of claim 101 wherein the transform is invertible to generate a corresponding transform between a corresponding feature in the search image and a corresponding feature in the input image.

### Claim 103 (depends on 86)

The method of claim 86 further comprising: selecting a first portion of a given image to be a search image, and selecting a second, different portion of the given image to be an input image.

### Claim 104 (depends on 86)

The method of claim 86 further comprising: applying comparison and search processes across a plurality of images simultaneously, such that one input image is simultaneously compared for correspondence with more than one search image.

### Claim 105 (depends on 86)

The method of claim 86 further comprising: applying comparison and search processes across a plurality of images simultaneously, such that multiple input images are simultaneously compared for correspondence with a given search image.

### Claim 106 (depends on 86)

The method of claim 86 wherein the input image differs in format from the search image.

### Claim 107 (depends on 106)

The method of claim 106 further comprising: executing a transform to enable comparison between an input image and a search image of different formats.

### Claim 108 (depends on 107)

The method of claim 107 wherein the applied comparison function adaptively interprets different respective sets of data associated with each image, to enable comparison between an input image and a search image of different formats.

### Claim 109

A system operable to search for correspondence between a first selected feature in a first image and an element within a selected search domain in a second image, the system comprising:
a digital processing resource comprising a digital processor, the digital processing resource being configured to:
A) utilize a multi-level aggregate digital data structure representative of the second image, the multi-level aggregate digital data structure comprising digital data representations at different resolutions;
B) compare a relatively lower resolution digital data representation of the first selected feature against a relatively lower resolution digital data representation of a first selected subset of the selected search domain in the multi-level aggregate data structure, the comparing comprising computation of a minimum difference value and a maximum difference value;
C) iteratively execute the comparing (B) against other subsets of the selected search domain;
D) identify a threshold difference value based on maximum difference values from the comparisons against subsets of the selected search domain; and
E) reject subsets for which the comparison results in minimum difference values that exceed the identified threshold difference value.

### Claim 110

A program product for use with a digital processing system to enable the digital processing system to search for correspondence between a first selected feature in a first image and an element within a selected search domain in a second image, the digital processing system comprising a digital processing resource, the digital processing resource comprising at least one digital processor, the program product comprising digital processor-executable program instructions stored on a non-transitory digital processor-readable medium, which when executed in the digital processing resource cause the digital processing resource to:
A) utilize a multi-level aggregate digital data structure representative of the second image, the multi-level aggregate digital data structure comprising digital data representations at different resolutions;
B) compare a relatively lower resolution digital data representation of the first selected feature against a relatively lower resolution digital data representation of a first selected subset of the selected search domain in the multi-level aggregate data structure, the comparing comprising computation of a minimum difference value and a maximum difference value:
C) iteratively execute the comparing (B) against other subsets of the selected search domain;
D) identify a threshold difference value based on maximum difference values from the comparisons against subsets of the selected search domain; and
E) reject subsets for which the comparison results in minimum difference values that exceed the identified threshold difference value.
