# US 2019/0318494 A1 · Parallelism in disparity map generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318494 A1 - Document ID: US20190318494A1 - Status: Published application - Kind: A1 - Application number: 16/456,356 - Filed: 2019-06-28 - Published: 2019-10-17 - Earliest claimed: 2019-06-28 - Assignees: Intel Corporation - Inventors: Wei-Yu Tsai; Amit Aneja; Maciej Adam Kaminski; Dhawal Srivastava; Jayaram Puttaswamy; Mithali Shivkumar - Web page: https://patentgrep.com/patent/US20190318494A1 ## CPC - G06T 7/593: from stereo images - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 18/22: Matching criteria, e.g. proximity measures - G06T 7/12: Edge-based segmentation - G06T 7/174: involving the use of two or more images - G06T 7/55: from multiple images - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)} - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06T 2207/10012: Stereo images - G06T 2207/20228: Disparity calculation for image-based rendering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Input images are partitioned into non-overlapping segments perpendicular to a disparity dimension of the input images. Each segment includes a contiguous region of pixels spanning from a first edge to a second edge of the image, with the two edges parallel to the disparity dimension. In some aspects, contiguous input image segments are assigned in a “round robin” manner to a set of sub-images. Each pair of input images generates a corresponding pair of sub-image sets. Semi-global matching processes are then performed on pairs of corresponding sub-images generated from each input image. The SGM processes may be run in parallel, reducing an elapsed time to generate respective disparity sub-maps. The disparity sub-maps are then combined to provide a single disparity map of equivalent size to the original two input images. ## Claims ### Claim 1 A system to generate a disparity map, comprising: hardware processing circuitry; one or more hardware memories storing instructions that when executed, configure the hardware processing circuitry to perform operations comprising: partitioning each of a first input image and a second input image into non-overlapping segments, each segment comprising a contiguous region of pixels from a first edge of the respective image to a second edge of the respective image, each of the first edge and second edge of the respective image parallel to a first dimension of the respective image; generating, based on the segments from the first input image, first sub-images by allocating contiguous segments from the first input image to different first sub-images; generating based on segments from the second input image, second sub-images by allocating contiguous segments from the second input image to different second sub-images; generating first dimension disparity sub-maps based on corresponding pairs of the first and second sub-images, each sub-image in a corresponding pair generated from corresponding segments in a respective input image; and generating a disparity map based on the disparity sub-maps. ### Claim 2 (depends on 1) The system of claim 1, the operations further comprising assigning generation of a first portion of the first-dimension disparity sub-maps to a first processing thread and assigning generation of a second portion of the first-dimension disparity sub-maps to a second processing thread. ### Claim 3 (depends on 2) The system of claim 2, wherein generating the first-dimension disparity sub-maps comprises executing a semi-global matching method. ### Claim 4 (depends on 1) The system of claim 1, wherein generating the first-dimension disparity sub-maps comprises searching a region of a first sub-image of the first sub-images for a first pixel matching a second pixel in a second sub-image of the second sub-images, the region spanning two first segments from the first input image. ### Claim 5 (depends on 1) The system of claim 1, wherein the input images are captured by respective imaging sensors, the method further comprising determining a distance of an object represented by the input images from the imaging sensors based on the disparity map. ### Claim 6 (depends on 5) The system of claim 5, the operations further comprising controlling a vehicle based on the distance. ### Claim 7 (depends on 1) The system of claim 1, the operations further comprising segmenting the input images such that each segment is of constant size in the first dimension. ### Claim 8 (depends on 1) The system of claim 1, wherein each sub-image in the first sub-images is generated by round robin allocating contiguous segments from the first input image to different sub-images of the first sub-images. ### Claim 9 (depends on 1) The system of claim 1, wherein generating the disparity map comprising generating contiguous segments of the disparity map from different disparity sub-maps. ### Claim 10 (depends on 1) The system of claim 1, wherein the disparity map is generated such that disparity values in the disparity map are based on pixel values in an input image having equivalent coordinates. ### Claim 11 A method of generating a disparity map with hardware processing circuitry, comprising: partitioning each of a first input image and a second input image into non-overlapping segments, each segment comprising a contiguous region of pixels from a first edge of the respective image to a second edge of the respective image, each of the first edge and second edge of the respective image parallel to a first dimension of the respective image; generating, based on the segments from the first input image, first sub-images by allocating contiguous segments from the first input image to different first sub-images; generating based on segments from the second input image, second sub-images by allocating contiguous segments from the second input image to different second sub-images; generating first dimension disparity sub-maps based on corresponding pairs of the first and second sub-images, each sub-image in a corresponding pair generated from corresponding segments in a respective input image; and generating a disparity map based on the disparity sub-maps. ### Claim 12 (depends on 11) The method of claim 11, further comprising assigning generation of a first portion of the first-dimension disparity sub-maps to a first processing thread and assigning generation of a second portion of the first-dimension disparity sub-maps to a second processing thread. ### Claim 13 (depends on 12) The method of claim 12, wherein generating the first-dimension disparity sub-maps comprises executing a semi-global matching method. ### Claim 14 (depends on 11) The method of claim 11, wherein generating the first-dimension disparity sub-maps comprises searching a region of a first sub-image of the first sub-images for a first pixel matching a second pixel in a second sub-image of the second sub-images, the region spanning two first segments from the first input image. ### Claim 15 (depends on 11) The method of claim 11, wherein the input images are captured by respective imaging sensors, the method further comprising determining a distance of an object represented by the input images from the imaging sensors based on the disparity map. ### Claim 16 (depends on 11) The method of claim 11, further comprising segmenting the input images such that each segment is of constant size in the first dimension. ### Claim 17 (depends on 11) The method of claim 11, wherein the first sub-images are generated by round robin allocating contiguous segments from the first input image to different sub-images of the first sub-images. ### Claim 18 (depends on 11) The method of claim 11, wherein generating the disparity map comprising generating contiguous segments of the disparity map from different disparity sub-maps. ### Claim 19 (depends on 11) The method of claim 11, wherein the disparity map is generated such that disparity values in the disparity map are based on pixel values in an input image having equivalent coordinates. ### Claim 20 A computer readable storage medium comprising instructions that when executed configure hardware processing circuitry to perform operations to generate a disparity map, the operations comprising: partitioning each of a first input image and a second input image into non-overlapping segments, each segment comprising a contiguous region of pixels from a first edge of the respective image to a second edge of the respective image, each of the first edge and second edge of the respective image parallel to a first dimension of the respective image; generating, based on the segments from the first input image, first sub-images by allocating contiguous segments from the first input image to different first sub-images; generating based on segments from the second input image, second sub-images by allocating contiguous segments from the second input image to different second sub-images; generating first dimension disparity sub-maps based on corresponding pairs of the first and second sub-images, each sub-image in a corresponding pair generated from corresponding segments in a respective input image; and generating a disparity map based on the disparity sub-maps. ### Claim 21 (depends on 20) The computer readable storage medium of claim 20, further comprising assigning generation of a first portion of the first-dimension disparity sub-maps to a first processing thread and assigning generation of a second portion of the first-dimension disparity sub-maps to a second processing thread. ### Claim 22 (depends on 21) The computer readable storage medium of claim 21, wherein generating the first-dimension disparity sub-maps comprises executing a semi-global matching method. ### Claim 23 (depends on 20) The computer readable storage medium of claim 20, wherein generating the first-dimension disparity sub-maps comprises searching a region of a first sub-image of the first sub-images for a first pixel matching a second pixel in a second sub-image of the second sub-images, the region spanning two first segments from the first input image. ### Claim 24 (depends on 20) The computer readable storage medium of claim 20, wherein the first sub-images are generated by round robin allocating contiguous segments from the first input image to different sub-images of the first sub-images.