# US 10,313,693 B2 · Method and apparatus for controlling a degree of compression of a digital image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,313,693 B2 - Document ID: US10313693B2 - Status: Granted - Kind: B2 - Application number: 15/344,079 - Filed: 2016-11-04 - Published: 2017-05-11 - Granted: 2019-06-04 - Earliest claimed: 2015-11-05 - Assignees: Axis AB - Applicants: Axis AB - Inventors: Xing Danielsson Fan; Niclas Danielsson - Web page: https://patentgrep.com/patent/US10313693B2 ## CPC - H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) - G06F 18/22: Matching criteria, e.g. proximity measures - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference - H04N 19/139: Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/182: the unit being a pixel - H04N 19/426: using memory downsizing methods - H04N 19/597: specially adapted for multi-view video sequence encoding - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Described is a method and apparatus for controlling a degree of compression of a digital image, and more specifically to such method and apparatus which receives a stereoscopic digital image, and controls a degree of compression of the digital image based on block matching characteristics between the two images of the stereoscopic digital image. ## Claims ### Claim 1 A method of controlling a degree of compression of a digital image representing a scene, said method comprising: receiving a first digital image and a second digital image representing a first and second view of the scene, respectively, determining a plurality of pairs of matching pixels between the first and second digital images by applying a block matching algorithm on the first and second digital images, wherein each pair of matching pixels is determined by: using the block matching algorithm, comparing a block of pixels in the first digital image to a plurality of blocks of pixels in the second digital image, and for each comparison calculating a difference value, determining a best matching block of pixels among the plurality of blocks of pixels in the second digital image, wherein the best matching block of pixels is the block of pixels which resulted in the lowest calculated difference value, determining the pair of matching pixels as being one pixel from the block of pixels in the first digital image and a corresponding pixel in the best matching block of pixels, for each determined pair of matching pixels, calculating a confidence metric, which defines how reliable the matching of the pair of matching pixels is, wherein the confidence metric is calculated at least in part by comparing the lowest calculated difference value with remaining calculated difference values, determining the digital image representing the scene, wherein the digital image representing the scene is determined to be one of: the first digital image, the second digital image, and a stereo image comprising the first and the second digital image; and encoding the digital image representing the scene by controlling a degree of compression of the digital image representing the scene based on the calculated confidence metrics, and by, for a portion of the digital image representing the scene comprising at least one pixel of a first pair of matching pixels among the pairs of matching pixels: comparing the calculated confidence metric of the first pair of matching pixels to a predetermined threshold value, upon determining that the calculated confidence metric of the first pair of matching pixels is equal to or above the threshold value, controlling the degree of compression of the portion of the digital image representing the scene using a first compression ratio, and upon determining that the calculated confidence metric is below the threshold value, controlling the degree of compression of the portion of the digital image representing the scene using a second compression ratio being higher than the first compression ratio. ### Claim 2 (depends on 1) The method of claim 1, wherein a first and a second pair of matching pixels are determined, wherein the calculating a confidence metric further comprises calculating a first confidence metric defining a matching confidence of the first pair of matching pixels, and a second confidence metric defining a matching confidence of the second pair of matching pixels, wherein the encoding of the digital image representing the scene further comprises: controlling a degree of compression of a first portion of the digital image representing the scene based on the first confidence metric, and controlling a degree of compression of a second portion the digital image representing the scene based on the second confidence metric. ### Claim 3 (depends on 1) The method of claim 1, wherein at least a third and a fourth pair of matching pixels are determined, wherein the calculating a confidence metric further comprises calculating a third confidence metric defining a matching confidence of the third pair of matching pixels, and a fourth confidence metric defining a matching confidence of the fourth pair of matching pixels, wherein the encoding of the digital image representing the scene further comprises: controlling a degree of compression of a third portion of the digital image representing the scene based on a combination of the third and the fourth confidence metric. ### Claim 4 (depends on 3) The method of claim 3, wherein an extent of the third portion of the digital image is based on an algorithm for detecting connected regions among the confidence metrics using a threshold value involving positions in the first and/or second image for a pixel in each of the third and fourth pair of matching pixels. ### Claim 5 (depends on 4) The method of claim 4, wherein the encoding the digital image representing the scene further comprises using a codec utilizing blocks of pixels of the digital image, wherein each block has a shape and a size, wherein the extent of the third portion further is based on shape and size of at least one block of pixels which are comprised in the third region. ### Claim 6 (depends on 1) The method of claim 1, wherein the encoding the digital image representing the scene further comprises using a codec utilizing blocks of pixels of the digital image, wherein for each block of pixels in the digital image representing the scene, the encoding the digital image representing the scene further comprises: controlling a degree of compression of the block of pixels by for each pixel of at least some pixels in the block of pixels, retrieving a confidence metric corresponding to the pixel, and controlling a degree of compression of the block of pixels based on the retrieved confidence metrics of the at least some of the pixels in the block of pixels. ### Claim 7 (depends on 6) The method of claim 6, wherein the controlling the degree of compression of the block of pixels comprises: comparing each retrieved confidence metric with a threshold value, counting the number of confidence metrics being equal to or above the threshold value, and controlling a degree of compression of the block of pixels based on the count. ### Claim 8 (depends on 6) The method of claim 6, wherein the controlling the degree of compression of the block of pixels comprises: defining a compression ratio for the block of pixels by calculating one of: a sum, an average value, a maximum value or a median value of the retrieved confidence metrics. ### Claim 9 (depends on 1) The method of claim 1, wherein the block matching algorithm comprises comparing a block of pixels in the first digital image to a plurality of blocks of pixels in the second digital image, and for each comparison calculating a difference value, wherein a confidence metric is calculated by counting a number of difference values that falls below a threshold difference value. ### Claim 10 A non-transitory computer-readable storage medium including a computer program stored thereon with instructions which when executed by a device having a processor adapt a system to carry out the method of controlling a degree of compression of a digital image representing a scene, said method comprising: receiving a first digital image and a second digital image representing a first and second view of the scene, respectively, determining a plurality of pairs of matching pixels between the first and second digital images by applying a block matching algorithm on the first and second digital images, wherein each pair of matching pixels is determined by: using the block matching algorithm, comparing a block of pixels in the first digital image to a plurality of blocks of pixels in the second digital image, and for each comparison calculating a difference value, determining a best matching block of pixels among the plurality of blocks of pixels in the second digital image, wherein the best matching block of pixels is the block of pixels which resulted in the lowest calculated difference value, determining the pair of matching pixels as being one pixel from the block of pixels in the first digital image and a corresponding pixel in the best matching block of pixels, for each determined pair of matching pixels, calculating a confidence metric, which defines how reliable the matching of the pair of matching pixels is, wherein the confidence metric is calculated at least in part by comparing the lowest calculated difference value with remaining calculated difference values, determining the digital image representing the scene, wherein the digital image representing the scene is determined to be one of: the first digital image, the second digital image, and a stereo image comprising the first and the second digital image; and encoding the digital image representing the scene by controlling a degree of compression of the digital image representing the scene based on the calculated confidence metrics, and by, for a portion of the digital image representing the scene comprising at least one pixel of a first pair of matching pixels among the pairs of matching pixels: comparing the calculated confidence metric of the first pair of matching pixels to a predetermined threshold value, upon determining that the calculated confidence metric of the first pair of matching pixels is equal to or above the threshold value, controlling the degree of compression of the portion of the digital image representing the scene using a first compression ratio, and upon determining that the calculated confidence metric is below the threshold value, controlling the degree of compression of the portion of the digital image representing the scene using a second compression ratio being higher than the first compression ratio. ### Claim 11 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein a first and a second pair of matching pixels are determined, wherein the calculating confidence metrics further comprises calculating a first confidence metric defining a matching confidence of the first pair of matching pixels, and a second confidence metric defining a matching confidence of the second pair of matching pixels, wherein the encoding of the digital image representing the scene further comprises: controlling a degree of compression of a first portion of the digital image representing the scene based on the first confidence metric, and controlling a degree of compression of a second portion the digital image representing the scene based on the second confidence metric. ### Claim 12 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein at least a third and a fourth pair of matching pixels are determined, wherein the calculating confidence metrics further comprises calculating a third confidence metric defining a matching confidence of the third pair of matching pixels, and a fourth confidence metric defining a matching confidence of the fourth pair of matching pixels, wherein the encoding of the digital image representing the scene further comprises: controlling a degree of compression of a third portion of the digital image representing the scene based on a combination of the third and the fourth confidence metric. ### Claim 13 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein an extent of the third portion of the digital image is based on an algorithm for detecting connected regions among the confidence metrics using a threshold value involving positions in the first and/or second image for a pixel in each of the third and fourth pair of matching pixels. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the encoding the digital image representing the scene further comprises using a codec utilizing blocks of pixels of the digital image, wherein each block has a shape and a size, wherein the extent of the third portion further is based on shape and size of at least one block of pixels which are comprised in the third region. ### Claim 15 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the encoding the digital image representing the scene further comprises using a codec utilizing blocks of pixels of the digital image, wherein for each block of pixels in the digital image representing the scene, the encoding the digital image representing the scene further comprises: controlling a degree of compression of the block of pixels by for each pixel of at least some pixels in the block of pixels, retrieving a confidence metric corresponding to the pixel, and controlling a degree of compression of the block of pixels based on the retrieved confidence metrics of the at least some of the pixels in the block of pixels. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the controlling the degree of compression of the block of pixels comprises: comparing each retrieved confidence metric with a threshold value, counting the number of confidence metrics being equal to or above the threshold value, and controlling a degree of compression of the block of pixels based on the count. ### Claim 17 (depends on 15) The non-transitory computer-readable storage medium claim 15, wherein the controlling the degree of compression of the block of pixels comprises: defining a compression ratio for the block of pixels by calculating one of: a sum, an average value, a maximum value or a median value of the retrieved confidence metrics. ### Claim 18 (depends on 10) The non-transitory computer-readable storage medium of claim 10, wherein the block matching algorithm comprises comparing a block of pixels in the first digital image to a plurality of blocks of pixels in the second digital image, and for each comparison calculating a difference value, wherein a confidence metric is calculated by counting a number of difference values that falls below a threshold difference value. ### Claim 19 An encoder for controlling a degree of compression of a digital image representing a scene, the encoder comprising: a receiving component, including a computer-readable communications medium, arranged to receive a first digital image and a second digital image representing a first and second view of the scene, respectively, a matching component, including a processor, arranged to: determine a plurality of pairs of matching pixels between the first and second digital images by applying a block matching algorithm on the first and second digital images, wherein each pair of matching pixels is determined by: using the block matching algorithm, comparing a block of pixels in the first digital image to a plurality of blocks of pixels in the second digital image, and for each comparison calculating a difference value, determining a best matching block of pixels among the plurality of blocks of pixels in the second digital image, wherein the best matching block of pixels is the block of pixels which resulted in the lowest calculated difference value, determining the pair of matching pixels as being one pixel from the block of pixels in the first digital image and a corresponding pixel in the best matching block of pixels, for each determined pair of matching pixels, calculate a confidence metric, which defines how reliable the matching of the pair of matching pixels is, wherein the confidence metric is calculated at least in part by comparing the lowest calculated difference value with remaining calculated difference values, and an encoding component, including the processor and the computer-readable communications medium, arranged to: determine the digital image representing the scene, wherein the digital image representing the scene is determined to be one of: the first digital image, the second digital image, and a stereo image comprising the first and the second digital image; and encode the digital image representing the scene by controlling a degree of compression of the digital image representing the scene based on the calculated confidence metrics, and by, for a portion of the digital image representing the scene comprising at least one pixel of a first pair of matching pixels among the pairs of matching pixels: compare the calculated confidence metric of the first pair of matching pixels to a predetermined threshold value, upon determining that the calculated confidence metric of the first pair of matching pixels is equal to or above the threshold value, control the degree of compression of the portion of the digital image representing the scene using a first compression ratio, upon determining that the calculated confidence metric is below the threshold value, control the degree of compression of the portion of the digital image representing the scene using a second compression ratio being higher than the first compression ratio. ### Claim 20 (depends on 19) A stereo camera comprising the encoder of claim 19 and arranged to capture the first digital image and the second digital image representing the first and second view of the scene. ### Claim 21 (depends on 19) The encoder of claim 19, wherein a first and a second pair of matching pixels are determined, wherein the matching component, including the processor, arranged to calculate a confidence metric is further arranged to calculate a first confidence metric defining a matching confidence of the first pair of matching pixels, and a second confidence metric defining a matching confidence of the second pair of matching pixels, and wherein the encoding component, including the processor and the computer-readable communications medium, arranged to encode the digital image representing the scene is further arranged to: control a degree of compression of a first portion of the digital image representing the scene based on the first confidence metric, and to control a degree of compression of a second portion the digital image representing the scene based on the second confidence metric. ### Claim 22 (depends on 19) The encoder of claim 19, wherein at least a third and a fourth pair of matching pixels are determined, wherein the matching component, including the processor, arranged to calculate a confidence metric is further arranged to calculate a third confidence metric defining a matching confidence of the third pair of matching pixels, and a fourth confidence metric defining a matching confidence of the fourth pair of matching pixels, and wherein the encoding component, including the processor and the computer-readable communications medium, arranged to encode the digital image representing the scene is further arranged to: control a degree of compression of a third portion of the digital image representing the scene based on a combination of the third and the fourth confidence metric. ### Claim 23 (depends on 22) The encoder of claim 22, wherein an extent of the third portion of the digital image is based on an algorithm for detecting connected regions among the confidence metrics using a threshold value involving positions in the first and/or second image for a pixel in each of the third and fourth pair of matching pixels. ### Claim 24 (depends on 23) The encoder of claim 23, wherein the encoding component, including the processor and the computer-readable communications medium, arranged to encode the digital image representing the scene is further arranged to use a codec utilizing blocks of pixels of the digital image, wherein each block has a shape and a size, wherein the extent of the third portion further is based on shape and size of at least one block of pixels which are comprised in the third region. ### Claim 25 (depends on 19) The encoder of claim 19, wherein the encoding component, including the processor and the computer-readable communications medium, arranged to encode the digital image representing the scene is further arranged to use a codec utilizing blocks of pixels of the digital image, and, for each block of pixels in the digital image representing the scene, is further arranged to: control a degree of compression of the block of pixels by for each pixel of at least some pixels in the block of pixels, retrieving a confidence metric corresponding to the pixel, and control a degree of compression of the block of pixels based on the retrieved confidence metrics of the at least some of the pixels in the block of pixels. ### Claim 26 (depends on 25) The encoder of claim 25, wherein to control the degree of compression of the block of pixels comprises: to compare each retrieved confidence metric with a threshold value, counting the number of confidence metrics being equal to or above the threshold value, and to control a degree of compression of the block of pixels based on the count. ### Claim 27 (depends on 25) The encoder of claim 25, wherein to control the degree of compression of the block of pixels comprises: to define a compression ratio for the block of pixels by calculating one of: a sum, an average value, a maximum value or a median value of the retrieved confidence metrics. ### Claim 28 (depends on 19) The encoder of claim 19, wherein the block matching algorithm comprises comparing a block of pixels in the first digital image to a plurality of blocks of pixels in the second digital image, and for each comparison calculating a difference value, wherein a confidence metric is calculated by counting a number of difference values that falls below a threshold difference value.