# US 11,443,521 B2 · Method and device for evaluating images, operational assistance method and operating device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,521 B2
- Document ID: US11443521B2
- Status: Granted
- Kind: B2
- Application number: 16/645,596
- Filed: 2018-08-21
- Published: 2021-11-25
- Granted: 2022-09-13
- Earliest claimed: 2017-09-12
- Assignees: Robert Bosch GmbH
- Applicants: Robert Bosch GmbH
- Inventors: Fernando Ortiz Cuesta; Stephan Simon; Arne Zender
- Web page: https://patentgrep.com/patent/US11443521B2

## CPC

- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06V 10/758: {Involving statistics of pixels or of feature values, e.g. histogram matching}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- B60T 7/12: for automatic initiation; for initiation not subject to will of driver or passenger {(limiting speed of vehicles other than rail vehicles B60K31/00)}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B60T: VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL (electrodynamic brake systems for vehicle, in general B60L; brakes per se, i.e. devices where braking effect occurs, including ultimate brake actuators, F16D); ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES

## Abstract

A method for evaluating images and, in particular, for evaluating correspondence hypotheses of images. The method includes (i) providing a hypothesis matrix of correspondence hypotheses between first and second images, each given as a corresponding image matrix, (ii) evaluating the hypothesis matrix and conditional verification of the image correspondence hypotheses and (iii) providing verified image correspondence hypotheses in a correspondence matrix of image correspondences as the evaluation result, the hypothesis matrix being evaluated by forming and evaluating a histogram with respect to the values of the component for each element of the hypothesis matrix for at least one component of correspondence hypotheses.

## Claims

### Claim 1

A method for evaluating images and for evaluating correspondence hypotheses of images, the method comprising the following steps:
providing correspondence hypotheses between first and second images each given as a corresponding image matrix, in a corresponding hypothesis matrix;
evaluating the hypothesis matrix and conditionally verifying the correspondence hypotheses; and
providing verified image correspondence hypotheses as image correspondences in a correspondence matrix as a result of the evaluation, the evaluation of the hypothesis matrix being performed in that for each element as a reference element of the hypothesis matrix in surroundings of the reference element for at least one component of the correspondence hypotheses, a histogram with respect to values of the component is formed and evaluated, wherein the formation of the histogram for the respective element of the hypothesis matrix takes place by classifying and totaling or totaling in a weighted manner the respective components of the correspondence hypotheses in a given histogram classification for all elements of the hypothesis matrix within an interior of the surroundings of the respective element, and wherein when forming the respective histogram, the surroundings for the respective element of the hypothesis matrix are provided in each case with a window overlaying wholly or partially the hypothesis matrix in the manner of a rectangle, of a polygon, of an oval, of an ellipsis or of a circle.

### Claim 2

A method for evaluating images and for evaluating correspondence hypotheses of images, the method comprising the following steps:
providing correspondence hypotheses between first and second images each given as a corresponding image matrix, in a corresponding hypothesis matrix;
evaluating the hypothesis matrix and conditionally verifying the correspondence hypotheses; and
providing verified image correspondence hypotheses as image correspondences in a correspondence matrix as a result of the evaluation, the evaluation of the hypothesis matrix being performed in that for each element as a reference element of the hypothesis matrix in surroundings of the reference element for at least one component of the correspondence hypotheses, a histogram with respect to values of the component is formed and evaluated, wherein the formation of the histogram for the respective element of the hypothesis matrix takes place by classifying and totaling or totaling in a weighted manner the respective components of the correspondence hypotheses in a given histogram classification for all elements of the hypothesis matrix within an interior of the surroundings of the respective element, wherein when evaluating and forming the histograms, all elements of the hypothesis matrix are captured by the respective surroundings or by a respective window, which are: (i) identical or similar for all elements of the hypothesis matrix with respect to form and/or extent, and/or (ii) configured as sliding surroundings or as a sliding window.

### Claim 3 (depends on 1)

The method as recited in claim 1, wherein when evaluating the histogram for the respective element of the hypothesis matrix on the basis of the histogram, an evaluation value is generated:
by reading out one or multiple values of the histogram in at least one histogram interval for the correspondence hypothesis or of least one component thereof assigned to the given element and by totaling them or totaling them in a weighted manner to form the evaluation value, and
based on one or multiple values from one or from multiple histogram interval adjacent to the assigned histogram interval or from the surroundings of adjacent histogram intervals.

### Claim 4 (depends on 3)

The method as recited in claim 3, wherein when verifying a verification of the correspondence hypothesis for the respective element, the hypothesis matrix or a component thereof is identified, if the evaluation value at least reaches a local or global predefined threshold value.

### Claim 5 (depends on 3)

The method as recited in claim 3, wherein when verifying the correspondence hypothesis to the respective element of the hypothesis matrix or of a component thereof, an assigned and verified correspondence or a component thereof is established in each case by an evaluation value and is entered as a corresponding element into the correspondence matrix.

### Claim 6 (depends on 1)

The method as recited in claim 1, wherein a respectively instantaneously generated histogram, which is stored in a histogram memory is used for the parallel and/or serial verification of multiple correspondence hypotheses for accelerating and/or for reducing a processing effort of the evaluation of the hypothesis matrix and/or of the conditional verification of the correspondence hypotheses.

### Claim 7

A method for evaluating images and for evaluating correspondence hypotheses of images, the method comprising the following steps:
providing correspondence hypotheses between first and second images each given as a corresponding image matrix, in a corresponding hypothesis matrix;
evaluating the hypothesis matrix and conditionally verifying the correspondence hypotheses; and
providing verified image correspondence hypotheses as image correspondences in a correspondence matrix as a result of the evaluation, the evaluation of the hypothesis matrix being performed in that for each element as a reference element of the hypothesis matrix in surroundings of the reference element for at least one component of the correspondence hypotheses, a histogram with respect to values of the component is formed and evaluated, wherein a respectively instantaneously generated histogram, which is stored in a histogram memory is used for the parallel and/or serial verification of multiple correspondence hypotheses for accelerating and/or for reducing a processing effort of the evaluation of the hypothesis matrix and/or of the conditional verification of the correspondence hypotheses, wherein the histogram memory is organized in such a way that an underlying clock cycle is sufficient in order to read out the value for a respectively up-to-date element of the hypothesis matrix from the histogram, as well as values for one or for multiple additional intervals of the histogram, (i) an underlying read address being calculated by adding or subtracting a value of 1 with respect to the value for the up-to-date element stored in the hypothesis matrix and/or (ii) by clustering intervals of the histogram with respect to a dual port of the underlying histogram memory.

### Claim 8 (depends on 1)

The method as recited in claim 1, wherein to accelerate the evaluation of the hypothesis matrix and/or of the conditional verification of the correspondence hypothesis:
the conditional verification is carried out in parallel and/or simultaneously for a plurality of correspondence hypotheses for a corresponding plurality of elements of the hypothesis matrix, and/or
not all elements of the hypothesis matrix within an interior of the surroundings of the respective element and, of the underlying sliding window, are taken into consideration when updating the histogram, but at most a predefined number, a priority sequence as a function of the position in the window establishing in which sequence the elements of the hypothesis matrix within the interior of the surroundings of the respectively given element are primarily taken into consideration.

### Claim 9

A driver assistance system for a vehicle, the driver assistance system configured to evaluate images and evaluate correspondence hypotheses of images, the driver assistance system configured to:
provide correspondence hypotheses between first and second images each given as a corresponding image matrix, in a corresponding hypothesis matrix;
evaluate the hypothesis matrix and conditionally verifying the correspondence hypotheses; and
provide verified image correspondence hypotheses as image correspondences in a correspondence matrix as a result of the evaluation, the evaluation of the hypothesis matrix being performed in that for each element as a reference element of the hypothesis matrix in surroundings of the reference element for at least one component of the correspondence hypotheses, a histogram with respect to values of the component is formed and evaluated;
wherein a result of the evaluation is used to control operation of the vehicle,
wherein the formation of the histogram for the respective element of the hypothesis matrix takes place by classifying and totaling or totaling in a weighted manner the respective components of the correspondence hypotheses in a given histogram classification for all elements of the hypothesis matrix within an interior of the surroundings of the respective element, and
wherein when forming the respective histogram, the surroundings for the respective element of the hypothesis matrix are provided in each case with a window overlaying wholly or partially the hypothesis matrix in the manner of a rectangle, of a polygon, of an oval, of an ellipsis or of a circle.

### Claim 10

A device for evaluating images for evaluating correspondence hypotheses of images, the device configured to:
provide correspondence hypotheses between first and second images each given as a corresponding image matrix, in a corresponding hypothesis matrix;
evaluate the hypothesis matrix and conditionally verifying the correspondence hypotheses; and
provide verified image correspondence hypotheses as image correspondences in a correspondence matrix as a result of the evaluation, the evaluation of the hypothesis matrix being performed in that for each element as a reference element of the hypothesis matrix in surroundings of the reference element for at least one component of the correspondence hypotheses, a histogram with respect to values of the component is formed and evaluated;
wherein the device is an ASIC, or a freely programmable digital signal processing device, or a combination of the ASIC and the feely programmable digital signal processing device,
wherein the formation of the histogram for the respective element of the hypothesis matrix takes place by classifying and totaling or totaling in a weighted manner the respective components of the correspondence hypotheses in a given histogram classification for all elements of the hypothesis matrix within an interior of the surroundings of the respective element, and
wherein when forming the respective histogram, the surroundings for the respective element of the hypothesis matrix are provided in each case with a window overlaying wholly or partially the hypothesis matrix in the manner of a rectangle, of a polygon, of an oval, of an ellipsis or of a circle.

### Claim 11

An operating device which is configured to be controlled during operation using a device for evaluating images for evaluating correspondence hypotheses of images, the device configured to:
provide correspondence hypotheses between first and second images each given as a corresponding image matrix, in a corresponding hypothesis matrix;
evaluate the hypothesis matrix and conditionally verifying the correspondence hypotheses; and
provide verified image correspondence hypotheses as image correspondences in a correspondence matrix as a result of the evaluation, the evaluation of the hypothesis matrix being performed in that for each element as a reference element of the hypothesis matrix in surroundings of the reference element for at least one component of the correspondence hypotheses, a histogram with respect to values of the component is formed and evaluated;
wherein the device is an ASIC, or a freely programmable digital signal processing device, or a combination of the ASIC and the feely programmable digital signal processing device,
wherein the formation of the histogram for the respective element of the hypothesis matrix takes place by classifying and totaling or totaling in a weighted manner the respective components of the correspondence hypotheses in a given histogram classification for all elements of the hypothesis matrix within an interior of the surroundings of the respective element, and
wherein when forming the respective histogram, the surroundings for the respective element of the hypothesis matrix are provided in each case with a window overlaying wholly or partially the hypothesis matrix in the manner of a rectangle, of a polygon, of an oval, of an ellipsis or of a circle.

### Claim 12 (depends on 11)

The operating device as recited in claim 11, wherein the operating device is a vehicle.

### Claim 13

A non-transitory machine-readable memory medium on which is stored a computer program for evaluating images and for evaluating correspondence hypotheses of images, the computer program, when executed by a computer or a digital signal processing unit, causing the computer or the digital signal processing unit to perform the following steps:
providing correspondence hypotheses between first and second images each given as a corresponding image matrix, in a corresponding hypothesis matrix;
evaluating the hypothesis matrix and conditionally verifying the correspondence hypotheses; and
providing verified image correspondence hypotheses as image correspondences in a correspondence matrix as a result of the evaluation, the evaluation of the hypothesis matrix being performed in that for each element as a reference element of the hypothesis matrix in surroundings of the reference element for at least one component of the correspondence hypotheses, a histogram with respect to values of the component is formed and evaluated,
wherein the formation of the histogram for the respective element of the hypothesis matrix takes place by classifying and totaling or totaling in a weighted manner the respective components of the correspondence hypotheses in a given histogram classification for all elements of the hypothesis matrix within an interior of the surroundings of the respective element, and
wherein when forming the respective histogram, the surroundings for the respective element of the hypothesis matrix are provided in each case with a window overlaying wholly or partially the hypothesis matrix in the manner of a rectangle, of a polygon, of an oval, of an ellipsis or of a circle.