# US 2022/0398852 A1 · Method for determining object information relating to an object in a vehicle environment, control unit and vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0398852 A1
- Document ID: US20220398852A1
- Status: Published application
- Kind: A1
- Application number: 17/821,378
- Filed: 2022-08-22
- Published: 2022-12-15
- Earliest claimed: 2020-03-09
- Assignees: ZF CV Systems Global GmbH
- Applicants: ZF CV Systems Global GmbH
- Inventors: Thomas Dieckmann; Tobias Klinger; Janik Ricke
- Web page: https://patentgrep.com/patent/US20220398852A1

## CPC

- G03B 35/06: with axial movement of lens or gate between exposures
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06T 7/593: from stereo images
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/771: Feature selection, e.g. selecting representative features from a multi-dimensional feature space
- H04N 13/211: using temporal multiplexing
- B60R 2300/107: using stereoscopic cameras
- B60R 2300/108: using 'non-standard' camera systems, e.g. camera sensor used for additional purposes i.a. rain sensor, camera sensor split in multiple image areas
- B60R 2300/8033: for pedestrian protection
- B60R 2300/8093: for obstacle warning
- G06T 2207/10012: Stereo images
- G06T 2207/10141: Special mode during image acquisition
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T 2207/30261: Obstacle
- G06V 2201/07: Target detection
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR

## Abstract

The disclosure relates to a method for determining object information relating to an abject in an environment of a vehicle having a camera. The method includes: capturing the environment with the camera from a first position; changing the position of the camera; capturing the environment with the camera from a second position; determining object information relating to an object by selecting at least one first pixel in the first image and at least one second pixel in the second image, by selecting the first pixel and the second pixel such that they are assigned to the same object point of the object, and determining object coordinates of the assigned object point by triangulation. Changing the position of the camera is brought about by controlling an active actuator system in the vehicle. The actuator system adjusts the camera by an adjustment distance without changing a driving condition of the vehicle.

## Claims

### Claim 1

A method for determining object information of an object in an environment of a vehicle, the vehicle having at least one camera, the at least one camera having a position, the method comprising:
capturing the environment with the at least one camera from a first position and, in dependence thereon, generating a first image having first pixels;
changing the position of the at least one camera;
capturing the environment with the at least one camera from a second position and, in dependence thereon, creating a second image having second pixels;
determining object information relating to the object in the captured environment by
selecting at least one first pixel in the first image and at least one second pixel in the second image, the at least one first pixel and the at least one second pixel being selected such that they are assigned to a same object point of the object in the captured environment, and
determining object coordinates of the assigned object point from first image coordinates of the at least one first pixel and from second image coordinates of the at least one second pixel by triangulation, assuming a base length between the two positions of the camera; and,
wherein said changing the position of the at least one camera from the first position to the second position is brought about by controlling an active actuator system in the vehicle, wherein the active actuator system adjusts the at least one camera by an adjustment distance without changing a driving condition of the vehicle.

### Claim 2 (depends on 1)

The method of claim 1 wherein the base length between the first position and the second position of the camera is determined from odometry data of the vehicle; the odometry data are generated at least depending on the adjustment distance and the odometry data characterize an adjustment of the camera between the first position and the second position.

### Claim 3 (depends on 1)

The method of claim 1, wherein a camera adjustment system is controlled as the active actuator system; the active actuator system comprises at least one of actuating motors, pneumatic cylinders, hydraulic cylinders, and electric servo cylinders; and, the at least one camera is fixed directly to the camera adjustment system so that when the camera adjustment system is controlled the at least one camera is adjusted by the adjustment distance to change the position of the at least one camera.

### Claim 4 (depends on 1)

The method of claim 1, wherein an active air suspension system with air springs or a chassis adjustment system is controlled as the active actuator system, wherein by controlling the active air suspension system or the chassis adjustment system, a vehicle body is adjusted in height by the adjustment distance so that the at least one camera fixed to the vehicle body is adjusted indirectly by the adjustment distance to change the position of the at least one camera.

### Claim 5 (depends on 1)

The method of claim 1, wherein a component adjustment system is controlled as the active actuator system, wherein by controlling the component adjustment system, a component of the vehicle is adjusted by the adjustment distance so that the at least one camera attached to this component is adjusted indirectly by the adjustment distance to change the position of the at least one camera.

### Claim 6 (depends on 1)

The method of claim 1, wherein, when the active actuator system is controlled, the vehicle is in the driving condition in which the vehicle has a vehicle speed which is lower than a threshold speed, or the vehicle is at a standstill.

### Claim 7 (depends on 1)

The method of claim 1, wherein the object coordinates for a plurality of object points are determined from the first image and the second image by triangulation; and, at least one of an object contour and an object form is determined from the plurality of object points.

### Claim 8 (depends on 7)

The method of claim 7, wherein the object is divided into object classes via at least one of the object contour and the object form.

### Claim 9 (depends on 1)

The method of claim 1, wherein a plurality of cameras are provided and object information relating to the object is determined via each camera by adjusting by the corresponding adjustment distance independently of one another.

### Claim 10 (depends on 9)

The method of claim 9, further comprising plausibility checking the object information determined by the plurality of cameras.

### Claim 11 (depends on 1)

The method of claim 1, wherein more than two images are acquired at different positions and, from each acquired image, pixels are selected which are assigned to the same object point of the object in the captured environment, wherein the object coordinates of the assigned object point are determined from image coordinates of the selected pixels by triangulation assuming a base length between respective positions of the camera.

### Claim 12 (depends on 1)

The method of claim 1, wherein a bundle adjustment is carried out via the first image and the second image.

### Claim 13 (depends on 1)

The method of claim 1, wherein a plurality of first pixels in the first image are combined into a first feature point and a plurality of second pixels in the second image are combined into a second feature point, the first feature point and the second feature point being selected such that they are assigned to a same feature of the object in the captured environment; and,
wherein the object coordinates of the assigned feature are determined from first image coordinates of the first feature point and from second image coordinates of the second feature point by triangulation, assuming a base length between the first position and the second position of the camera.

### Claim 14 (depends on 1)

The method of claim 1 further comprising plausibility checking the determined object information, which follows from an adjustment of the camera by the adjustment distance via the active actuator system with object information which follows from odometry data of the vehicle, wherein the odometry data includes at least one of a wheel speed signal, a vehicle speed, a steering angle, and bending angle.

### Claim 15 (depends on 5)

The method of claim 5, wherein the component is at least one of a driver's cab and an aerodynamic component.

### Claim 16

A control unit for determining object information of an object in an environment of a vehicle having a camera, the control unit comprising:
a processor;
a non-transitory computer readable storage medium having program code stored thereon;
said program code being configured to capture the environment with said camera from a first position and, in dependence thereon, generating a first image having first pixels;
said program code being configured to change the position of the camera and to capture the environment via the camera from a second position and, in dependence thereon, generate a second image having second pixels;
said program code being configured to determine the abject information in the captured environment by
selecting at least one first pixel in the first image and at least one second pixel in the second image, the at least one first pixel and the at least one second pixel being selected such that they are assigned to a same object point of the object in the captured environment, and
determining object coordinates of the assigned object point from first image coordinates of the at least one first pixel and from second image coordinates of the at least one second pixel by triangulation assuming, a base length between the two positions of the camera; and,
wherein the changing the position of said camera via the control unit is brought about by controlling an active actuator system in the vehicle, wherein the active actuator system adjusts said camera by an adjustment distance without changing a driving condition of the vehicle.

### Claim 17

A vehicle comprising:
a camera defining a position;
a control unit for determining object information of an object in an environment of the vehicle;
said control unit being configured to capture the environment with said camera from a first position and, in dependence thereon, generating a first image having first pixels;
said control unit being configured to change the position of the camera and to capture the environment via the camera from a second position and, in dependence thereon, generate a second image having second pixels;
said control unit being further configured to determine the object information in the captured environment by
selecting at least one first pixel in the first image and at least one second pixel in the second image, the first pixel and the second pixel being selected such that they are assigned to a same object point of the object in the captured environment, and
determining the object coordinates of the assigned object point from first image coordinates of the at least one first pixel and from second image coordinates of the at least one second pixel by triangulation assuming a base length between the two positions of the camera; and,
wherein the changing the position of said camera via the control unit is brought about by controlling an active actuator system in the vehicle, wherein the active actuator system adjusts said camera by an adjustment distance without changing a driving condition of the vehicle.

### Claim 18 (depends on 17)

The vehicle of claim 17, wherein the vehicle is a multipart vehicle having a towing vehicle and a trailer; and, said camera is arranged on at least one of said towing vehicle and said trailer.
