# US 2014/0350834 A1 · Vehicle vision system using kinematic model of vehicle motion

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0350834 A1
- Document ID: US20140350834A1
- Status: Published application
- Kind: A1
- Application number: 14/282,029
- Filed: 2014-05-20
- Published: 2014-11-27
- Earliest claimed: 2013-05-21
- Assignees: Magna Electronics Inc.
- Applicants: Magna Electronics Inc.
- Inventors: James Turk
- Web page: https://patentgrep.com/patent/US20140350834A1

## CPC

- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- B60R 2300/105: using multiple cameras
- B60R 2300/30: characterised by the type of image processing
- B60R 2300/402: Image calibration
- B60R 2300/802: for monitoring and displaying vehicle exterior blind spot views
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR

## Abstract

A vision system of a vehicle includes a camera disposed at the vehicle and having a field of view exterior of the vehicle. The camera is operable to capture frames of image data. Responsive to image processing by an image processor of captured image data, a control is operable to determine objects present in the field of view of the camera. Responsive to vehicle data, the control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle. The control determines movement of an object relative to the vehicle via image processing of at least two frames of captured image data during driving of the vehicle by the driver of the vehicle. The control compares the determined relative movement of the object to the determined vehicle motion vector, and responsive to the comparison, the control may determine a misalignment of the camera.

## Claims

### Claim 1

A vision system of a vehicle comprising:
a camera disposed at a vehicle and having a field of view exterior of the vehicle;
wherein said camera comprises a two dimensional array of a plurality of photosensing elements;
wherein said camera is operable to capture frames of image data;
a control, said control comprising an image processor operable to process image data captured by said camera;
wherein, responsive to image processing of captured image data, said control is operable to determine objects present in the field of view of said camera;
wherein, responsive to vehicle data, said control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle;
wherein the vehicle data includes at least two of (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle;
wherein said control determines movement of an object relative to the vehicle via image processing of at least two frames of captured image data during driving of the vehicle by the driver of the vehicle;
wherein said control compares the determined relative movement of the object to the determined vehicle motion vector; and
wherein, responsive to a difference between the determined relative movement of the object and the determined vehicle motion vector, said control determines a misalignment of said camera.

### Claim 2 (depends on 1)

The vision system of claim 1, wherein the vehicle data is communicated to the control via a communication bus of the vehicle.

### Claim 3 (depends on 1)

The vision system of claim 1, wherein a kinematic model comprising a bicycle model is used to represent vehicle motion.

### Claim 4 (depends on 3)

The vision system of claim 3, wherein said kinematic model utilizes at least one of (i) a wheel steering angle, (ii) a wheel pulse, (iii) a heading angle and (iv) translational vehicle motion.

### Claim 5 (depends on 1)

The vision system of claim 1, wherein the vehicle data includes (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle.

### Claim 6 (depends on 1)

The vision system of claim 1, wherein, responsive to a determined misalignment of said camera, said vision system is operable to calibrate said camera.

### Claim 7 (depends on 1)

The vision system of claim 1, wherein said control compares the determined vehicle motion vector to an object vector determined between a first position of the object captured in a first frame of image data and a second position of the object captured in a second frame of image data.

### Claim 8 (depends on 7)

The vision system of claim 7, wherein said control determines a misalignment of said camera responsive to a difference between a direction or magnitude of the determined vehicle motion vector and a direction or magnitude of the determined object vector.

### Claim 9 (depends on 1)

The vision system of claim 1, wherein said vision system comprises a plurality of cameras disposed at the vehicle, each having a respective field of view exterior of the vehicle.

### Claim 10 (depends on 9)

The vision system of claim 9, wherein image data captured by at least some of said plurality of cameras is used for a surround view system of the vehicle.

### Claim 11 (depends on 10)

The vision system of claim 10, wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver side camera disposed at a driver side exterior rearview mirror assembly of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger side camera disposed at a passenger side exterior rearview mirror assembly of the vehicle and at least having a generally sideward and rearward field of view.

### Claim 12 (depends on 1)

The vision system of claim 1, wherein said camera comprises a CMOS camera.

### Claim 13

A vision system of a vehicle comprising:
a plurality of cameras disposed at a vehicle, each having a respective field of view exterior of the vehicle;
wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver side camera disposed at a driver side of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger side camera disposed at a passenger side of the vehicle and at least having a generally sideward and rearward field of view;
wherein said cameras are operable to capture frames of image data;
a control, said control comprising an image processor operable to process image data captured by said cameras;
wherein, responsive to image processing of captured image data, said control is operable to determine objects present in the fields of view of said cameras;
wherein, responsive to vehicle data, said control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle;
wherein the vehicle data includes at least two of (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle;
wherein said control determines movement of an object relative to the vehicle via image processing of at least two frames of image data captured by at least one of said cameras during driving of the vehicle by the driver of the vehicle;
wherein said control compares the determined relative movement of the object to the determined vehicle motion vector; and
wherein, responsive to a difference between the determined relative movement of the object and the determined vehicle motion vector, said control determines a misalignment of said at least one of said cameras.

### Claim 14 (depends on 13)

The vision system of claim 13, wherein each of said cameras comprises a two dimensional array of a plurality of photosensing elements.

### Claim 15 (depends on 13)

The vision system of claim 13, wherein the vehicle data is communicated to the control via a communication bus of the vehicle.

### Claim 16 (depends on 13)

The vision system of claim 13, wherein, responsive to a determined misalignment of said at least one of said cameras, said vision system is operable to calibrate said at least one of said cameras.

### Claim 17 (depends on 13)

The vision system of claim 13, wherein said control compares the determined vehicle motion vector to an object vector determined between a first position of the object captured in a first frame of image data by said at least one of said cameras and a second position of the object captured in a second frame of image data by said at least one of said cameras.

### Claim 18 (depends on 17)

The vision system of claim 17, wherein said control determines a misalignment of said at least one of said cameras responsive to a difference between a direction or magnitude of the determined vehicle motion vector and a direction or magnitude of the determined object vector.

### Claim 19

A vision system of a vehicle comprising:
a plurality of cameras disposed at a vehicle, each having a respective field of view exterior of the vehicle;
wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver side camera disposed at a driver side of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger side camera disposed at a passenger side of the vehicle and at least having a generally sideward and rearward field of view;
wherein each of said cameras comprises a two dimensional array of a plurality of photosensing elements;
wherein said cameras are operable to capture frames of image data;
a control, said control comprising an image processor operable to process image data captured by said cameras;
wherein, responsive to image processing of captured image data, said control is operable to determine objects present in the fields of view of said cameras;
wherein, responsive to vehicle data, said control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle;
wherein the vehicle data includes at least two of (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle;
wherein the vehicle data is communicated to the control via a communication bus of the vehicle;
wherein said control determines movement of an object relative to the vehicle via image processing of at least two frames of image data captured by at least one of said cameras during driving of the vehicle by the driver of the vehicle;
wherein said control compares the determined relative movement of the object to the determined vehicle motion vector;
wherein, responsive to a difference between the determined relative movement of the object and the determined vehicle motion vector, said control determines a misalignment of said at least one of said cameras; and
wherein, responsive to a determined misalignment of said at least one of said cameras, said vision system is operable to calibrate said at least one of said cameras.

### Claim 20 (depends on 19)

The vision system of claim 19, wherein said control compares the determined vehicle motion vector to an object vector determined between a first position of the object captured in a first frame of image data by said at least one of said cameras and a second position of the object captured in a second frame of image data by said at least one of said cameras, and wherein said control determines a misalignment of said at least one of said cameras responsive to a difference between a direction or magnitude of the determined vehicle motion vector and a direction or magnitude of the determined object vector.
