# US 11,308,337 B2 · Image capturing device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,337 B2
- Document ID: US11308337B2
- Status: Granted
- Kind: B2
- Application number: 16/970,399
- Filed: 2019-02-04
- Published: 2021-04-22
- Granted: 2022-04-19
- Earliest claimed: 2018-02-20
- Assignees: Hitachi Astemo, Ltd.
- Applicants: Hitachi Automotive Systems, Ltd.
- Inventors: Maho Horinaga; Naoya Tada; Yuji Otsuka
- Web page: https://patentgrep.com/patent/US11308337B2

## CPC

- B62D 1/28: non-mechanical {, e.g. following a line or other known markers}
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- B60W 30/12: Lane keeping
- B62D 15/021: {Determination of steering angle}
- B62D 15/025: {Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation (B62D1/28 takes precedence)}
- G06T 7/00: Image analysis
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N 7/183: {for receiving images from a single remote source}
- B60W 2420/403: Image sensing, e.g. optical camera
- B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image capturing device that detects a change in lane width for a vehicle without using information on an adjacent lane is mounted in the vehicle and includes a first camera which captures a first image and a control device which recognizes, in the first image, first and second compartment line L1 and L2 that regulate lanes. The control device determines the presence or absence of a change in lane width on the basis of relationships among the widths of the first compartment line L1 at two points PV1, PV2 at which the distances from a vehicle on the first image are different, the widths of the second compartment line L2 at the two points PV1, PV2 on the first image, and the distances between the first compartment line L1 and the second compartment line L2 at the two points PV1, PV2 on the first image.

## Claims

### Claim 1

An image capturing device, which is mounted in a vehicle, comprising:
a first camera which captures a first image; and
a control device which recognizes a first compartment line and a second compartment line that regulate a lane in the first image,
wherein the control device comprises a lane width determination unit which determines presence or absence of a change in width of the lane based on relationships among widths of the first compartment line at two points on the first image at which distances from the vehicle are different, widths of the second compartment line at the two points on the first image, and distances between the first compartment line and the second compartment line at the two points on the first image, and
wherein, when a first straight line and a second straight line, which are two straight lines that pass through the two points, respectively, and are parallel to each other, are set on the first image,
a width of the first compartment line, a width of the second compartment line, and a distance between the first compartment line and the second compartment line at one of the two points on the first image are a width of the first compartment line, a width of the second compartment line, and a distance between the first compartment line and the second compartment line on the first straight line, and
a width of the first compartment line, a width of the second compartment line, and a distance between the first compartment line and the second compartment line at another point of the two points on the first image are a width of the first compartment line, a width of the second compartment line, and a distance between the first compartment line and the second compartment line on the second straight line.

### Claim 2 (depends on 1)

The image capturing device according to claim 1, wherein, when a first straight line and a second straight line, which are two straight lines that pass through the two points, respectively, and are parallel to each other, are set on the first image, an intersection point between the first straight line and a left end of the first compartment line is P1, an intersection point between the first straight line and a right end of the first compartment line is P2, an intersection point between the first straight line and a left end of the second compartment line is P3, an intersection point between the first straight line and a right end of the second compartment line is P4, an intersection point between the second straight line and a left end of the first compartment line is P5, an intersection point between the second straight line and a right end of the first compartment line is P6, an intersection point between the second straight line and a left end of the second compartment line is P7, an intersection point between the second straight line and a right end of the second compartment line is P8, and following Formulas (1) to (7) are set,
(1) distance between two points P2 and P3/distance between two points P1 and P3=distance between two points P6 and P7/distance between two points P5 and P6
(2) distance between two points P1 and P3/distance between two points P1 and P2=distance between two points P5 and P7/distance between two points P5 and P6
(3) distance between two points P2 and P3/distance between two points P1 and P3=distance between two points P6 and P7/distance between two points P5 and P7
(4) distance between two points P2 and P3/distance between two points P3 and P4=distance between two points P6 and P7/distance between two points P7 and P8
(5) distance between two points P2 and P4/distance between two points P2 and P3=distance between two points P6 and P8/distance between two points P6 and P7
(6) distance between two points P3 and P4/distance between two points P2 and P4=distance between two points P7 and P8/distance between two points P6 and P8
(7) distance between two points P2 and P3/distance between two points P1 and P4=distance between two points P6 and P7/distance between two points P5 and P8,
the lane width determination unit determines that there is no change in the width of the lane in a case of considering that any one of the above Formulas (1) to (3) and any one of the above Formulas (4) to (6) are established, a case of considering that any one of the above Formulas (1) to (3) and the above Formula (7) are established, or a case of considering that any one of the above Formulas (4) to (6) and the above Formula (7) are established, and determines that there is the change in the width of the lane in a case other than the above cases.

### Claim 3 (depends on 1)

The image capturing device according to claim 1, wherein the lane width determination unit determines whether the width of the lane is constant based on a result of the determination on the presence or absence of the change in the width of the lane.

### Claim 4 (depends on 1)

The image capturing device according to claim 1, wherein the control device further includes a branch road determination unit which determines that the lane branches when the lane width determination unit determines that there is the change in the width of the lane.

### Claim 5 (depends on 1)

The image capturing device according to claim 1, further comprising a second camera which captures a second image,
wherein the control device further includes:
a parallax calculation unit which calculates a parallax based on the first image and the second image;
a parallax correction unit which corrects the parallax calculated by the parallax calculation unit based on a position of a vanishing point, which is an intersection point between contour lines of the first compartment line and the second compartment line, when the lane width determination unit determines that there is no change in the width of the lane; and
a distance calculation unit which calculates a distance to an object based on the parallax corrected by the parallax correction unit.

### Claim 6 (depends on 1)

The image capturing device according to claim 1, wherein a case where the lane width determination unit determines that there is no change in the width of the lane is a case where the presence or absence of the change in the width of the lane between the two points is determined based on a plurality of first images obtained at different capturing times, and it is determined that there is no change in the width of the lane in all of a plurality of determinations.

### Claim 7 (depends on 1)

The image capturing device according to claim 1, wherein the first compartment line and the second compartment line are parallel.

### Claim 8 (depends on 1)

A vehicle control system comprising:
the image capturing device according to claim 1;
a vehicle control device which comprises a steering angle command calculation unit that calculates a steering angle command for control of a steering angle of steered wheels of the vehicle based on an external environment recognition result of the image capturing device; and
a steered wheel drive device which drives the steered wheels of the vehicle based on the steering angle command output from the steering angle command calculation unit,
wherein the vehicle control device further includes a parallel determination unit that calculates an extending direction of one of the first compartment line and the second compartment line and a traveling direction of the vehicle and determines whether the extending direction and the traveling direction of the vehicle are parallel, and
the steering angle command calculation unit corrects a steering angle regulated by the steering angle command to be zero when the steering angle regulated by the steering angle command is not zero in a case where the lane width determination unit determines that there is no change in the width of the lane and the parallel determination unit determines that the extending direction and the traveling direction of the vehicle are parallel.

### Claim 9

An image capturing device, which is mounted in a vehicle, comprising:
a first camera which captures a first image; and
a control device which recognizes a first compartment line that regulates a lane in the first image,
wherein the control device comprises a lane width determination unit which determines presence or absence of a change in width of the lane based on relationships among widths of the first compartment line at two points on the first image at which distances from the vehicle are different and distances between the vehicle and the first compartment line at the two points when the vehicle travels along the first compartment line, and
wherein, when a first straight line and a second straight line, which are two straight lines that pass through the two points, respectively, and are parallel to each other, are set on the first image, one of intersection points between the first straight line and the first compartment line is P11, another intersection point between the first straight line and the first compartment line is P12, an intersection point between the first straight line and a third straight line indicating the traveling direction of the vehicle is P13, one of intersection points between the second straight line and the first compartment line is P15, another intersection point between the second straight line and the first compartment line is P1, an intersection point between the second straight line and the third straight line is P17, and following Formulas (11) to (13) are set,
(11) distance between two points P12 and P13/distance between two points P11 and P12=distance between two points P16 and P17/distance between two points P15 and P16
(12) distance between two points P11 and P13/distance between two points P11 and P12=distance between two points P15 and P17/distance between two points P15 and P16
(13) distance between two points P12 and P13/distance between two points P11 and P13=distance between two points P16 and P17/distance between two points P15 and P17,
the lane width determination unit determines that there is no change in the width of the lane in a case of considering that one of the above Formulas (11) to (13) is established, and determines that there is the change in the width of the lane in another case.