# US 2023/0410661 A1 · Method for warning collision of vehicle, system, vehicle, and computer readable storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410661 A1
- Document ID: US20230410661A1
- Status: Published application
- Kind: A1
- Application number: 18/121,572
- Filed: 2023-03-14
- Published: 2023-12-21
- Earliest claimed: 2022-06-15
- Assignees: Hon Hai Precision Industry Co., Ltd.
- Applicants: Hon Hai Precision Industry Co., Ltd.
- Inventors: WAN-JHEN LEE; CHIN-PIN KUO
- Web page: https://patentgrep.com/patent/US20230410661A1

## CPC

- G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection}
- G06T 7/194: involving foreground-background segmentation
- G06T 7/215: Motion-based segmentation
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G08G 1/056: with provision for distinguishing direction of travel
- G06T 2207/30256: Lane; Road marking
- G06T 2207/30261: Obstacle
- G06V 2201/08: Detecting or categorising vehicles
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for warning collision of vehicle is provided. The method obtains a pre-detected video. The pre-detected video includes a number of video frames, and the video frames are continuous. The method detects one or more vehicles in each of the video frames and determines a movement state of each of the vehicles via an optical flow method. The method detects one or more lane lines in each of the video frames to determine lane information, and determines whether to provide a collision warning according to the lane information and the movement state of each of the vehicles. A related vehicle and a non-transitory storage medium are provided.

## Claims

### Claim 1

A method for warning collision of vehicle comprising:
obtaining a pre-detected video; the pre-detected video comprising a plurality of video frames, and the video frames being continuous;
detecting one or more vehicles in each of the video frames and determining a movement state of each of the vehicles via an optical flow method;
detecting one or more lane lines in each of the video frames to determine lane information; and
determining whether to provide a collision warning according to the lane information and the movement state of each of the vehicles.

### Claim 2 (depends on 1)

The method according to claim 1, wherein:
the lane information comprises whether the vehicle and the instant vehicle are at the same lane, and the movement state of the vehicle comprises a movement direction of the vehicle;
before the determining whether to provide the collision warning according to the lane information and the movement state of each of the vehicles, the method comprises:
obtaining a movement direction of the instant vehicle;
the determining whether to provide the collision warning according to the lane information and the movement state of each of the vehicles comprises:
providing the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at the same lane, and the movement direction of the vehicle is opposite to the movement direction of the instant vehicle.

### Claim 3 (depends on 1)

The method according to claim 1, wherein:
the lane information comprises a first lane that an instant vehicle is at and a second lane that the vehicle is at, and the movement state of the vehicle comprises a movement direction of the vehicle;
before the determining whether to provide the collision warning according to the lane information and the movement state of each of the vehicles, the method comprises:
obtaining a movement direction of the instant vehicle;
the determining whether to provide the collision warning according to the lane information and the movement state of each of the vehicles comprises:
providing the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at different lanes, the movement direction of the instant vehicle is going straight along the second lane, and the movement direction of the vehicle intersects with the second lane.

### Claim 4 (depends on 1)

The method according to claim 1, wherein:
the lane information comprises a first lane that an instant vehicle is at and a second lane that the vehicle is at, and the movement state of the vehicle comprises a movement direction of the vehicle;
before the determining whether to provide the collision warning according to the lane information and the movement state of each of the vehicles, the method comprises:
obtaining a movement direction of the instant vehicle;
the determining whether to provide the collision warning according to the lane information and the movement state of each of the vehicles comprises:
not provide the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at different lanes, and the movement direction of the vehicle does not intersect with the movement direction of the instant vehicle.

### Claim 5 (depends on 1)

The method according to claim 1, wherein:
the lane information comprises a first lane that an instant vehicle is at, a second lane that the vehicle is at, and whether the vehicle and the instant vehicle are at two lanes in the same direction; and the movement state of the vehicle comprises a movement direction of the vehicle;
before the determining whether to provide the collision warning according to the lane information and the movement state of each of the vehicles, the method comprises:
obtaining a movement direction of the instant vehicle;
the determining whether to provide the collision warning according to the lane information and the movement state of each of the vehicles comprises:
not providing the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at two lanes in an opposite direction, the movement direction of the instant vehicle is turning right from the first lane to a right lane of the first lane, and the movement direction of the vehicle is turning right from the second lane to a right lane of the second lane.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the determining a movement state of each of the vehicles via an optical flow method comprises:
segmenting each of the video frames into a foreground image and a background image; the foreground image comprising a pixel region of the one or more vehicles in a corresponding video frame, and the background region comprising a pixel region excluding the pixel region of the one or more vehicles in the corresponding video frame;
obtaining a plurality of first corners of the one or more vehicles in the foreground image and employing an optical flow method to determine a first optical flow velocity of the first corners of each of the vehicles in the foreground image;
obtaining a plurality of second corners in the background image and employing an optical flow method to determine a second optical flow velocity of the second corners in the background image;
determining a movement state of each of the vehicles according to the first optical flow velocity of the first corners of each of the vehicles in the foreground image and the second optical flow velocity of the second corners in the background image.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the determining a movement state of each of the vehicles according to the first optical flow velocity of the first corners of each of the vehicles in the foreground image and the second optical flow velocity of the second corners in the background image comprises:
determining that the vehicle is still if a different value between the first optical flow velocity of the first corners of each of the vehicles in the foreground image and the second optical flow velocity of the second corners in the background image is within a preset range;
determining that the vehicle is moving and determining a movement direction of the vehicle if the different value between the first optical flow velocity of the first corners of each of the vehicles in the foreground image and the second optical flow velocity of the second corners in the background image is outside the preset range.

### Claim 8

A vehicle comprising:
a storage device;
at least one processor; and
the storage device storing one or more programs, which when executed by the at least one processor, cause the at least one processor to:
obtain a pre-detected video; the pre-detected video comprising a plurality of video frames, and the video frames being continuous;
detect one or more vehicles in each of the video frames and determining a movement state of each of the vehicles via an optical flow method;
detect one or more lane lines in each of the video frames to determine lane information; and
determine whether to provide a collision warning according to the lane information and the movement state of each of the vehicles.

### Claim 9 (depends on 8)

The vehicle according to claim 8, wherein:
the lane information comprises whether the vehicle and the instant vehicle are at the same lane, and the movement state of the vehicle comprises a movement direction of the vehicle;
the vehicle further causes the at least one processor to:
obtain a movement direction of the instant vehicle;
provide the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at the same lane, and the movement direction of the vehicle is opposite to the movement direction of the instant vehicle.

### Claim 10 (depends on 8)

The vehicle according to claim 8, wherein:
the lane information comprises a first lane that an instant vehicle is at and a second lane that the vehicle is at, and the movement state of the vehicle comprises a movement direction of the vehicle;
the vehicle further causes the at least one processor to:
obtain a movement direction of the instant vehicle;
provide the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at different lanes, the movement direction of the instant vehicle is going straight along the second lane, and the movement direction of the vehicle intersects with the second lane.

### Claim 11 (depends on 8)

The vehicle according to claim 8, wherein:
the lane information comprises a first lane that an instant vehicle is at and a second lane that the vehicle is at, and the movement state of the vehicle comprises a movement direction of the vehicle;
the vehicle further causes the at least one processor to:
obtain a movement direction of the instant vehicle;
not provide the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at different lanes, and the movement direction of the vehicle does not intersect with the movement direction of the instant vehicle.

### Claim 12 (depends on 8)

The vehicle according to claim 8, wherein:
the lane information comprises a first lane that an instant vehicle is at, a second lane that the vehicle is at, and whether the vehicle and the instant vehicle are at two lanes in the same direction; and the movement state of the vehicle comprises a movement direction of the vehicle;
the vehicle further causes the at least one processor to:
obtain a movement direction of the instant vehicle;
not provide the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at two lanes in an opposite direction, the movement direction of the instant vehicle is turning right from the first lane to a right lane of the first lane, and the movement direction of the vehicle is turning right from the second lane to a right lane of the second lane.

### Claim 13 (depends on 8)

The vehicle according to claim 8, wherein further causes the at least one processor to:
segment each of the video frames into a foreground image and a background image; the foreground image comprising a pixel region of the one or more vehicles in a corresponding video frame, and the background region comprising a pixel region excluding the pixel region of the one or more vehicles in the corresponding video frame;
obtain a plurality of first corners of the one or more vehicles in the foreground image and employ an optical flow method to determine a first optical flow velocity of the first corners of each of the vehicles in the foreground image;
obtain a plurality of second corners in the background image and employ an optical flow method to determine a second optical flow velocity of the second corners in the background image;
determine a movement state of each of the vehicles according to the first optical flow velocity of the first corners of each of the vehicles in the foreground image and the second optical flow velocity of the second corners in the background image.

### Claim 14 (depends on 13)

The vehicle according to claim 13, wherein further causes the at least one processor to:
determine that the vehicle is still if a different value between the first optical flow velocity of the first corners of each of the vehicles in the foreground image and the second optical flow velocity of the second corners in the background image is within a preset range;
determine that the vehicle is moving and determine a movement direction of the vehicle if the different value between the first optical flow velocity of the first corners of each of the vehicles in the foreground image and the second optical flow velocity of the second corners in the background image is outside the preset range.

### Claim 15

A non-transitory storage medium storing a set of commands, when the commands being executed by at least one processor of a vehicle, causing the at least one processor to:
obtain a pre-detected video; the pre-detected video comprising a plurality of video frames, and the video frames being continuous;
detect one or more vehicles in each of the video frames and determining a movement state of each of the vehicles via an optical flow method;
detect one or more lane lines in each of the video frames to determine lane information; and
determine whether to provide a collision warning according to the lane information and the movement state of each of the vehicles.

### Claim 16 (depends on 15)

The non-transitory storage medium according to claim 15, wherein:
the lane information comprises whether the vehicle and the instant vehicle are at the same lane, and the movement state of the vehicle comprises a movement direction of the vehicle;
the non-transitory storage medium further causes the at least one processor to:
obtain a movement direction of the instant vehicle;
provide the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at the same lane, and the movement direction of the vehicle is opposite to the movement direction of the instant vehicle.

### Claim 17 (depends on 15)

The non-transitory storage medium according to claim 15, wherein:
the lane information comprises a first lane that an instant vehicle is at and a second lane that the vehicle is at, and the movement state of the vehicle comprises a movement direction of the vehicle;
the non-transitory storage medium further causes the at least one processor to:
obtain a movement direction of the instant vehicle;
provide the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at different lanes, the movement direction of the instant vehicle is going straight along the second lane, and the movement direction of the vehicle intersects with the second lane.

### Claim 18 (depends on 15)

The non-transitory storage medium according to claim 15, wherein:
the lane information comprises a first lane that an instant vehicle is at and a second lane that the vehicle is at, and the movement state of the vehicle comprises a movement direction of the vehicle;
the non-transitory storage medium further causes the at least one processor to:
obtain a movement direction of the instant vehicle;
not provide the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at different lanes, and the movement direction of the vehicle does not intersect with the movement direction of the instant vehicle.

### Claim 19 (depends on 15)

The non-transitory storage medium according to claim 15, wherein:
the lane information comprises a first lane that an instant vehicle is at, a second lane that the vehicle is at, and whether the vehicle and the instant vehicle are at two lanes in the same direction; and the movement state of the vehicle comprises a movement direction of the vehicle;
the non-transitory storage medium further causes the at least one processor to:
obtain a movement direction of the instant vehicle;
not provide the collision warning if the vehicle is moving, the vehicle and the instant vehicle are at two lanes in an opposite direction, the movement direction of the instant vehicle is turning right from the first lane to a right lane of the first lane, and the movement direction of the vehicle is turning right from the second lane to a right lane of the second lane.

### Claim 20 (depends on 15)

The non-transitory storage medium according to claim 15, wherein further causes the at least one processor to:
segment each of the video frames into a foreground image and a background image; the foreground image comprising a pixel region of the one or more vehicles in a corresponding video frame, and the background region comprising a pixel region excluding the pixel region of the one or more vehicles in the corresponding video frame;
obtain a plurality of first corners of the one or more vehicles in the foreground image and employ an optical flow method to determine a first optical flow velocity of the first corners of each of the vehicles in the foreground image;
obtain a plurality of second corners in the background image and employ an optical flow method to determine a second optical flow velocity of the second corners in the background image;
determine a movement state of each of the vehicles according to the first optical flow velocity of the first corners of each of the vehicles in the foreground image and the second optical flow velocity of the second corners in the background image.
