# US 2022/0044560 A1 · Roadside sensing method, electronic device, storage medium, and roadside equipment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044560 A1
- Document ID: US20220044560A1
- Status: Published application
- Kind: A1
- Application number: 17/511,121
- Filed: 2021-10-26
- Published: 2022-02-10
- Earliest claimed: 2020-12-03
- Applicants: Apollo Intelligent Connectivity (Beijing) Technology Co., Ltd.
- Inventors: Libin Yuan
- Web page: https://patentgrep.com/patent/US20220044560A1

## CPC

- G06T 3/12: Panospheric to cylindrical image transformations
- G08G 1/04: using optical or ultrasonic detectors
- G08G 1/0116: {from roadside infrastructure, e.g. beacons}
- G08G 1/0129: {for creating historical data or processing based on historical data}
- G08G 1/0145: {for active traffic flow control}
- H04W 4/40: for vehicles, e.g. vehicle-to-pedestrians [V2P]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

A sensing method, an electronic device, and a storage medium are provided, which are related to a field of road cooperation of intelligent traffic. The specific implementation solution includes: acquiring a wide-angle image captured by a wide-angle camera; performing a de-distortion process on the wide-angle image, to obtain an image directly below the wide-angle camera; and performing a projective transformation on the wide-angle image to at least one viewing angle through a spherical projection model, to obtain at least one planar projection image, wherein each planar projection image corresponds to one viewing angle.

## Claims

### Claim 1

A roadside sensing method, comprising:
acquiring a wide-angle image captured by a wide-angle camera:
performing a de-distortion process on the wide-angle image, to obtain an image directly below the wide-angle camera; and
performing a projective transformation on the wide-angle image to at least one viewing angle through a spherical projection model, to obtain at least one planar projection image, wherein each planar projection image corresponds to one viewing angle.

### Claim 2 (depends on 1)

The method according to claim 1, wherein performing the projective transformation on the wide-angle image to at least one viewing angle through the spherical projection model comprises:
determining a corresponding relationship between pixel coordinates of the planar projection image and pixel coordinates of the wide-angle image; and
performing the projective transformation on the wide-angle image to the at least one viewing angle through the spherical projection model according to the corresponding relationship.

### Claim 3 (depends on 2)

The method according to claim 2, wherein determining the corresponding relationship between the pixel coordinates of the planar projection image and the pixel coordinates of the wide-angle image comprises:
determining a focal length of the wide-angle camera by utilizing a radius of the wide-angle image;
determining a length of a vector from a projection center of the spherical projection model to an arbitrary point projected onto a coordinate system of the wide-angle image by utilizing the focal length of the wide-angle camera and an angular relationship between the arbitrary point in the planar projection image and a space coordinate system, wherein the space coordinate system is a coordinate system corresponding to a spherical projection surface of the wide-angle camera;
determining a pixel coordinate of a projection point of the arbitrary point on a planar in which the wide-angle image is located by utilizing the length, the angular relation between the arbitrary point in the planar projection image and the space coordinate system, and a pixel coordinate of a central point of the wide-angle image; and
determining the corresponding relationship between the pixel coordinates of the planar projection image and the pixel coordinates of the wide-angle image according to a pixel coordinate of the arbitrary point on the coordinate system of the planar projection image and the pixel coordinate of the projection point of the arbitrary point on the planar in which the wide-angle image is located.

### Claim 4 (depends on 3)

The method according to claim 3, wherein performing the projective transformation on the wide-angle image to the at least one viewing angle through the spherical projection model according to the corresponding relationship comprises:
selecting at least one viewing angle;
determining an angular relationship between the viewing angle and the space coordinate system;
determining the angular relationship between the arbitrary point in the planar projection image and the space coordinate system by utilizing the angular relationship between the viewing angle and the space coordinate system; and
performing the projective transformation on the wide-angle image to the viewing angle by utilizing the corresponding relationship and the angular relationship between the arbitrary point in the planar projection image and the space coordinate system.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the wide-angle camera comprises a fish-eye camera and the wide-angle image comprises a fish-eye image.

### Claim 6 (depends on 2)

The method according to claim 2, wherein the wide-angle camera comprises a fish-eye camera and the wide-angle image comprises a fish-eye image.

### Claim 7 (depends on 3)

The method according to claim 3, wherein the wide-angle camera comprises a fish-eye camera and the wide-angle image comprises a fish-eye image.

### Claim 8 (depends on 4)

The method according to claim 4, wherein the wide-angle camera comprises a fish-eye camera and the wide-angle image comprises a fish-eye image.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the wide-angle camera is provided at a road junction with one wide-angle camera provided in each direction of the road junction.

### Claim 10 (depends on 2)

The method according to claim 2, wherein the wide-angle camera is provided at a road junction with one wide-angle camera provided in each direction of the road junction.

### Claim 11 (depends on 3)

The method according to claim 3, wherein the wide-angle camera is provided at a road junction with one wide-angle camera provided in each direction of the road junction.

### Claim 12 (depends on 4)

The method according to claim 4, wherein the wide-angle camera is provided at a road junction with one wide-angle camera provided in each direction of the road junction.

### Claim 13

An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein
the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, enable the at least one processor to:
acquire a wide-angle image captured by a wide-angle camera;
perform a de-distortion process on the wide-angle image, to obtain an image directly below the wide-angle camera; and
perform a projective transformation on the wide-angle image to at least one viewing angle through a spherical projection model, to obtain at least one planar projection image, wherein each planar projection image corresponds to one viewing angle.

### Claim 14 (depends on 13)

The electronic device according to claim 13, wherein the instructions are executed by the at least one processor to further enable the at least one processor to:
determine a corresponding relationship between pixel coordinates of the planar projection image and pixel coordinates of the wide-angle image; and
perform the projective transformation on the wide-angle image to the at least one viewing angle through the spherical projection model according to the corresponding relationship.

### Claim 15 (depends on 13)

The electronic device according to claim 13, wherein the instructions are executed by the at least one processor to further enable the at least one processor to:
determine a focal length of the wide-angle camera by utilizing a radius of the wide-angle image;
determine a length of a vector from a projection center of the spherical projection model to an arbitrary point projected onto a coordinate system of the wide-angle image by utilizing the focal length of the wide-angle camera and an angular relationship between the arbitrary point in the planar projection image and a space coordinate system, wherein the space coordinate system is a coordinate system corresponding to a spherical projection surface of the wide-angle camera;
determine a pixel coordinate of a projection point of the arbitrary point on a planar in which the wide-angle image is located by utilizing the length, the angular relation between the arbitrary point in the planar projection image and the space coordinate system, and a pixel coordinate of a central point of the wide-angle image; and
determine the corresponding relationship between the pixel coordinates of the planar projection image and the pixel coordinates of the wide-angle image according to a pixel coordinate of the arbitrary point on the coordinate system of the planar projection image and the pixel coordinate of the projection point of the arbitrary point on the planar in which the wide-angle image is located.

### Claim 16 (depends on 13)

The electronic device according to claim 13, wherein the instructions are executed by the at least one processor to further enable the at least one processor to:
select at least one viewing angle;
determine an angular relationship between the viewing angle and the space coordinate system;
determine the angular relationship between the arbitrary point in the planar projection image and the space coordinate system by utilizing the angular relationship between the viewing angle and the space coordinate system; and
perform the projective transformation on the wide-angle image to the viewing angle by utilizing the corresponding relationship and the angular relationship between the arbitrary point in the planar projection image and the space coordinate system.

### Claim 17 (depends on 13)

The electronic device according to claim 13, wherein the wide-angle camera comprises a fish-eye camera and the wide-angle image comprises a fish-eye image.

### Claim 18 (depends on 13)

The electronic device according to claim 13, wherein the wide-angle camera is provided at a road junction with one wide-angle camera provided in each direction of the road junction.

### Claim 19

A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions, when executed by a computer, cause the computer to:
acquire a wide-angle image captured by a wide-angle camera;
perform a de-distortion process on the wide-angle image, to obtain an image directly below the wide-angle camera; and
perform a projective transformation on the wide-angle image to at least one viewing angle through a spherical projection model, to obtain at least one planar projection image, wherein each planar projection image corresponds to one viewing angle.

### Claim 20 (depends on 13)

A roadside equipment comprising the electronic device of claim 13.
