# US 12,114,100 B2 · Image processing method in remote control, device, apparatus and program product

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,114,100 B2
- Document ID: US12114100B2
- Status: Granted
- Kind: B2
- Application number: 17/519,375
- Filed: 2021-11-04
- Published: 2022-02-24
- Granted: 2024-10-08
- Earliest claimed: 2020-12-22
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science Technology Co., Ltd.
- Inventors: Qingrui Sun; Jingchao Feng; Liming Xia; Zhuo Chen
- Web page: https://patentgrep.com/patent/US12114100B2

## CPC

- H04N 7/181: {for receiving images from a plurality of remote sources}
- G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras
- G06F 18/25: Fusion techniques
- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00)
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10)
- H04L 65/61: for supporting one-way streaming services, e.g. Internet radio
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

An image processing method in remote control, a device, an apparatus and a program product are provided and related to the field of automatic driving technologies. The specific implementation solution includes receiving image information sent by a vehicle, wherein the image information includes multiple-channel image data collected by the vehicle; performing a stitching process on the multiple-channel image data, to obtain a stitched image; sending the stitched image to a remote cockpit apparatus for controlling the vehicle.

## Claims

### Claim 1

An image processing method in remote control, performed by a cloud apparatus, comprising:
receiving image information sent by a vehicle, wherein the image information comprises multiple-channel image data collected by the vehicle;
performing a stitching process on the multiple-channel image data to obtain a stitched image; and
sending the stitched image to a remote cockpit apparatus for controlling the vehicle;
wherein the multiple-channel image data comprises a first channel image data, a second channel image data and a third channel image data, the performing the stitching process on the multiple-channel image data to obtain the stitched image comprises:
extracting a first common pixel point of the first channel image data and the second channel image data;
creating a first coordinate system based on the first common pixel point, mapping the first channel image data to the first coordinate system to obtain first image mapping data, and mapping the second channel image data to the first coordinate system to obtain second image mapping data;
replicating the first image mapping data and the second image mapping data into a first blank image, respectively, and performing weighted fusion of overlapping image data of the first image mapping data and the second image mapping data in the first blank image to obtain a first stitched sub-image;
extracting a second common pixel point of the third channel image data and the second channel image data;
creating a second coordinate system based on the second common pixel point, mapping the third channel image data to the second coordinate system to obtain third image mapping data, and mapping the second channel image data to the second coordinate system to obtain fourth image mapping data;
replicating the third image mapping data and the fourth image mapping data into a second blank image, respectively, and performing weighted fusion of overlapping image data of the third image mapping data and the fourth image mapping data in the second blank image to obtain a second stitched sub-image; and
stitching the first stitched sub-image and the second stitched sub-image to obtain the stitched image.

### Claim 2 (depends on 1)

The method according to claim 1, wherein:
the receiving the image information sent by the vehicle comprises receiving first image compression information sent by the vehicle via real-time communication (RTC), wherein the first image compression information comprises the multiple-channel image data collected by the vehicle; and
performing the stitching process on the multiple-channel image data comprises performing a decompression process on the first image compression information to obtain the multiple-channel image data, and performing the stitching process on the multiple-channel image data to obtain the stitched image.

### Claim 3 (depends on 1)

The method according to claim 1, wherein sending the stitched image to the remote cockpit apparatus for controlling the vehicle comprises:
compressing the stitched image to obtain second image compression information, and sending the second image compression information via RTC to the remote cockpit apparatus for controlling the vehicle.

### Claim 4

An electronic apparatus, comprising:
at least one processor; and
a storage communicatively connected to the at least one processor, wherein,
instructions executable by the at least one processor are stored in the storage, the instructions are performed by the at least one processor to cause the at least one processor to perform an image processing method in remote control, the method comprising,
receiving image information sent by a vehicle, wherein the image information comprises multiple-channel image data collected by the vehicle;
performing a stitching process on the multiple-channel image data to obtain a stitched image;
sending the stitched image to a remote cockpit apparatus for controlling the vehicle;
wherein the multiple-channel image data comprises a first channel image data, a second channel image data and a third channel image data, the performing the stitching process on the multiple-channel image data to obtain the stitched image comprises:
extracting a first common pixel point of the first channel image data and the second channel image data;
creating a first coordinate system based on the first common pixel point, mapping the first channel image data to the first coordinate system to obtain first image mapping data, and mapping the second channel image data to the first coordinate system to obtain second image mapping data;
replicating the first image mapping data and the second image mapping data into a first blank image, respectively, and performing weighted fusion of overlapping image data of the first image mapping data and the second image mapping data in the first blank image to obtain a first stitched sub-image;
extracting a second common pixel point of the third channel image data and the second channel image data;
creating a second coordinate system based on the second common pixel point, mapping the third channel image data to the second coordinate system to obtain third image mapping data, and mapping the second channel image data to the second coordinate system to obtain fourth image mapping data;
replicating the third image mapping data and the fourth image mapping data into a second blank image, respectively, and performing weighted fusion of overlapping image data of the third image mapping data and the fourth image mapping data in the second blank image to obtain a second stitched sub-image; and
stitching the first stitched sub-image and the second stitched sub-image to obtain the stitched image.

### Claim 5 (depends on 4)

The electronic apparatus according to claim 4, wherein:
receiving the image information sent by the vehicle comprises receiving first image compression information sent by the vehicle via real-time communication (RTC), wherein the first image compression information comprises the multiple-channel image data collected by the vehicle; and
performing the stitching process on the multiple-channel image data comprises performing a decompression process on the first image compression information to obtain the multiple-channel image data, and performing the stitching process on the multiple-channel image data to obtain the stitched image.

### Claim 6 (depends on 4)

The electronic apparatus according to claim 4, wherein sending the stitched image to the remote cockpit apparatus for controlling the vehicle comprises:
compressing the stitched image to obtain second image compression information, and sending the second image compression information via RTC to the remote cockpit apparatus for controlling the vehicle.

### Claim 7

A non-transitory computer readable storage medium, storing therein computer instructions to cause a computer to perform an image processing method in remote control, the method comprising:
receiving image information sent by a vehicle, wherein the image information comprises multiple-channel image data collected by the vehicle;
performing a stitching process on the multiple-channel image data to obtain a stitched image; and
sending the stitched image to a remote cockpit apparatus for controlling the vehicle;
wherein the multiple-channel image data comprises a first channel image data, a second channel image data and a third channel image data, the performing the stitching process on the multiple-channel image data to obtain the stitched image comprises:
extracting a first common pixel point of the first channel image data and the second channel image data;
creating a first coordinate system based on the first common pixel point, mapping the first channel image data to the first coordinate system to obtain first image mapping data, and mapping the second channel image data to the first coordinate system to obtain second image mapping data;
replicating the first image mapping data and the second image mapping data into a first blank image, respectively, and performing weighted fusion of overlapping image data of the first image mapping data and the second image mapping data in the first blank image to obtain a first stitched sub-image;
extracting a second common pixel point of the third channel image data and the second channel image data;
creating a second coordinate system based on the second common pixel point, mapping the third channel image data to the second coordinate system to obtain third image mapping data, and mapping the second channel image data to the second coordinate system to obtain fourth image mapping data;
replicating the third image mapping data and the fourth image mapping data into a second blank image, respectively, and performing weighted fusion of overlapping image data of the third image mapping data and the fourth image mapping data in the second blank image to obtain a second stitched sub-image; and
stitching the first stitched sub-image and the second stitched sub-image to obtain the stitched image.

### Claim 8 (depends on 7)

The storage medium according to claim 7, wherein:
receiving the image information sent by the vehicle comprises receiving first image compression information sent by the vehicle via real-time communication (RTC), wherein the first image compression information comprises the multiple-channel image data collected by the vehicle; and
performing the stitching process on the multiple-channel image data comprises performing a decompression process on the first image compression information to obtain the multiple-channel image data, and performing the stitching process on the multiple-channel image data to obtain the stitched image.

### Claim 9 (depends on 7)

The storage medium according to claim 7, wherein sending the stitched image to the remote cockpit apparatus for controlling the vehicle comprises:
compressing the stitched image to obtain second image compression information, and sending the second image compression information via RTC to the remote cockpit apparatus for controlling the vehicle.

### Claim 10 (depends on 4)

A cloud apparatus, comprising the electronic apparatus according to claim 4.