# US 2022/0028177 A1 · Tour guiding method, electronic device and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0028177 A1
- Document ID: US20220028177A1
- Status: Published application
- Kind: A1
- Application number: 17/498,204
- Filed: 2021-10-11
- Published: 2022-01-27
- Earliest claimed: 2020-11-11
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science Technology Co., Ltd.
- Inventors: Yuhua DONG; Ziwen QIN; Zhaohu LI; Zhaoliang LIU; Zhihao WANG; Haoxing WANG
- Web page: https://patentgrep.com/patent/US20220028177A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06Q 50/14: Travel agencies
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06V 20/20: in augmented reality scenes
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/64: Three-dimensional [3D] objects
- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/4223: Cameras (H04N23/00 takes precedence)
- H04N 21/4722: for requesting additional data associated with the content
- H04N 21/816: {involving special video data, e.g 3D video}
- H04W 4/021: Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
- H04W 4/025: {using location based information parameters}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Provided are a tour guiding method and apparatus, an electronic device and a storage medium. The method includes receiving a video stream collected by a user in a current location area of a target scenic spot and sent by an application device; determining, based on the collected video stream and prestored three-dimensional point cloud data of the target scenic spot, an augmented-reality-based content resource package corresponding to the current location area, where the augmented-reality-based content resource package includes at least six-degrees-of-freedom attitude data of the application device; and sending the augmented-reality-based content resource package corresponding to the current location area to the application device to enable the application device to provide the augmented-reality-based content resource package corresponding to the front location area for the user.

## Claims

### Claim 1

A tour guiding method, comprising:
receiving a video stream collected by a user in a current location area of a target scenic spot and sent by an application device;
determining, based on the collected video stream and prestored three-dimensional point cloud data of the target scenic spot, an augmented-reality-based content resource package corresponding to the current location area, wherein the augmented-reality-based content resource package comprises at least six-degrees-of-freedom attitude data of the application device; and
sending the augmented-reality-based content resource package corresponding to the current location area to the application device to enable the application device to provide the augmented-reality-based content resource package corresponding to the front location area for the user.

### Claim 2 (depends on 1)

The method of claim 1, before determining, based on the collected video stream and the prestored three-dimensional point cloud data of the target scenic spot, the augmented-reality-based content resource package corresponding to the current location area, the method further comprising:
classifying the collected video stream to obtain a visual-map data category corresponding to the collected video stream; and
extracting, based on the visual-map data category corresponding to the collected video stream, the three-dimensional point cloud data of the target scenic spot from a preconstructed visual-map database.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining, based on the collected video stream and the prestored three-dimensional point cloud data of the target scenic spot, the augmented-reality-based content resource package corresponding to the current location area comprises:
determining, based on the collected video stream and the three-dimensional point cloud data of the target scenic spot, three-dimensional point cloud data of the current location area by using a lightweight three-dimensional visual-map generation model; and
acquiring, based on the three-dimensional point cloud data of the current location area, the augmented-reality-based content resource package corresponding to the current location area.

### Claim 4 (depends on 2)

The method of claim 2, before receiving the video stream collected by the user in the current location area of the target scenic spot and sent by the application device, the method further comprising:
receiving a video recorded for the target scenic spot and sent by a production device; and
generating the three-dimensional point cloud data of the target scenic spot based on the video recorded for the target scenic spot and storing the three-dimensional point cloud data of the target scenic spot in the visual-map database.

### Claim 5 (depends on 4)

The method of claim 4, wherein receiving the video recorded for the target scenic spot and sent by the production device comprises:
in response to the target scenic spot being a single wall, receiving a video recorded for the target scenic spot by using a data collection method corresponding to the single wall and sent by the production device; in response to the target scenic spot being a single building, receiving a video recorded for the target scenic spot by using a data collection method corresponding to the single building and sent by the production device; in response to the target scenic spot being a courtyard type building, receiving a video recorded for the target scenic spot by using a data collection method corresponding to the courtyard type building and sent by the production device.

### Claim 6 (depends on 4)

The method of claim 4, before generating the three-dimensional point cloud data of the target scenic spot based on the video recorded for the target scenic spot, the method further comprising:
analyzing feasibility for the video recorded for the target scenic spot to obtain a feasibility analysis result of the video recorded for the target scenic spot; and
in response to the feasibility analysis result of the video recorded for the target scenic spot being feasible, performing the operation of generating the three-dimensional point cloud data of the target scenic spot based on the video recorded for the target scenic spot.

### Claim 7

A tour guiding method, comprising:
acquiring a video stream collected by a user in a current location area of a target scenic spot;
sending the collected video stream to a server device to enable the server device to determine, based on the collected video stream and prestored three-dimensional point cloud data of the target scenic spot, an augmented-reality-based content resource package corresponding to the current location area, wherein the augmented-reality-based content resource package comprises at least six-degrees-of-freedom attitude data of an application device; and
receiving the augmented-reality-based content resource package corresponding to the current location area and sent by the server device and providing the augmented-reality-based content resource package corresponding to the current location area for the user.

### Claim 8 (depends on 7)

The method of claim 7, wherein providing the augmented-reality-based content resource package corresponding to the current location area for the user comprises:
providing, based on a location-based service (LBS) of the Global Positioning System (GPS), the augmented-reality-based content resource package corresponding to the current location area for the user; or providing, based on a text-to-speech (TTS) audio broadcast service, the augmented-reality-based content resource package corresponding to the current location area for the user.

### Claim 9

An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the following steps:
receiving a video stream collected by a user in a current location area of a target scenic spot and sent by an application device;
determining, based on the collected video stream and prestored three-dimensional point cloud data of the target scenic spot, an augmented-reality-based content resource package corresponding to the current location area, wherein the augmented-reality-based content resource package comprises at least six-degrees-of-freedom attitude data of the application device; and
sending the augmented-reality-based content resource package corresponding to the current location area to the application device to enable the application device to provide the augmented-reality-based content resource package corresponding to the front location area for the user.

### Claim 10 (depends on 9)

The electronic device of claim 9, wherein the at least one processor is configured to further perform, before determining, based on the collected video stream and the prestored three-dimensional point cloud data of the target scenic spot, the augmented-reality-based content resource package corresponding to the current location area, the following steps:
classifying the collected video stream to obtain a visual-map data category corresponding to the collected video stream; and
extracting, based on the visual-map data category corresponding to the collected video stream, the three-dimensional point cloud data of the target scenic spot from a preconstructed visual-map database.

### Claim 11 (depends on 9)

The electronic device of claim 9, wherein the at least one processor is configured to perform determining, based on the collected video stream and the prestored three-dimensional point cloud data of the target scenic spot, the augmented-reality-based content resource package corresponding to the current location area by:
determining, based on the collected video stream and the three-dimensional point cloud data of the target scenic spot, three-dimensional point cloud data of the current location area by using a lightweight three-dimensional visual-map generation model; and
acquiring, based on the three-dimensional point cloud data of the current location area, the augmented-reality-based content resource package corresponding to the current location area.

### Claim 12 (depends on 10)

The electronic device of claim 10, the at least one processor is configured to further perform, before receiving the video stream collected by the user in the current location area of the target scenic spot and sent by the application device, the following steps:
receiving a video recorded for the target scenic spot and sent by a production device; and
generating the three-dimensional point cloud data of the target scenic spot based on the video recorded for the target scenic spot and storing the three-dimensional point cloud data of the target scenic spot in the visual-map database.

### Claim 13 (depends on 12)

The electronic device of claim 12, wherein the at least one processor is configured to perform receiving the video recorded for the target scenic spot and sent by the production device by:
in response to the target scenic spot being a single wall, receiving a video recorded for the target scenic spot by using a data collection method corresponding to the single wall and sent by the production device; in response to the target scenic spot being a single building, receiving a video recorded for the target scenic spot by using a data collection method corresponding to the single building and sent by the production device; in response to the target scenic spot being a courtyard type building, receiving a video recorded for the target scenic spot by using a data collection method corresponding to the courtyard type building and sent by the production device.

### Claim 14 (depends on 12)

The electronic device of claim 12, the at least one processor is configured to further perform, before generating the three-dimensional point cloud data of the target scenic spot based on the video recorded for the target scenic spot, the following steps:
analyzing feasibility for the video recorded for the target scenic spot to obtain a feasibility analysis result of the video recorded for the target scenic spot; and
in response to the feasibility analysis result of the video recorded for the target scenic spot being feasible, performing the operation of generating the three-dimensional point cloud data of the target scenic spot based on the video recorded for the target scenic spot.

### Claim 15 (depends on 7)

An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of claim 7.

### Claim 16 (depends on 15)

The electronic device of claim 15, wherein the at least one processor is configured to perform providing the augmented-reality-based content resource package corresponding to the current location area for the user by:
providing, based on a location-based service (LBS) of the Global Positioning System (GPS), the augmented-reality-based content resource package corresponding to the current location area for the user; or providing, based on a text-to-speech (TTS) audio broadcast service, the augmented-reality-based content resource package corresponding to the current location area for the user.

### Claim 17 (depends on 1)

A non-transitory computer-readable storage medium, storing computer instructions for causing a computer to perform the method of claim 1.

### Claim 18 (depends on 17)

The storage medium of claim 17, wherein the computer is configured to further perform, before determining, based on the collected video stream and the prestored three-dimensional point cloud data of the target scenic spot, the augmented-reality-based content resource package corresponding to the current location area, the following steps:
classifying the collected video stream to obtain a visual-map data category corresponding to the collected video stream; and
extracting, based on the visual-map data category corresponding to the collected video stream, the three-dimensional point cloud data of the target scenic spot from a preconstructed visual-map database.

### Claim 19 (depends on 17)

The storage medium of claim 17, wherein the computer is configured to perform determining, based on the collected video stream and the prestored three-dimensional point cloud data of the target scenic spot, the augmented-reality-based content resource package corresponding to the current location area by:
determining, based on the collected video stream and the three-dimensional point cloud data of the target scenic spot, three-dimensional point cloud data of the current location area by using a lightweight three-dimensional visual-map generation model; and
acquiring, based on the three-dimensional point cloud data of the current location area, the augmented-reality-based content resource package corresponding to the current location area.

### Claim 20 (depends on 7)

A non-transitory computer-readable storage medium, storing computer instructions for causing a computer to perform the method of claim 7.
