# US 2021/0174534 A1 · Ai apparatus and method for determining location of user

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174534 A1
- Document ID: US20210174534A1
- Status: Published application
- Kind: A1
- Application number: 16/488,307
- Filed: 2019-06-05
- Published: 2021-06-10
- Earliest claimed: 2019-06-05
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Sihyuk YI
- Web page: https://patentgrep.com/patent/US20210174534A1

## CPC

- G01S 5/20: Position of source determined by a plurality of spaced direction-finders
- G01S 5/28: by co-ordinating position lines of different shape, e.g. hyperbolic, circular, elliptical or radial
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/73: using feature-based methods
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/20: in augmented reality scenes
- G06V 40/19: Sensors therefor
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/24: Speech recognition using non-acoustical features
- G10L 25/51: for comparison or discrimination
- G01S 3/8083: {determining direction of source}
- G06N 3/08: Learning methods
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06T 2207/30244: Camera pose
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Provided is an AI apparatus for determining a location of a user including: a communication unit configured to communicate with at least one external AI apparatus obtaining first image data and first sound data; a memory configured to store location information on the at least one external AI apparatus and the AI apparatus; a camera configured to obtain second image data; a microphone configured to obtain second sound data; and a processor configured to: generate first recognition information on the user based on the second image data; generate second recognition information on the user based on the second sound data; obtain, from the at least one external AI apparatus, third recognition information on the user generated based on the first image data and fourth recognition information on the user generated based on the first sound data; determine the user's location based on the location information, the first recognition information, and the third recognition information; and calibrate the determined user's location based on the second recognition information and the fourth recognition information.

## Claims

### Claim 1

An AI apparatus for determining a location of a user, the AI apparatus comprising:
a communication unit configured to communicate with at least one external AI apparatus obtaining first image data and first sound data;
a memory configured to store location information on the at least one external AI apparatus and the AI apparatus;
a camera configured to obtain second image data;
a microphone configured to obtain second sound data; and
a processor configured to:
generate first recognition information on the user based on the second image data;
generate second recognition information on the user based on the second sound data;
obtain, from the at least one external AI apparatus, third recognition information on the user generated based on the first image data and fourth recognition information on the user generated based on the first sound data;
determine the user's location based on the location information, the first recognition information, and the third recognition information; and
calibrate the determined user's location based on the second recognition information and the fourth recognition information.

### Claim 2 (depends on 1)

The AI apparatus of claim 1, wherein the location information includes distance information from the AI apparatus to the at least one external AI apparatus and camera direction information of the at least one external AI apparatus,
wherein each of the first recognition information to the fourth recognition information includes direction information of the user with respect to the corresponding AI apparatus or at least one external AI apparatus,
wherein the processor is configured to determine and calibrate the location of the user based on the distance information, the camera direction information, and the direction information of the user.

### Claim 3 (depends on 2)

The AI apparatus of claim 2, wherein the location information is set by input of the user or determined based on image recognition information generated based on the first image data and the second image data when the AI apparatus or the at least one external AI apparatus is installed.

### Claim 4 (depends on 3)

The AI apparatus of claim 3, wherein the location information includes locations in map data corresponding to installation spaces of the at least one external AI apparatus and the AI apparatus, respectively, and
wherein the determined user location includes a location of the user in the map data.

### Claim 5 (depends on 1)

The AI apparatus of claim 1, the processor is configured to:
assign higher priorities to the recognition information respectively acquired from fixed AI apparatuses that has not moved for a predetermined time among the AI apparatus and the at least one external AI apparatus than recognition information respectively acquired from non-fixed AI apparatuses; and
determine the location of the user in consideration of the priorities for the recognition information.

### Claim 6 (depends on 1)

The AI apparatus of claim 1, wherein the processor is configured to:
determine the location of the user when utterance voice data is acquired from the user; and
determine a target object corresponding to the utterance voice data using the determined location of the user.

### Claim 7 (depends on 6)

The AI apparatus of claim 6, wherein the processor is configured to:
acquire action information of the user based on the second image data; and
determine the target object corresponding to the utterance voice data using the action information.

### Claim 8 (depends on 6)

The AI apparatus of claim 6, wherein the processor is configured to:
acquire gaze information of the user based on the second image data; and
determine the target object corresponding to the utterance voice data using the gaze information.

### Claim 9 (depends on 1)

The AI apparatus of claim 1, wherein the processor is configured to generate the first recognition information or the second recognition information using a user recognition model learned by using a machine learning algorithm or a deep learning algorithm.

### Claim 10

A method for determining a location of a user using an AI apparatus, the method comprising:
obtaining location information of at least one external AI apparatus obtaining first image data and first sound data and the AI apparatus;
obtaining second image data;
obtaining second sound data;
generating first recognition information on the user based on the second image data;
generating second recognition information on the user based on the second sound data;
obtaining, from the at least one external AI apparatus, third recognition information on the user generated based on the first image data and fourth recognition information on the user generated based on the first sound data;
determining the user's location based on the location information, the first recognition information, and the third recognition information; and
calibrating the determined user's location based on the second recognition information and the fourth recognition information.

### Claim 11

A storage medium having a program stored therein, wherein the program is configured for performing a method for determining a location of a user using an AI apparatus, the method including:
obtaining location information of at least one external AI apparatus obtaining first image data and first sound data and the AI apparatus;
obtaining second image data;
obtaining second sound data;
generating first recognition information on the user based on the second image data;
generating second recognition information on the user based on the second sound data;
obtaining, from the at least one external AI apparatus, third recognition information on the user generated based on the first image data and fourth recognition information on the user generated based on the first sound data;
determining the user's location based on the location information, the first recognition information, and the third recognition information; and
calibrating the determined user's location based on the second recognition information and the fourth recognition information.
