# US 2021/0067684 A1 · Equipment utilizing human recognition and method for utilizing the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0067684 A1
- Document ID: US20210067684A1
- Status: Published application
- Kind: A1
- Application number: 16/688,823
- Filed: 2019-11-19
- Published: 2021-03-04
- Earliest claimed: 2019-08-27
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Young Han KIM; Sang Hyun LEE; Sang Hyun JUNG
- Web page: https://patentgrep.com/patent/US20210067684A1

## CPC

- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06F 18/23: Clustering techniques
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06V 10/762: using clustering, e.g. of similar faces in social networks
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 40/16: Human faces, e.g. facial parts, sketches or expressions
- G06V 40/165: {using facial parts and geometric relationships}
- G06V 40/172: {Classification, e.g. identification}
- H04N 23/611: where the recognised objects include parts of the human body
- H04N 23/63: by using electronic viewfinders
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- G06N 3/043: based on fuzzy logic, fuzzy membership or fuzzy inference, e.g. adaptive neuro-fuzzy inference systems [ANFIS]
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for utilizing human recognition and a method utilizing the same are provided. The method for utilizing human recognition includes updating a moving image database to include information about a moving image in which a cluster subject appears, the information being extracted based on clustering using a face feature; receiving a search condition; and detecting moving image information using the database. According to the present disclosure, a skeleton can be analyzed and a face can be recognized using an artificial intelligence (AI) model performing deep learning through a fifth generation (5G) network, and using the analysis result, a photographing composition can be determined, and moving image information can be constructed at an edge.

## Claims

### Claim 1

A method for human recognition, the method comprising:
extracting information from a first moving image in which a cluster subject appears, wherein the extracting is based on clustering using a face feature of the cluster subject depicted in the first moving image;
storing, in a moving image database, the information from the first moving image;
receiving a search condition for searching for moving images in which the cluster subject appears, wherein the moving images are included in the moving image database; and
searching the moving image database to identify one or more moving images, among the moving images included in the moving image database, which meet the search condition.

### Claim 2 (depends on 1)

The method of claim 1, wherein the storing occurs according to an update time based on time information and a determination of being in an idle mode.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
extracting a face feature of a person for each frame of the first moving image;
forming a cluster for each cluster subject using the face feature and a clustering model based on machine learning;
generating further information about a section in the first moving image in which the each cluster subject appears; and
storing, in the moving image database, the further information.

### Claim 4 (depends on 3)

The method of claim 3, wherein the forming of the cluster for each cluster subject comprises:
determining a similarity of the face feature; and
forming a cluster for each cluster subject based on the similarity.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
performing the searching using a face feature of a person as the search condition.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
performing the searching using as the search condition information about a person selected from a list of people extracted from the moving image database.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
displaying, on a display, information about the identified one or more moving images that includes at least one of a thumbnail, a moving image identification (ID), a person identification (ID), or information about a section in which a person appears.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
performing, according to a mobile gesture by a user, at least one of playing back a selected section in which a person appears, continuously playing back the selected section in which the person appears, and moving to and playing back another section in which the person appears.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
generating a highlight image using a moving image of a section in which the cluster subject appears, based on the information from the first moving image.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
generating a moving image edited using a moving image of a section in which the cluster subject appears, based on the information from the first moving image.

### Claim 11

An apparatus for human recognition, the apparatus comprising:
a user input interface; and
a processor configured to:
extract information from a first moving image in which a cluster subject appears, wherein the extract is based on clustering using a face feature of the cluster subject depicted in the first moving image;
store, in a moving image database, the information from the first moving image;
receive, via the user input interface, a search condition for searching for moving images in which the cluster subject appears, wherein the moving images are included in the moving image database; and
search the moving image database to identify one or more moving images, among the moving images included in the moving image database, which meet the search condition.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the store occurs according to an update time based on time information and a determination of being in an idle mode.

### Claim 13 (depends on 11)

The apparatus of claim 11, wherein the processor is further configured to:
extract a face feature of a person for each frame of the first moving image;
form a cluster for each cluster subject using the face feature and a clustering model based on machine learning;
generate further information about a section in the first moving image in which the each cluster subject appears; and
store, in the moving image database, the further information.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the form the cluster for each cluster subject comprises:
determine a similarity of the face feature; and
form a cluster for each cluster subject based on the similarity.

### Claim 15 (depends on 11)

The apparatus of claim 11, wherein the processor is further configured to:
perform the search using a face feature of a person as the search condition.

### Claim 16 (depends on 11)

The apparatus of claim 11, wherein the processor is further configured to:
perform the search using as the search condition information about a person selected from a list of people extracted from the moving image database.

### Claim 17 (depends on 11)

The apparatus of claim 11, further comprising:
a display, wherein the processor is further configured to:
cause the display to display information about the identified one or more moving images that includes at least one of a thumbnail, a moving image identification (ID), a person identification (ID), or information about a section in which a person appears.

### Claim 18 (depends on 11)

The apparatus of claim 11, wherein the processor is further configured to:
perform, according to a mobile gesture by a user, at least one of playing back a selected section in which a person appears, continuously playing back the selected section in which the person appears, and moving to and playing back another section in which the person appears.

### Claim 19 (depends on 11)

The apparatus of claim 11, wherein the processor is further configured to:
generate a highlight image using a moving image of a section in which the cluster subject appears, based on the information from the first moving image.

### Claim 20 (depends on 11)

The apparatus of claim 11, wherein the processor is further configured to:
generate a moving image edited using a moving image of a section in which the cluster subject appears, based on the information from the first moving image.
