# US 2024/0202999 A1 · Method and apparatus for organizing and merging scene understanding information of artificial intelligence agent

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202999 A1
- Document ID: US20240202999A1
- Status: Published application
- Kind: A1
- Application number: 18/480,177
- Filed: 2023-10-03
- Published: 2024-06-20
- Earliest claimed: 2022-12-19
- Assignees: Electronics and Telecommunications Research Institute
- Applicants: Electronics and Telecommunications Research Institute
- Inventors: Yong-Ju LEE
- Web page: https://patentgrep.com/patent/US20240202999A1

## CPC

- G06T 11/26: Drawing of charts or graphs
- G06V 10/82: using neural networks
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V 2201/07: Target detection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Disclosed herein is a method for organizing and merging scene understanding information of an AI agent. The method includes acquiring an image of a first space, recognizing objects in the image, structuring information about the relationship between the object and information about the states of the objects in the form of a graph, and merging the structured information with information received from a nearby AI agent.

## Claims

### Claim 1

A method for organizing and merging a scene of an Artificial Intelligence (AI) agent, comprising:
acquiring an image of a first space;
recognizing objects in the image;
structuring information about a relationship between the objects and information about states of the objects in a form of a graph; and
merging the structured information with information received from a nearby AI agent.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the objects are recognized based on a first AI neural network that is pretrained by receiving an image of a specific space as input, and
the information about the states of the objects is acquired by inputting the objects to a second AI neural network that is trained based on training data formed by labeling objects with respective states.

### Claim 3 (depends on 1)

The method of claim 1, wherein merging the structured information comprises merging information corresponding to the first space with information corresponding to a second space of the nearby AI agent.

### Claim 4 (depends on 3)

The method of claim 3, wherein the merged information is configured such that a positional relationship between the first space and the second space is represented in a form of a link.

### Claim 5 (depends on 1)

The method of claim 1, wherein
merging the structured information comprises determining whether to merge the structured information with the information received from the nearby AI agent based on metadata of the nearby AI agent, and
the metadata includes timestamp information corresponding to the information about the states of the objects.

### Claim 6 (depends on 5)

The method of claim 5, wherein merging the structured information comprises merging the information about the states of the objects determined to have been updated within a preset time period back from a current time using the timestamp information.

### Claim 7 (depends on 1)

The method of claim 1, wherein merging the structured information comprises exchanging information about an object, a state of which changes over time, with the nearby AI agent.

### Claim 8 (depends on 1)

The method of claim 1, wherein merging the structured information comprises exchanging information in units of sentences generated based on the information about the states of the objects with the nearby AI agent.

### Claim 9

An apparatus for organizing and merging a scene of an Artificial Intelligence (AI) agent, comprising:
memory in which at least one program is recorded; and
a processor for executing the program,
wherein the program includes instructions for performing
acquiring an image of a first space,
recognizing objects in the image,
structuring information about a relationship between the objects and information about states of the objects in a form of a graph; and
merging the structured information with information received from a nearby AI agent.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein:
the objects are recognized based on a first AI neural network that is pretrained by receiving an image of a specific space as input, and
the information about the states of the objects is acquired by inputting the objects to a second AI neural network that is trained based on training data formed by labeling objects with respective states.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein merging the structured information comprises merging information corresponding to the first space with information corresponding to a second space of the nearby AI agent.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the merged information is configured such that a positional relationship between the first space and the second space is represented in a form of a link.

### Claim 13 (depends on 9)

The apparatus of claim 9, wherein
merging the structured information comprises determining whether to merge the structured information with the information received from the nearby AI agent based on metadata of the nearby AI agent, and
the metadata includes timestamp information corresponding to the information about the states of the objects.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein merging the structured information comprises merging the information about the states of the objects determined to have been updated within a preset time period back from a current time using the timestamp information.

### Claim 15 (depends on 9)

The apparatus of claim 9, wherein merging the structured information comprises exchanging information about an object, a state of which changes over time, with the nearby AI agent.

### Claim 16 (depends on 9)

The apparatus of claim 9, wherein merging the structured information comprises exchanging information in units of sentences generated based on the information about the states of the objects with the nearby AI agent.
