# US 2026/0187983 A1 · Information processing device, information processing method, and recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0187983 A1
- Document ID: US20260187983A1
- Status: Published application
- Kind: A1
- Application number: 19/420,974
- Filed: 2025-12-16
- Published: 2026-07-02
- Earliest claimed: 2024-12-26
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Chun Yang TAN; Yasunori Babazaki
- Web page: https://patentgrep.com/patent/US20260187983A1

## CPC

- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/248: {by interactive preprocessing or interactive shape modelling, e.g. feature points assigned by a user}
- G06V 10/40: Extraction of image or video features
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 30/1613: {Interactive preprocessing or shape modelling, e.g. assignment of feature points by a user}
- G06V 30/19093: {Proximity measures, i.e. similarity or distance measures}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06V 10/82: using neural networks
- G06V 30/18152: {Extracting features based on a plurality of salient regional features, e.g. "bag of words"}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

In the information processing device, the visual feature extraction unit extracts visual features from object visual information. The text feature extraction unit extracts text features from a text related to the visual features. The feature adjustment unit performs feature alignment of the text features and the visual features based on a mutual relationship between the text features and the visual features.

## Claims

### Claim 1

An information processing device comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to:
extract visual features from visual information;
extract text features from a text related to the visual features; and
perform feature adjustment of the text features and the visual features based on a mutual relationship between the text features and the visual features.

### Claim 2 (depends on 1)

The information processing device according to claim 1, wherein the at least one processor performs the feature adjustment of the text features and the visual features in a same feature space.

### Claim 3 (depends on 1)

The information processing device according to claim 1, wherein the at least one processor performs the feature adjustment by:
performing the feature alignment of the text features;
performing the feature alignment of the visual features; and
performing mutual adjustment of reflecting feature alignment information for performing the feature alignment based on the mutual relationship between the text features and the visual features on at least one of the feature alignment of the text features and the feature alignment of the visual features.

### Claim 4 (depends on 3)

The information processing device according to claim 3, wherein the at least one processor determines the mutual relationship between the text features and the visual features based on an input to each of the feature alignment of the text features and the feature alignment of the the visual features, an output of each of the feature alignment of the text features and the feature alignment of the the visual features, or intermediate features of each of the feature alignment of the text features and the feature alignment of the the visual features.

### Claim 5 (depends on 3)

The information processing device according to claim 3, wherein the at least one processor determines the mutual relationship between the text features and the visual features by using a cross attention mechanism, and generates the feature alignment information.

### Claim 6 (depends on 1)

The information processing device according to claim 1, wherein the at least one processor generates the text feature by merging features extracted from a plurality of texts.

### Claim 7 (depends on 1)

The information processing device according to claim 1, wherein the at least one processor is further configured to execute the instructions to classify the visual information into one of a plurality of texts based on similarity between the text features obtained by the feature alignment and the visual features obtained by the feature alignment.

### Claim 8 (depends on 7)

The information processing device according to claim 7,
wherein the visual information is information obtained by capturing an action state of a person,
wherein the text indicates an action of the person, and
wherein the at least one processor recognizes the action of the person included in the visual information.

### Claim 9

An information processing method executed by a computer, comprising:
extracting visual features from visual information;
extracting text features from a text related to the visual features; and
performing feature alignment of the text features and the visual features based on a mutual relationship between the text features and the visual features.

### Claim 10

A non-transitory computer-readable recording medium storing a program, the program causing a computer to execute processing comprising:
extracting visual features from visual information;
extracting text features from a text related to the visual features; and
performing feature alignment of the text features and the visual features based on a mutual relationship between the text features and the visual features.
