# US 2026/0288860 A1 · System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0288860 A1
- Document ID: US20260288860A1
- Status: Published application
- Kind: A1
- Application number: 19/561,682
- Filed: 2026-03-10
- Published: 2026-09-24
- Earliest claimed: 2025-03-19
- Assignees: SoftBank Group Corp.
- Applicants: SoftBank Group Corp.
- Inventors: Tsukasa SEKITA
- Web page: https://patentgrep.com/patent/US20260288860A1

## CPC

- G06F 16/583: using metadata automatically derived from the content
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system includes a processor that is configured to extract visual features from an image or a video, generate a prompt sentence that instructs a search of an information set based on the extracted visual features, and search the information set by using a generative AI model and generate an information list having high relevance.

## Claims

### Claim 1

A system comprising:
circuitry configured to:
acquire, via a communication interface coupled to a packet-switched network, media data comprising visual frame data transmitted from a terminal device;
convert the media data into a normalized tensor representation and extract a multidimensional feature vector from the normalized tensor representation by propagating the normalized tensor representation through a neural network feature extractor;
construct a prompt data structure on the basis of the multidimensional feature vector and transmit the prompt data structure to a generative neural network model to obtain response data comprising candidate identifiers;
compute a similarity measure between the multidimensional feature vector and a stored feature representation associated with each of the candidate identifiers to generate a relevance-scored data structure;
apply a ranking model to the relevance-scored data structure to compute a ranking score for each candidate identifier and generate a ranked data structure ordered by the ranking scores; and
transmit the ranked data structure to the terminal device via the packet-switched network.

### Claim 2 (depends on 1)

The system according to claim 1, wherein the circuitry is configured to convert the media data into the normalized tensor representation by decoding the visual frame data into pixel matrices, resizing the pixel matrices to a predetermined resolution, and normalizing pixel values to a predetermined numeric range, and to extract low-level feature descriptors comprising at least a color histogram computed over a selected color space, an edge descriptor derived from edge detection filtering, and a texture descriptor derived from local pattern operators.

### Claim 3 (depends on 2)

The system according to claim 2, wherein the neural network feature extractor comprises a convolutional neural network including a plurality of convolutional layers, batch normalization layers, and non-linear activation layers, and wherein the circuitry is configured to remove a final classification layer of the convolutional neural network and obtain the multidimensional feature vector from an intermediate feature extraction layer as a fixed-dimensional dense vector encoding semantic properties of the media data.

### Claim 4 (depends on 3)

The system according to claim 3, wherein the media data further comprises a temporal sequence of visual frames, and wherein the circuitry is configured to extract motion descriptors from inter-frame differences or motion vector histograms computed across the temporal sequence, and to concatenate the motion descriptors with the multidimensional feature vector to generate an integrated feature vector representing both spatial and temporal characteristics.

### Claim 5 (depends on 4)

The system according to claim 4, wherein the circuitry is configured to apply dimensionality reduction to the integrated feature vector by projecting the integrated feature vector into a lower-dimensional subspace, and to store the reduced integrated feature vector in a feature store indexed by a media identifier for subsequent similarity retrieval.

### Claim 6 (depends on 1)

The system according to claim 1, wherein the circuitry is configured to construct the prompt data structure by mapping numerical ranges of the multidimensional feature vector to qualitative descriptive labels representing visual characteristics of the media data, and assembling the qualitative descriptive labels together with metadata of the candidate identifiers into a natural language template that specifies ranking criteria and output format instructions for the generative neural network model.

### Claim 7 (depends on 6)

The system according to claim 6, wherein the circuitry is configured to parse the response data received from the generative neural network model by segmenting output text into individual candidate entries using pattern matching, extracting identifiers and associated descriptive attributes from each candidate entry, and matching the extracted identifiers against records in a storage device to obtain complete metadata for each candidate identifier.

### Claim 8 (depends on 7)

The system according to claim 7, wherein the circuitry is configured to store, as record information, the prompt data structure, the candidate identifiers, and the response data received from the generative neural network model, and to analyze an accumulation of the record information to update generation conditions used when constructing a subsequent prompt data structure, thereby iteratively refining the prompt construction logic on the basis of observed outcomes.

### Claim 9 (depends on 1)

The system according to claim 1, wherein the circuitry is configured to acquire preference history data of a user associated with the terminal device from a storage device, perform aggregation processing and recency-based weighting processing on the preference history data to generate preference profile data representing a preference tendency of the user, and include the preference profile data in the prompt data structure transmitted to the generative neural network model.

### Claim 10 (depends on 9)

The system according to claim 9, wherein the ranking model comprises a neural network including embedding layers for user feature vectors and item feature vectors, concatenation layers, and fully connected layers with non-linear activations, and wherein the circuitry is configured to compute the ranking score as an individuality evaluation value representing an estimated probability that the user will select a given candidate identifier.

### Claim 11 (depends on 10)

The system according to claim 10, wherein the circuitry is configured to receive selection result data from the terminal device indicating which candidate identifiers the user selected from the ranked data structure, record the selection result data in the storage device as updated preference history data, and retrain or incrementally update parameters of the ranking model on the basis of the updated preference history data using a gradient-based optimization method with a pairwise ranking loss function.

### Claim 12 (depends on 1)

The system according to claim 1, wherein the circuitry is configured to apply an emotion analysis model to data received from the terminal device to determine an emotional state of a user, and to include information representing the determined emotional state in the prompt data structure transmitted to the generative neural network model, such that the response data reflects the emotional state in selection or ordering of the candidate identifiers.

### Claim 13 (depends on 12)

The system according to claim 12, wherein the emotion analysis model maps input data to a probability distribution over a plurality of emotion categories arranged on an emotion map with valence and arousal dimensions, and wherein the circuitry is configured to derive an affective label from a region of the emotion map corresponding to a highest-probability emotion category and embed the affective label as a constraint term in the prompt data structure.

### Claim 14 (depends on 1)

The system according to claim 1, wherein the circuitry is configured to acquire content items corresponding to the candidate identifiers in the ranked data structure from an external information providing service via an application programming interface, and to attach metadata of the acquired content items to corresponding entries in the ranked data structure before transmitting the ranked data structure to the terminal device.

### Claim 15 (depends on 14)

The system according to claim 14, wherein the content items comprise at least one of audiovisual media items, product items, and document items, and wherein the circuitry is configured to filter the content items on the basis of availability constraints and region compatibility constraints before including the content items in the ranked data structure.

### Claim 16 (depends on 1)

The system according to claim 1, wherein the terminal device comprises at least one of a smart device having a display and a touch panel, smart glasses having a speaker and a microphone, a headset-type terminal having a display and a speaker, and a robot having a speaker and a motor-driven control target, and wherein the circuitry is configured to format the ranked data structure in a presentation format corresponding to an output capability of the terminal device.

### Claim 17 (depends on 16)

The system according to claim 16, wherein the circuitry is configured to include, in the ranked data structure, explanation information generated by the generative neural network model indicating a reason for each candidate identifier being included in the ranked data structure, and to transmit the explanation information together with the ranked data structure to the terminal device for rendering in association with each candidate identifier.

### Claim 18

A system comprising:
circuitry configured to:
acquire, via a communication interface coupled to a packet-switched network, media data comprising visual frame data including at least one of image data and video data transmitted from a terminal device;
convert the media data into a normalized tensor representation by decoding the visual frame data into pixel matrices and normalizing pixel values, and extract a multidimensional feature vector by propagating the normalized tensor representation through a convolutional neural network feature extractor comprising a plurality of convolutional layers and obtaining activations from an intermediate feature extraction layer;
construct a prompt data structure by mapping numerical ranges of the multidimensional feature vector to qualitative descriptive labels and assembling the qualitative descriptive labels with preference profile data of a user into a natural language template, and transmit the prompt data structure to a generative neural network model comprising a Transformer-based architecture that predicts output tokens via softmax probability computation to obtain response data comprising candidate identifiers with associated descriptive attributes;
compute a similarity measure between the multidimensional feature vector and a stored feature representation associated with each of the candidate identifiers using a vector similarity operation to generate a relevance-scored data structure;
apply a ranking model comprising a neural network with embedding layers and fully connected layers to the relevance-scored data structure and the preference profile data to compute an individuality evaluation value for each candidate identifier, and generate a ranked data structure ordered by the individuality evaluation values;
store, as record information, the prompt data structure, the candidate identifiers, and the response data, and update generation conditions for constructing subsequent prompt data structures on the basis of accumulated record information; and
transmit the ranked data structure to the terminal device via the packet-switched network.

### Claim 19 (depends on 18)

The system according to claim 18, wherein the circuitry is configured to receive selection result data from the terminal device indicating which candidate identifiers the user selected, record the selection result data as updated preference history data, and incrementally update parameters of the ranking model using a gradient-based optimization method on the basis of the updated preference history data, and wherein the circuitry is configured to apply an emotion analysis model to data received from the terminal device to determine an emotional state and include information representing the emotional state in a subsequent prompt data structure.

### Claim 20

A method performed by circuitry of a server coupled to a packet-switched network via a communication interface, the method comprising:
acquiring, via the communication interface, media data comprising visual frame data transmitted from a terminal device;
converting the media data into a normalized tensor representation and extracting a multidimensional feature vector from the normalized tensor representation by propagating the normalized tensor representation through a neural network feature extractor;
constructing a prompt data structure on the basis of the multidimensional feature vector and transmitting the prompt data structure to a generative neural network model to obtain response data comprising candidate identifiers;
computing a similarity measure between the multidimensional feature vector and a stored feature representation associated with each of the candidate identifiers to generate a relevance-scored data structure;
applying a ranking model to the relevance-scored data structure to compute a ranking score for each candidate identifier and generating a ranked data structure ordered by the ranking scores; and
transmitting the ranked data structure to the terminal device via the packet-switched network.
