System
AssigneeSoftBank Group Corp.
InventorNatsumi KOBAYASHI
Life of this patent
Estimating the term…
Abstract
A system includes a processor that is configured to analyze prompt data input by a user for instructing a generative AI model to execute a specific processing operation, and search for visual information related to the prompt data by using the generative AI model, analyze the visual information by using the generative AI model and generate a descriptive sentence that explains content of the visual information, and generate, by using the generative AI model and based on the generated descriptive sentence, a summary sentence including information that an advertiser desires to add.
Claims20
A system comprising:
- circuitry configured to:
- receive, via a communication interface coupled to a packet-switched network, prompt data from a terminal device operated by a user, and analyze the prompt data using a natural language processing resource to extract feature terms;
- retrieve, based on the extracted feature terms, visual information from an information source via the packet-switched network, and select visual information from a plurality of pieces of acquired visual information in accordance with a relevance criterion;
- input the selected visual information to an image analysis resource and acquire analysis result data comprising object information and scene information;
- construct a prompt data structure for a generative neural network model based on the analysis result data, and cause the generative neural network model to generate a descriptive output sentence describing content of the visual information;
- cause the generative neural network model to generate, based on the descriptive output sentence and supplemental attribute data stored in a storage device, a composite output sentence incorporating the supplemental attribute data;
- and
- transmit, via the communication interface, the composite output sentence and the visual information to the terminal device for presentation via a user interface of the terminal device.
A system comprising:
- circuitry configured to:
- receive, via a communication interface coupled to a packet-switched network, prompt data from a terminal device, and analyze the prompt data using a natural language processing resource to extract feature terms;
- retrieve visual information based on the extracted feature terms from an information source via the packet-switched network, input the visual information to an image analysis resource comprising a convolutional neural network or a transformer-based vision model, and acquire analysis result data comprising object labels, scene labels, and confidence scores;
- construct a first prompt data structure based on the analysis result data and cause a generative neural network model comprising a transformer architecture with self-attention layers to generate a descriptive output sentence describing content of the visual information;
- construct a second prompt data structure comprising the descriptive output sentence and supplemental attribute data stored in a storage device, and cause the generative neural network model to generate a composite output sentence incorporating the supplemental attribute data;
- estimate an emotional state of a user of the terminal device and adjust at least one of a content or a presentation order of the composite output sentence based on the estimated emotional state;
- and
- transmit the composite output sentence and the visual information to the terminal device for presentation via a user interface of the terminal device.
A method comprising:
- receiving, via a communication interface coupled to a packet-switched network, prompt data from a terminal device operated by a user, and analyzing the prompt data using a natural language processing resource to extract feature terms;
- retrieving, based on the extracted feature terms, visual information from an information source via the packet-switched network, and selecting visual information from a plurality of pieces of acquired visual information in accordance with a relevance criterion;
- inputting the selected visual information to an image analysis resource and acquiring analysis result data comprising object information and scene information;
- constructing a prompt data structure for a generative neural network model based on the analysis result data, and causing the generative neural network model to generate a descriptive output sentence describing content of the visual information;
- causing the generative neural network model to generate, based on the descriptive output sentence and supplemental attribute data stored in a storage device, a composite output sentence incorporating the supplemental attribute data;
- and
- transmitting, via the communication interface, the composite output sentence and the visual information to the terminal device for presentation via a user interface of the terminal device.
Dependencies are inferred from the claim text.