# US 2025/0095008 A1 · Multi-modal data-driven design concept evaluator > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0095008 A1 - Document ID: US20250095008A1 - Status: Published application - Kind: A1 - Application number: 18/726,669 - Filed: 2023-01-04 - Published: 2025-03-20 - Earliest claimed: 2022-01-05 - Assignees: Northeastern University - Applicants: Northeastern University - Inventors: Mohsen Moghaddam; Tucker J. Marion; Chenxi Yuan; Yi Han - Web page: https://patentgrep.com/patent/US20250095008A1 ## CPC - G06N 3/0475: Generative networks - G06Q 30/0202: Market predictions or forecasting for commercial activities - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/048: Activation functions - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N 3/09: Supervised learning - G06N 3/094: Adversarial learning - G06Q 30/0203: Market surveys; Market polls - G06Q 30/0282: Rating or review of business operators or products - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract A computer-implemented method for predicting customer sentiment for a product is provided. Customer data is received for multiple products and a vector of customer sentiments is generated that is associated with different aspects of the products. Images are received for multiple products and a latent vector for the images are generated using a pre-trained image processing model. Textual description for products is received and a latent vector for the textual description is generated using a pre-trained natural language processing model. A deep multimodal design evaluation (DMDE) model is designed to integrate the latent vectors of the images and the textual descriptions to predict customer sentiment for new product designs based on their images and textual descriptions. A new product design is provided to the trained DMDE model to obtain predicted customer sentiments for one or more attributes or a new product design is generated with favorable predicted customer sentiments. ## Claims ### Claim 1 A computer-implemented method of predicting customer sentiment for a product and aspects thereof, comprising the steps of: receiving customer data for a plurality of products, and generating a vector of customer sentiments associated with different aspects of each of the plurality of products based on the customer data; receiving images for each of the plurality of products, and generating a latent vector for the images for each product by fine-tuning a pre-trained image processing model; receiving a textual description for each of the plurality of products, and generating a latent vector for the textual description for each product by fine-tuning a pre-trained natural language processing model; designing a deep multimodal design evaluation (DMDE) model with a self-attention fusion mechanism to integrate the latent vectors of the images and the textual descriptions for each product to predict customer sentiment for new product designs based on their images and textual descriptions; and providing a new product design to the trained DMDE model to obtain predicted customer sentiments for one or more attributes of the new product design or generating a new product design having one or more attributes having favorable predicted customer sentiments using the trained DMDE model and a generative design model. ### Claim 2 (depends on 1) The method of claim 1, wherein the customer data comprises customer reviews or customer survey results. ### Claim 3 (depends on 2) The method of claim 2, wherein the customer reviews comprise online reviews posted by customers scraped from online sources. ### Claim 4 (depends on 1) The method of claim 1, wherein generating the latent vector for the images comprises, for the customer data for each product, identifying the attributes of the product discussed in the customer data, identifying the sentiments expressed for each attribute of the product, and identifying an intensity and polarity of each sentiment. ### Claim 5 (depends on 4) The method of claim 4, wherein generating the latent vector for the images further comprises aggregating customer sentiments identified for each product. ### Claim 6 (depends on 1) The method of claim 1, further comprising combining the latent vector for the textual description for each product and the latent vector for the images for each product prior to designing the DMDE model using a multimodal data concatenation process. ### Claim 7 (depends on 1) The method of claim 1, wherein the DMDE model and the generative design model comprise neural network models. ### Claim 8 (depends on 1) The method of claim 1, wherein the images comprise multiple different views of each of the plurality of products. ### Claim 9 A computer program product for predicting customer sentiment for a product and aspects thereof, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising: receiving customer data for a plurality of products, and generating a vector of customer sentiments associated with different aspects of each of the plurality of products based on the customer data; receiving images for each of the plurality of products, and generating a latent vector for the images for each product by fine-tuning a pre-trained image processing model; receiving a textual description for each of the plurality of products, and generating a latent vector for the textual description for each product by fine-tuning a pre-trained natural language processing model; designing a deep multimodal design evaluation (DMDE) model with a self-attention fusion mechanism to integrate the latent vectors of the images and the textual descriptions for each product to predict customer sentiment for new product designs based on their images and textual descriptions; and providing a new product design to the trained DMDE model to obtain predicted customer sentiments for one or more attributes of the new product design or generating a new product design having one or more attributes having favorable predicted customer sentiments using the trained DMDE model and a generative design model. ### Claim 10 (depends on 9) The computer program product of claim 9, wherein the customer data comprises customer reviews or customer survey results. ### Claim 11 (depends on 10) The computer program product of claim 10, wherein the customer reviews comprise online reviews posted by customers scraped from online sources. ### Claim 12 (depends on 9) The computer program product of claim 9, wherein generating the latent vector for the images comprises, for the customer data for each product, identifying the attributes of the product discussed in the customer data, identifying the sentiments expressed for each attribute of the product, and identifying an intensity and polarity of each sentiment. ### Claim 13 (depends on 12) The computer program product of claim 12, wherein generating the latent vector for the images further comprises aggregating customer sentiments identified for each product. ### Claim 14 (depends on 9) The computer program product of claim 9, wherein the method further comprises combining the latent vector for the textual description for each product and the latent vector for the images for each product prior to designing the DMDE model using a multimodal data concatenation process. ### Claim 15 (depends on 9) The computer program product of claim 9, wherein the DMDE model and the generative design model comprise neural network models. ### Claim 16 (depends on 9) The computer program product of claim 9, wherein the images comprise multiple different views of each of the plurality of products. ### Claim 17 A computer system, comprising: at least one processor; memory associated with the at least one processor; and a program stored in the memory for predicting customer sentiment for a product and aspects thereof, the program containing a plurality of instructions which, when executed by the at least one processor, cause the at least one processor to: receiving customer data for a plurality of products, and generating a vector of customer sentiments associated with different aspects of each of the plurality of products based on the customer data; receiving images for each of the plurality of products, and generating a latent vector for the images for each product by fine-tuning a pre-trained image processing model; receiving a textual description for each of the plurality of products, and generating a latent vector for the textual description for each product by fine-tuning a pre-trained natural language processing model; designing a deep multimodal design evaluation (DMDE) model with a self-attention fusion mechanism to integrate the latent vectors of the images and the textual descriptions for each product to predict customer sentiment for new product designs based on their images and textual descriptions; and providing a new product design to the trained DMDE model to obtain predicted customer sentiments for one or more attributes of the new product design or generating a new product design having one or more attributes having favorable predicted customer sentiments using the trained DMDE model and a generative design model. ### Claim 18 (depends on 17) The computer system of claim 17, wherein the customer data comprises customer reviews or customer survey results. ### Claim 19 (depends on 18) The computer system of claim 18, wherein the customer reviews comprise online reviews posted by customers scraped from online sources. ### Claim 20 (depends on 17) The computer system of claim 17, wherein generating the latent vector for the images comprises, for the customer data for each product, identifying the attributes of the product discussed in the customer data, identifying the sentiments expressed for each attribute of the product, and identifying an intensity and polarity of each sentiment.