Multi-modal data-driven design concept evaluator

A computer program product for predicting customer sentiment for a product and aspects thereof, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:

• receiving customer data for a plurality of products, and generating a vector of customer sentiments associated with different aspects of each of the plurality of products based on the customer data;
• receiving images for each of the plurality of products, and generating a latent vector for the images for each product by fine-tuning a pre-trained image processing model;
• receiving a textual description for each of the plurality of products, and generating a latent vector for the textual description for each product by fine-tuning a pre-trained natural language processing model;
• designing a deep multimodal design evaluation (DMDE) model with a self-attention fusion mechanism to integrate the latent vectors of the images and the textual descriptions for each product to predict customer sentiment for new product designs based on their images and textual descriptions;
• and
• providing a new product design to the trained DMDE model to obtain predicted customer sentiments for one or more attributes of the new product design or generating a new product design having one or more attributes having favorable predicted customer sentiments using the trained DMDE model and a generative design model.

A computer system, comprising:

• at least one processor;
• memory associated with the at least one processor;
• and
• a program stored in the memory for predicting customer sentiment for a product and aspects thereof, the program containing a plurality of instructions which, when executed by the at least one processor, cause the at least one processor to:
• receiving customer data for a plurality of products, and generating a vector of customer sentiments associated with different aspects of each of the plurality of products based on the customer data;
• receiving images for each of the plurality of products, and generating a latent vector for the images for each product by fine-tuning a pre-trained image processing model;
• receiving a textual description for each of the plurality of products, and generating a latent vector for the textual description for each product by fine-tuning a pre-trained natural language processing model;
• designing a deep multimodal design evaluation (DMDE) model with a self-attention fusion mechanism to integrate the latent vectors of the images and the textual descriptions for each product to predict customer sentiment for new product designs based on their images and textual descriptions;
• and
• providing a new product design to the trained DMDE model to obtain predicted customer sentiments for one or more attributes of the new product design or generating a new product design having one or more attributes having favorable predicted customer sentiments using the trained DMDE model and a generative design model.