# US 12,118,602 B2 · Recommendation system, method and computer program product based on a user's physical features > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,602 B2 - Document ID: US12118602B2 - Status: Granted - Kind: B2 - Application number: 17/471,490 - Filed: 2021-09-10 - Published: 2021-12-30 - Granted: 2024-10-15 - Earliest claimed: 2017-11-29 - Assignees: Ditto Technologies, Inc. - Applicants: Ditto Technologies, Inc. - Inventors: Dmitry Kornilov - Web page: https://patentgrep.com/patent/US12118602B2 ## CPC - G06Q 30/0631: {Recommending goods or services} - G06Q 30/0282: Rating or review of business operators or products - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06V 40/165: {using facial parts and geometric relationships} - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06V 40/172: {Classification, e.g. identification} - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Embodiments of the present disclosure provide a recommendation system based on a user's physical/biometric features. In various embodiments, a system includes a processor configured to determine a physical characteristic of a user based at least in part on an image of the user. The processor is further configured to determine a correlation between the physical characteristic and a product, and generate a product recommendation based at least in part on the determined correlation. ## Claims ### Claim 1 A system, comprising: a processor; and a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to: generate a three-dimensional (3D) model of a head of a user based on an image of the user, including generating a set of parameter values to a statistical model that corresponds to the image of the user by evaluating candidate parameter values using a cost function determined based at least in part on optical flow; extract a ratio between a plurality of facial feature distances to determine a facial proportion of the user; determine a physical characteristic of the user based at least in part on the image of the user, wherein the physical characteristic is determined at least in part by extracting a parameter from the 3D model, wherein the physical characteristic comprises the facial proportion; determine a correlation between the physical characteristic and a product, wherein the correlation is performed using a 3D model of a user's face and a 3D model of the product; and generate a product recommendation based at least in part on the determined correlation, the product recommendation including a recommendation explanation. ### Claim 2 (depends on 1) The system of claim 1, wherein the determination of the physical characteristic includes a fit of the product to the user. ### Claim 3 (depends on 1) The system of claim 1, wherein the physical characteristic includes a face shape, the shape being defined with respect to a degree to which the shape falls into a plurality of categories. ### Claim 4 (depends on 1) The system of claim 1, wherein the physical characteristic includes a plurality of facial proportions. ### Claim 5 (depends on 1) The system of claim 1, wherein the physical characteristic includes at least one of eye color, skin color, and hair color, the color being normalized based at least in part on a lighting environment in which the at least one image is captured. ### Claim 6 (depends on 1) The system of claim 1, wherein the determination of the correlation is based on a weighted combination of at least one of: the physical characteristic, user browsing data, and user profile data. ### Claim 7 (depends on 6) The system of claim 6, wherein weights assigned to the at least one of: the physical characteristic, user browsing data, and user profile data is based on an output from a machine learning model. ### Claim 8 (depends on 6) The system of claim 6, wherein the physical characteristic is more heavily weighted than at least one of: user browsing data and user profile data. ### Claim 9 (depends on 6) The system of claim 6, wherein weights assigned to the physical characteristic, the user browsing data, and the user profile data are adjustable. ### Claim 10 (depends on 6) The system of claim 6, wherein the processor is further configured to adjust weights used for the weighted combination in response to user feedback on the product recommendation. ### Claim 11 (depends on 1) The system of claim 1, wherein the statistical model is derived from Principal Component Analysis based at least in part on 3D scans of the user's face, and the statistical model includes shape components and texture components. ### Claim 12 (depends on 1) The system of claim 1, wherein the 3D model is a mesh model including polygons defining at least one of: shape of facial features and head shape. ### Claim 13 (depends on 1) The system of claim 1, wherein the determination of a physical characteristic includes determining a skin color based on a color of a polygon of the 3D model. ### Claim 14 (depends on 1) The system of claim 1, wherein the generation of a product recommendation is based on attributes of a product and the attributes of the product are determined based at least in part on a 3D model of the product. ### Claim 15 (depends on 1) The system of claim 1, wherein the generation of a product recommendation is based on fit of the 3D model of the product to a 3D model of the user. ### Claim 16 (depends on 1) The system of claim 1, wherein: the product recommendation includes a group of products; and the processor is further configured to update the product recommendation responsive to user selection of at least one product in the group. ### Claim 17 (depends on 1) The system of claim 1, wherein the product recommendation includes an explanation for the product recommendation. ### Claim 18 A method comprising: generating, using a processor, a three-dimensional (3D) model of a head of a user based on an image of the user, including generating a set of parameter values to a statistical model that corresponds to the image of the user by evaluating candidate parameter values using a cost function determined based at least in part on optical flow; extracting, using the processor, a ratio between a plurality of facial feature distances to determine a facial proportion of the user; extracting, using the processor, a physical characteristic of the user based at least in part on the image of the user, wherein the physical characteristic is determined at least in part by extracting a parameter from the 3D model, wherein the physical characteristic comprises the facial proportion; determining, using the processor, a correlation between the physical characteristic and a product, wherein the correlation is performed using a 3D model of a user's face and a 3D model of the product; and generating, using the processor, a product recommendation based at least in part on the determined correlation, the product recommendation including a recommendation explanation. ### Claim 19 A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for: generating a three-dimensional (3D) model of a head of a user based on an image of the user, including generating a set of parameter values to a statistical model that corresponds to the image of the user by evaluating candidate parameter values using a cost function determined based at least in part on optical flow; extracting a ratio between a plurality of facial feature distances to determine a facial proportion of the user; extracting a physical characteristic of the user based at least in part on the image of the user, wherein the physical characteristic is determined at least in part by extracting a parameter from the 3D model, wherein the physical characteristic comprises the facial proportion; determining a correlation between the physical characteristic and a product, comprising constructing and training a data model to determine a correlation between a user's face and the product or a purchase, wherein the correlation is performed using a 3D model of the user's face and a 3D model of the product; and generating a product recommendation based at least m part on the determined correlation.