# US 10,872,354 B2 · System and method for personalized preference optimization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,354 B2 - Document ID: US10872354B2 - Status: Granted - Kind: B2 - Application number: 15/256,543 - Filed: 2016-09-03 - Published: 2017-03-09 - Granted: 2020-12-22 - Earliest claimed: 2015-09-04 - Applicants: Robin S Slomkowski; Richard A Rothschild - Inventors: Robin S Slomkowski; Richard A Rothschild - Web page: https://patentgrep.com/patent/US10872354B2 ## CPC - G06Q 30/0269: {based on user profile or attribute} - G06F 16/436: {using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures} - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G06V 40/174: {Facial expression recognition} - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - G06V 40/15: {Biometric patterns based on physiological signals, e.g. heartbeat, blood flow} - H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP] - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system and method is provided for using biometric data from an individual to determine at least one emotion, mood, physical state, or mental state (“state”) of the individual, which is then used, either alone or together with other data, to provide the individual with certain web-based data. In one embodiment of the present invention, a Web host is in communication with at least one network device, where each network device is operated by an individual and is configured to communicate biometric data of the individual to the Web host. The Web host is then configured to use the biometric data to determine at least one state of the individual. The determined state, either alone or together with other data (e.g., interest data), is then used to provide the individual with certain content (e.g., web-based data) or to perform a particular action. ## Claims ### Claim 1 A method for using at least self-reported and biometric data to determine at least one state of a user and to perform at least one action in response to said determined state, said determined state being one of an emotional state and a physical state of said user, comprising the steps of: receiving by a processor first biometric data of said user, said first biometric data being received from at least one sensor; using at least said first biometric data to infer a first state of said user at a time that said first biometric data is received; receiving by said processor self-reporting data from said user, said self-reporting data being received immediately after said first biometric data is received and allowing said user to explicitly identify their current state, thereby indicating whether said inferred state is correct; storing said first biometric data and said self-reporting data in a memory to associate said first biometric with said self-reporting data; receiving by said processor a request from said user to perform an action, said request being received after said biometric data; using at least said first biometric data, said request, and said self-reporting data to determine a second state of said user; and performing said at least one action based at least on said second state, wherein said at least one action is said requested action if said second state is normal and therefore consistent with said performance of said requested action and said at least one action is a second action if said second state is abnormal and therefore inconsistent with said performance of said requested action, said second action including notifying said user of (i) said second state and (ii) an alternate action that could be performed instead of said requested action, wherein said alternate action is an unrequested action and different from said requested action, said alternate action being performed if said user indicates in the affirmative that said alternate action should be performed, otherwise, in the absence of said affirmative indication, performing said requested action. ### Claim 2 (depends on 1) The method of claim 1, wherein said first biometric data includes at least one change in facial muscle expressions, breathing rates, speech pitch, auditory changes, and body movement. ### Claim 3 (depends on 1) The method of claim 1, wherein said first state comprises one of happiness, sadness, surprise, anger, disgust, fear, euphoria, attraction, love, calmness, amusement, excitement, tiredness, hunger, thirst, well-being, sick, failure, triumph, interest, enthusiasm, animation, reinvigoration, and satisfaction. ### Claim 4 (depends on 1) The method of claim 1, wherein said step of receiving said self-reporting data from said user comprising receiving at least one of audible data from said user via a microphone and tactile data from said user via an input device. ### Claim 5 (depends on 1) The method of claim 1, further comprising the step of receiving by said processor ambient data comprising at least one of location of said user, temperature at said location, altitude at said location, and time of day, wherein said processor further uses said ambient data to determine said second state of said user at said time that said second biometric data is received. ### Claim 6 (depends on 1) The method of claim 1, wherein said step of using said first biometric data to infer said first state further comprises comparing said first biometric data to known correlations between different biometric data and different states to infer said first state. ### Claim 7 (depends on 1) The method of claim 1, wherein said second state is further determined from other biometric data and other self-reporting data, where said individual ones of said other self-reporting data are received immediately after individual ones of said other biometric data. ### Claim 8 (depends on 1) The method of claim 1, further comprising a step of performing both said requested action and said alternate action. ### Claim 9 (depends on 1) The method of claim 1, further comprising the step of prompting said user to explicitly identify their current state. ### Claim 10 (depends on 9) The method of claim 9, wherein said step of prompting said user to explicitly identify their current state further includes providing said user with said first inferred state. ### Claim 11 (depends on 1) The method of claim 1, wherein said step of using at least said first biometric data and said self-reporting data to determine said second state comprises checking whether said explicitly identified state matches said first inferred state. ### Claim 12 (depends on 1) The method of claim 1, wherein second biometric data is extracted from said request and used to along with said first biometric data and said self-reporting data to determine said second state. ### Claim 13 (depends on 1) The method of claim 1, further comprising the step of providing said first state to said user prior to receiving said self-reporting data. ### Claim 14 (depends on 13) The method of claim 13, wherein said self-reporting data comprises at least one of confirming, denying, and correcting said first state. ### Claim 15 (depends on 1) The method of claim 1, further comprising the step of determining an intensity level for said second state, said intensity level being a numerical value corresponding to a strength of said second state, said action being further based on at least said intensity level. ### Claim 16 A system for determining at least one state of a user, said at least one state being one of an emotional state and a physical state of said user, comprising: at least one processor; and at least one memory device in communication with said processor and for storing machine readable instructions, said machine readable instructions being adapted to perform the step of: receiving first biometric data of said user, said first biometric data being received from at least one sensor; using at least said first biometric data to infer a first state of said user at a time that said first biometric data is received; receiving self-reporting data from said user, said self-reporting data being received immediately after said first biometric data is received and allowing said user to explicitly identify their current state, said state being said first state or a third state; storing said first biometric data and said self-reporting data in said at least one memory device so that said first biometric data is linked to said self-reporting data; receiving a request from said user to perform an action, said request being received after said self-reporting data has been received; using at least said first biometric data, said request, and said self-reporting data to determine a second state of said user; and performing at least one action, said action being based on at least said second state, wherein said action is said requested action if said second state is normal and therefore consistent with said performance of said requested action and said at least one action is a second action if said second state is abnormal and therefore inconsistent with said performance of said requested action, said second action including then notifying said user of (i) said second state and (ii) an alternate action that could be performed instead of said requested action, wherein said alternate action is an unrequested action and different from said requested action, said alternate action being performed if said user indicates in the affirmative that said alternate action should be performed, otherwise, in the absence of said affirmative indication, performing said requested action. ### Claim 17 (depends on 16) The system of claim 16, wherein said first biometric data includes at least one change in facial muscle expressions, breathing rates, speech pitch, auditory changes, and body movement. ### Claim 18 (depends on 16) The system of claim 16, further comprises at least one of a microphone and a keyboard for receiving said self-reporting data from said user. ### Claim 19 (depends on 16) The system of claim 16, wherein said second state is further determined from other biometric data and other self-reporting data from at least one other user. ### Claim 20 (depends on 16) The system of claim 16, wherein said sensor is a microphone. ### Claim 21 (depends on 16) The system of claim 16, further comprising an Internet of Things (IOT), said IOT comprising said at least one processor and said at least one memory device. ### Claim 22 (depends on 16) The system of claim 16, further comprising prompting said user to identify their current state. ### Claim 23 (depends on 16) The system of claim 16, wherein said machine readable instructions are further adapted to perform said requested action in addition to said alternate action. ### Claim 24 (depends on 23) The system of claim 23, wherein said digital data is further selected from said plurality of digital data based on at least one thing in a proximity of said user at said time that said request is received said requested action is performed before said alternate action. ### Claim 25 (depends on 16) The method of claim 16, wherein said machine readable instructions are further adapted to provide said first state to said user, said self-reporting data comprising at least one of confirming, denying, and correcting said first state. ### Claim 26 (depends on 16) The method of claim 16, wherein said machine readable instructions are further adapted to determine a strength of said second state, said action being further based on at least said strength of said second state. ### Claim 27 A system that uses artificial intelligence (AI) to determine a state and to use said state to perform at least one action, comprising: at least one server in communication with a wide area network (WAN); a mobile device in communication with said at least one server via said WAN, said mobile device comprising: at least one processor for downloading machine readable instructions from said at least one server; and at least one memory device for storing said machine readable instructions that are adapted to perform the step of: receiving first biometric data from said user, said first biometric data being acquire using at least one sensor and analyzed to infer a first state at a time that said first biometric data is received; receiving self-reporting data from said user, said first self-reporting data being received immediately after said first biometric data is received and allowing said user to provide the current state, thereby indicating whether said inferred state is correct; receiving a request from said user to perform an action, said request being received after said self-reporting data has been received; and wherein at least said first biometric data, said request, and said self-reporting data are used to determine a second state at a time that said request is received; wherein at least one action is performed after said second state is determined and in response to said request, said at least one action (i) being based on said second state, (ii) being said requested action if said second state is normal and therefore consistent with said performance of said requested action, and (iii) being a second action if said second state is abnormal and therefore inconsistent with said performance of said requested action, said second action including then notifying said user of (i) said second state and (ii) an alternate action that could be performed instead of said requested action, wherein said alternate action is an unrequested action and different from said requested action, said alternate action being performed if said user indicates in the affirmative that said alternate action should be performed, otherwise, in the absence of said affirmative indication, performing said requested action. ### Claim 28 (depends on 27) The system of claim 27, wherein using at least said first biometric data, said result, and said self-reporting data to determine said second state further comprises (i) using said self-reporting data to check whether said first inferred state should be adjusted, and (ii) using at least the same and said result to determine said second state.