# US 10,503,970 B1 · Method and system for identifying biometric characteristics using machine learning techniques > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,503,970 B1 - Document ID: US10503970B1 - Status: Granted - Kind: B1 - Application number: 15/837,522 - Filed: 2017-12-11 - Granted: 2019-12-10 - Earliest claimed: 2017-12-11 - Assignees: State Farm Mutual Automobile Insurance Company - Applicants: State Farm Mutual Automobile Insurance Company - Inventors: Dingchao Zhang; Michael Bernico; Peter Laube; Utku Pamuksuz; Jeffrey S. Myers; Marigona Bokshi-Drotar; Edward W. Breitweiser - Web page: https://patentgrep.com/patent/US10503970B1 ## CPC - G06V 10/82: using neural networks - A61B 5/117: Identification of persons (methods or arrangements for recognising patterns, e.g. fingerprints, G06F18/00, G06V40/00; identification of persons by analysing their voice or speech G10L17/00) - A61B 5/1176: {Recognition of faces} - A61B 5/6891: {Furniture} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06V 40/172: {Classification, e.g. identification} - G06V 40/15: {Biometric patterns based on physiological signals, e.g. heartbeat, blood flow} - G06V 40/178: {estimating age from face image; using age information for improving recognition} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method and system may use computer vision techniques and machine learning analysis to automatically identify a user's biometric characteristics. A user's client computing device may capture a video of the user. Feature data and movement data may be extracted from the video and applied to statistical models for determining several biometric characteristics. The determined biometric characteristic values may be used to identify individual health scores and the individual health scores may be combined to generate an overall health score and longevity metric. An indication of the user's biometric characteristics which may include the overall health score and longevity metric may be displayed on the user's client computing device. ## Claims ### Claim 1 A method for identifying biometric characteristics of a user based on audiovisual data, the method executed by one or more processors programmed to perform the method, the method comprising: obtaining, at one or more processors, a plurality of sets of audiovisual data corresponding to a plurality of people and one or more biometric characteristics for each of the plurality of people; for each of the one or more biometric characteristics: analyzing, by the one or more processors, each of the plurality of sets of audiovisual data to identify a plurality of features; analyzing, by the one or more processors, the plurality of features in each set of audiovisual data to identify movement data corresponding to changes in the plurality of features over time; and generating, by the one or more processors a model for determining an unknown biometric characteristic of a user based on the plurality of features, the movement data, and the obtained biometric characteristic for each of the plurality of people; receiving, at the one or more processors, a set of audiovisual data corresponding to a user, wherein the audiovisual data includes a plurality of frames captured over a threshold time period; applying, by the one or more processors, features within the set of audiovisual data corresponding to the user to the one or more models to determine the one or more biometric characteristics of the user; and providing, by the one or more processors, an indication of the determined one or more biometric characteristics of the user to a client computing device. ### Claim 2 (depends on 1) The method of claim 1, further comprising: determining, by the one or more processors, an individual health indicator for each of the one or more determined biometric characteristics of the user; and combining, by the one or more processors, the individual health indicators for the user to generate an overall health indicator, wherein providing the indication of the determined one or more biometric characteristics of the user to the client computing device includes providing the overall health indicator to the client computing device. ### Claim 3 (depends on 2) The method of claim 2, wherein the overall health indicator is used to predict remaining longevity for the user. ### Claim 4 (depends on 1) The method of claim 1, wherein analyzing the plurality of sets of audiovisual data to identify a plurality of features includes: analyzing, by the one or more processors, each frame in the plurality of sets of audiovisual data to identify a portion of each frame depicting a face of the person using face detection techniques; and analyzing, by the one or more processors, the plurality of portions depicting faces to identify a plurality of feature vectors corresponding to the faces. ### Claim 5 (depends on 1) The method of claim 1, wherein the model for determining an unknown biometric characteristic of a user is generated using one or more machine learning techniques. ### Claim 6 (depends on 1) The method of claim 1, wherein the one or more biometric characteristics include at least one of: age, gender, body mass index (BMI), heart rate, body temperature, galvanic skin response (GSR), smoking status, or emotional state. ### Claim 7 A server computing device for identifying biometric characteristics of a user based on audiovisual data, the server computing device comprising: one or more processors; and a non-transitory computer-readable memory coupled to the one or more processors, and storing thereon instructions that, when executed by the one or more processors, cause the server computing device to: obtain a plurality of sets of audiovisual data corresponding to a plurality of people and one or more biometric characteristics for each of the plurality of people; for each of the one or more biometric characteristics: analyze each of the plurality of sets of audiovisual data to identify a plurality of features; analyze the plurality of features in each set of audiovisual data to identify movement data corresponding to changes in the plurality of features over time; and generate a model for determining an unknown biometric characteristic of a user based on the plurality of features, the movement data, and the obtained biometric characteristic for each of the plurality of people; receive a set of audiovisual data corresponding to a user, wherein the audiovisual data includes a plurality of frames captured over a threshold time period; apply features within the set of audiovisual data corresponding to the user to the one or more models to determine the one or more biometric characteristics of the user; and provide an indication of the determined one or more biometric characteristics of the user to a client computing device. ### Claim 8 (depends on 7) The server computing device of claim 7, wherein the instructions further cause the server computing device to: determine an individual health indicator for each of the one or more determined biometric characteristics of the user; and combine the individual health indicators for the user to generate an overall health indicator, wherein providing the indication of the determined one or more biometric characteristics of the user to the client computing device includes providing the overall health indicator to the client computing device. ### Claim 9 (depends on 8) The server computing device of claim 8, wherein the overall health indicator is used to predict remaining longevity for the user. ### Claim 10 (depends on 7) The server computing device of claim 7, wherein to analyze the plurality of sets of audiovisual data to identify a plurality of features, the instructions cause the server computing device to: analyze each frame in the plurality of sets of audiovisual data to identify a portion of each frame depicting a face of the person using face detection techniques; and analyze the plurality of portions depicting faces to identify a plurality of feature vectors corresponding to the faces. ### Claim 11 (depends on 7) The server computing device of claim 7, wherein the model for determining an unknown biometric characteristic of a user is generated using one or more machine learning techniques. ### Claim 12 (depends on 7) The server computing device of claim 7, wherein the one or more biometric characteristics include at least one of: age, gender, body mass index (BMI), heart rate, body temperature, galvanic skin response (GSR), smoking status, or emotional state. ### Claim 13 A non-transitory computer-readable memory storing thereon instructions that, when executed by one or more processors, cause the one or more processors to: obtain a plurality of sets of audiovisual data corresponding to a plurality of people and one or more biometric characteristics for each of the plurality of people; for each of the one or more biometric characteristics: analyze the plurality of sets of audiovisual data to identify a plurality of features; analyze the plurality of features in each set of audiovisual data to identify movement data corresponding to changes in the plurality of features over time; and generate a model for determining an unknown biometric characteristic of a user based on the plurality of features, the movement data, and the obtained biometric characteristic for each of the plurality of people; receive a set of audiovisual data corresponding to a user, wherein the audiovisual data includes a plurality of frames captured over a threshold time period; apply features within the set of audiovisual data corresponding to the user to the one or more models to determine the one or more biometric characteristics of the user; and provide an indication of the determined one or more biometric characteristics of the user to a client computing device. ### Claim 14 (depends on 13) The non-transitory computer-readable memory of claim 13, wherein the instructions further cause the one or more processors to: determine an individual health indicator for each of the one or more determined biometric characteristics of the user; and combine the individual health indicators for the user to generate an overall health indicator, wherein providing the indication of the determined one or more biometric characteristics of the user to the client computing device includes providing the overall health indicator to the client computing device. ### Claim 15 (depends on 13) The non-transitory computer-readable memory of claim 13, wherein to analyze the plurality of sets of audiovisual data to identify a plurality of features, the instructions cause the one or more processors to: analyze each frame in the plurality of sets of audiovisual data to identify a portion of each frame depicting a face of the person using face detection techniques; and analyze the plurality of portions depicting faces to identify a plurality of feature vectors corresponding to the faces. ### Claim 16 (depends on 13) The non-transitory computer-readable memory of claim 13, wherein the model for determining an unknown biometric characteristic of a user is generated using one or more machine learning techniques. ### Claim 17 (depends on 13) The non-transitory computer-readable memory of claim 13, wherein the one or more biometric characteristics include at least one of: age, gender, body mass index (BMI), heart rate, body temperature, galvanic skin response (GSR), smoking status, or emotional state.