# US 11,026,634 B2 · Image-based system and method for predicting physiological parameters

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,026,634 B2
- Document ID: US11026634B2
- Status: Granted
- Kind: B2
- Application number: 15/946,629
- Filed: 2018-04-05
- Published: 2018-10-11
- Granted: 2021-06-08
- Earliest claimed: 2017-04-05
- Applicants: doc.ai incorporated
- Inventors: Walter De Brouwer; Apurv Mishra; Samia De Brouwer
- Web page: https://patentgrep.com/patent/US11026634B2

## CPC

- A61B 5/7275: {Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor}
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- A61B 5/1032: {Determining colour of tissue for diagnostic purposes}
- A61B 5/1072: {measuring distances on the body, e.g. measuring length, height or thickness (A61B5/1076 takes precedence)}
- A61B 5/441: {Skin evaluation, e.g. for skin disorder diagnosis}
- A61B 5/4872: {Body fat}
- A61B 5/7267: {involving training the classification device}
- G06F 18/24133: {Distances to prototypes}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06N 3/096: Transfer learning
- G06N 5/046: Forward inferencing; Production systems
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 40/169: {Holistic features and representations, i.e. based on the facial image taken as a whole}
- G06V 40/172: {Classification, e.g. identification}
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/30: for calculating health indices; for individual health risk assessment
- A61B 5/024: Measuring pulse rate or heart rate
- A61B 5/02405: {Determining heart rate variability}
- A61B 5/163: {by tracking eye movement, gaze, or pupil change}
- A61B 5/442: {Evaluating skin mechanical properties, e.g. elasticity, hardness, texture, wrinkle assessment}
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06V 40/178: {estimating age from face image; using age information for improving recognition}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

System and method for determining physiological parameters of a person are disclosed. A physiological parameter may be obtained by analyzing a facial image of a person, and determining, from the facial image, a physiological parameter of the person by processing the facial image with a data processor. A neural network model such as regression deep learning convolutional neural network is used to predict the physiological parameter. An image processor screens out images which can't be recognized as facial images and adjust facial images to frontal facial images for predicting of physiological parameters.

## Claims

### Claim 1

A system for predicting physiological parameter of a person based on a facial image thereof, comprising:
an image processor, electrically coupled with a network, configured to:
receive from a digital device an image including facial and upper body features of a person;
process the image to generate a frontal facial image;
provide the frontal facial image, comprising the facial and upper body features, to a trained neural network model configured to predict weight and height of the person based on the facial and upper body features;
receive predicted weight and height of the person from the neural network model;
determine body mass index value associated with the person; and
provide for display the determined body mass index value; and wherein
the trained neural network model is a regression deep learning convolutional neural network model.

### Claim 2 (depends on 1)

The system of claim 1, wherein the neural network model has three input parameters and seventeen hidden convolutional layers.

### Claim 3 (depends on 2)

The system of claim 2, wherein the three input parameters are three color channels corresponding to one or more images.

### Claim 4 (depends on 1)

The system of claim 1, wherein the regression deep learning convolutional neural network model is a Network-in-Network neural network model.

### Claim 5 (depends on 1)

The system of claim 1, wherein said frontal facial image is provided via three color channels.

### Claim 6 (depends on 1)

The system of claim 1, further comprising a server, electrically coupled with the network, wherein the trained neural network model is stored on the server.

### Claim 7 (depends on 1)

The system of claim 1, further comprising a digital device configured to capture an image including a facial and upper body features of the person, wherein the digital device is electrically coupled with the network.

### Claim 8 (depends on 1)

The system of claim 1, wherein the image processor is further configured to evaluate the image to determine if the image is a qualified image of the person.

### Claim 9

A method for predicting physiological parameter of a person based on a facial image thereof, comprising:
receiving a request for body mass index value;
acquiring an image with at least facial and upper body features of a person; processing the image to a frontal facial image;
applying the frontal facial image, comprising the facial and upper body features, to a trained neural network model to predict weight and height of the person based on the facial and upper body features;
determining body mass index value based on the predicted weight and height; and
providing, in response to the request, the determined body mass index value; and wherein
said trained neural network model is a regression deep learning convolutional neural network model.

### Claim 10 (depends on 9)

The method of claim 9, wherein the neural network model has three input parameters and seventeen hidden convolutional layers.

### Claim 11 (depends on 9)

The method of claim 9, wherein the regression deep learning convolutional neural network model is a Network-in-Network neural network model.

### Claim 12 (depends on 9)

The method of claim 9, wherein said frontal facial image is applied via three color channels.

### Claim 13

A non-transitory computer readable medium including a set of instructions that are executable by one or more processers of a computer to cause the computer to perform a method for predicting body mass index value based on a facial image of a person, the method comprising:
receiving a request for body mass index value;
acquiring an image with at least facial and upper body features of a person; processing the image to a frontal facial image;
applying the frontal facial image, comprising the facial and upper body features, to a trained neural network model to predict weight and height of the person based on the facial and upper body features;
determining body mass index value based on the predicted weight and height; and
providing, in response to the request, the determined body mass index value; and wherein
said trained neural network model is a regression deep learning convolutional neural network model.

### Claim 14 (depends on 13)

The method of claim 13, wherein the neural network model has three input parameters and seventeen hidden convolutional layers.

### Claim 15 (depends on 13)

The method of claim 13, wherein the regression deep learning convolutional neural network model is a Network-in-Network neural network model.

### Claim 16 (depends on 13)

The method of claim 13, wherein said frontal facial image is applied via three color channels.

### Claim 17

A system for predicting physiological parameter of a person based on a facial image thereof, comprising:
an image processor, electrically coupled with a network, said image processor being configured for:
receiving from a digital device a captured image including upper body features of a person; said upper body features including a face and a neck and shoulders of said person, and said upper body features excluding body features that are lower than said shoulders of said person;
pre-processing said captured image to generate a pre-processed image, said pre-processed image being a frontal facial image, said frontal facial image including said upper body features, and said frontal facial image further excluding body features that are lower than said shoulders of said person;
providing said pre-processed image as input into to a trained neural network model, said trained neural network model being configured to predict weight and height of the person based on said pre-processed image; and
receiving a predicted weight and height of the person from the neural network model.

### Claim 18 (depends on 17)

The system of claim 17, wherein said pre-processed image is input into said trained neural network model via three color channels.

### Claim 19 (depends on 17)

The system of claim 17, wherein a supplementary rectangle is employed to indicate a width of said shoulders of said person.

### Claim 20 (depends on 17)

The system of claim 17, further configured for determining a predicted body mass index value associated with said person.

### Claim 21 (depends on 17)

The system of claim 17, wherein said the trained neural network model is a regression deep learning convolutional neural network model.

### Claim 22 (depends on 21)

The system of claim 21, wherein the regression deep learning convolutional neural network model is a Network-in-Network neural network model.

### Claim 23 (depends on 17)

The system of claim 17, wherein said trained neural network model is trained by images of persons, each of said images including said upper body features of a person and excluding body features that are lower than said shoulders of said person, and each of said images being associated with a corresponding known weight value and a known height value values of each said person shown within each of said images of persons.

### Claim 24 (depends on 17)

The system of claim 17, further comprising a server, electrically coupled with the network, wherein the trained neural network model is stored on the server.

### Claim 25 (depends on 17)

The system of claim 17, further comprising a digital device configured to capture an image including a facial and upper body features of said person, wherein the digital device is electrically coupled with the network.

### Claim 26 (depends on 17)

The system of claim 17, wherein the image processor is further configured to evaluate the image to determine if the image is a qualified image of said person.