Back to results
US 11,688,062 B2Granted
Medical device for transcription of appearances in an image to text with machine learning
AssigneeKepler Vision Technologies B.V.
InventorsMarc Jean Baptist Van Oldenborgh, Henricus Meinardus Gerardus Stokman, Ran Tao
Life of this patent
Estimating the term…
Abstract
There is provided a device configured to transcribe an appearance of a human being, said device comprising a common housing holding an image capturing sensor, a computing device comprising a data processor, and a computer program product comprising a first machine learning model trained for detecting and labeling human beings, a second machine learning model trained for detecting appearances of human beings and a transcription module to transcribe the detected appearances of human beings to text.
Claims12
A medical device configured to transcribe an appearance of a human being, said device comprising:
- a common housing holding:
- an image capturing sensor;
- a computing device comprising a data processor and a computer program product comprising:
- a first machine learning model trained for detecting and labeling human beings in at least one image;
- a second machine learning model trained for detecting appearances of human beings in at least one image;
- a transcription module configured to transcribe the detected appearances of human beings to text, wherein said computer program product when running on said data processor causes said computing device to:
- retrieve at least one image from said image capturing sensor;
- analyze said at least one image, wherein analyzing comprises inputting said at least one image to said first machine learning model said first machine learning model detecting presence of a human being in said at least one image and said first machine learning model labeling the detected human being in said at least one image using a label;
- input at least a part of said at least one image to said second machine learning model, said part of said at least one image comprising the labeled human being, and said second machine learning model providing said appearance of said labeled human being as an output;
- and
- apply said transcription module to transcribe the retrieved appearance of said labeled human being to text and outputs said text, wherein the transcription to text in said transcription module involves creating a medical record and output said text into said medical record;
- and
- said second machine learning model comprising:
- a first deep neural network which captures the skeleton data of said human being in said at least a part of said at least one image, said first deep neural network using said at least a part of said least one image as an input and outputs said skeleton data;
- a second deep neural network which captures a first appearance of said human being, said second deep neural network using said skeleton data from said first deep neural network as an input and outputs said first appearance in first appearance data;
- a third deep neural network which captures a second appearance of said human being in said at least a part of said at least one image, said third deep neural network using said at least a part of said at least one image as an input and outputs said second appearance in second appearance data, and
- a fourth deep neural network which captures a third appearance of said human being using said first and second appearance data as an input and outputs third appearance data, said third appearance data comprising a prediction of probabilities of said appearance.
14 more elements
A computer program product comprising a non-transitory computer readable medium and for running on a computing device of a medical device configured to transcribe an appearance of a human being, wherein said medical device comprises a common housing holding:
- an image capturing sensor;
- said computing device comprising a data processor, wherein said computer program product comprises:
- a first machine learning model trained for detecting and labeling human beings in at least one image;
- a second machine learning model trained for detecting appearances of human beings in at least one image;
- a transcription module to transcribe the detected appearances of human beings to text;
- and said computer program product when running on said data processor, causes said computing device to:
- receive at least one image from said image capturing sensor;
- analyze said at least one image, the analyzing comprises:
- input said at least one image to said first machine learning model;
- retrieve from said first machine learning model a presence of a human being in said at least one image;
- retrieve a label from said first machine learning model of the detected human being in said at least one image;
- input at least a part of said at least one image, said part of said at least one image comprising the labeled human being, to said second machine learning model;
- retrieve said appearance of said labeled human being from said second machine learning model;
- apply said transcription module to transcribe the retrieved appearance of said labeled human being to text, and
- output said text, wherein the transcription to text in said transcription module involves creating a medical record and output said text into said medical record;
- and
- said second machine learning model comprising:
- a first deep neural network which captures the skeleton data of said human being in said at least a part of said at least one image, said first deep neural network using said at least a part of said at least one image as an input and outputs said skeleton data;
- a second deep neural network which captures a first appearance of said human being, said second deep neural network using said skeleton data from said first deep neural network as an input and outputs said first appearance in first appearance data;
- a third deep neural network which captures a second appearance of said human being in said at least a part of said at least one image, said third deep neural network using said at least a part of said at least one image as an input and outputs said second appearance in second appearance data, and
- a fourth deep neural network which captures a third appearance of said human being using said first and second appearance data as an input and outputs third appearance data, said third appearance data comprising a prediction of probabilities of said appearance.
18 more elements
Dependencies are inferred from the claim text.