Combining natural language understanding and image segmentation to intelligently populate text reports
AssigneeSynthesis Health Inc.
InventorsMurray Aaron Reicher, Jesse Wakley
Life of this patent
Estimating the term…
Abstract
An AI system may be configured to combine a marked area of interest (“AOI”) in a medical image (such as from a user clicking an area of a displayed medical image) with natural language understanding (“NLU”) of a description of the medical image (such as provided by the user via voice input) to determine a more precise region for the AOI. This facilitates more accurate and simplified reporting by using the more precise AOI to determine where to automatically place the description in the report and/or to determine how to best supplement the description with automatically determined anatomical information.
Claims22
A computing system comprising:
- a hardware computer processor;
- a non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the computing system to perform operations comprising:
- applying a segmentation algorithm to a medical image to determine a plurality of segments of patient anatomy associated with the medical image;
- displaying the medical image on a display device of a user;
- receiving input from the user selecting a portion of the medical image;
- determining two or more segments associated with the selected portion of the medical image that overlap at the user-selected portion of the medical image;
- receiving input indicating a description of an area of interest;
- and
- determining the area of interest based on the two or more segments associated with the selected portion of the medical image and a natural language understanding of the description of the area of interest to determine a related segment that is used to select one of the two or more segments as the area of interest.
A computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising:
- applying a segmentation algorithm to a medical image to determine a plurality of segments of patient anatomy associated with the medical image;
- displaying the medical image on a display device of a user;
- receiving input from the user selecting a portion of the medical image;
- determining two or more segments associated with the selected portion of the medical image that overlap at the user-selected portion of the medical image;
- receiving input indicating a description of an area of interest;
- and
- determining the area of interest based on the two or more segments associated with the selected portion of the medical image and a natural language understanding of the description of the area of interest to determine a related segment that is used to select one of the two or more segments as the area of interest.
Dependencies are inferred from the claim text.