# US 12,119,117 B2 · Method and system for disease quantification of anatomical structures > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,119,117 B2 - Document ID: US12119117B2 - Status: Granted - Kind: B2 - Application number: 17/726,307 - Filed: 2022-04-21 - Published: 2022-11-03 - Granted: 2024-10-15 - Earliest claimed: 2021-04-23 - Assignees: Shenzhen Keya Medical Technology Corporation - Applicants: Shenzhen Keya Medical Technology Corporation - Inventors: Xin Wang; Youbing Yin; Bin Kong; Yi Lu; Hao-Yu Yang; Xinyu Guo; Qi Song - Web page: https://patentgrep.com/patent/US12119117B2 ## CPC - G16H 50/30: for calculating health indices; for individual health risk assessment - G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06N 3/09: Supervised learning - G06T 7/0012: {Biomedical image inspection} - G06T 7/0014: {using an image reference approach} - G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215) - G06V 10/42: Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation - G06V 10/426: Graphical representations - G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components - G06V 10/82: using neural networks - G16H 30/40: for processing medical images, e.g. editing - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G06T 2207/10081: Computed x-ray tomography [CT] - G06T 2207/10088: Magnetic resonance imaging [MRI] - G06T 2207/10132: Ultrasound image - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30104: Vascular flow; Blood flow; Perfusion - G06T 2207/30172: Centreline of tubular or elongated structure - G06V 2201/03: Recognition of patterns in medical or anatomical images - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract This disclosure discloses a method and system for predicting disease quantification parameters for an anatomical structure. The method includes extracting a centerline structure based on a medical image. The method further includes predicting the disease quantification parameter for each sampling point on the extracted centerline structure by using a GNN, with each node corresponds to a sampling point on the extracted centerline structure and each edge corresponds to a spatial constraint relationship between the sampling points. For each node, a local feature is extracted based on the image patch for the corresponding sampling point by using a local feature encoder, and a global feature is extracted by using a global feature encoder based on a set of image patches for a set of sampling points, which include the corresponding sampling point and have a spatial constraint relationship defined by the centerline structure. Then, an embed feature is obtained based on both the local feature and the global feature and input into to the node. The method is able to integrate local and global consideration factors of the sampling points into the GNN to improve the prediction accuracy. ## Claims ### Claim 1 A computer-implemented method of predicting disease quantification parameters for an anatomical structure, comprising: receiving a medical image containing the anatomical structure; extracting, by at least one processor, a centerline structure based on the medical image; and predicting the disease quantification parameter for each sampling point the extracted centerline structure by using a graph neural network (GNN), wherein each node of the GNN corresponds to a sampling point on the extracted centerline structure and each edge of the GNN corresponds to a spatial constraint relationship between two sampling points, wherein predicting the disease quantification parameter for each sampling point further comprises: extracting a local feature based on an image patch for the sampling point by using a local feature encoder; extracting a global feature by using a global feature encoder based on a set of image patches for a set of sampling points, which include the sampling point and have a spatial constraint relationship defined by the centerline structure; and obtaining an embed feature based on both the local feature and the global feature and inputting the embed feature to the node of the GNN corresponding to the sampling point. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein each of the local feature encoder and the global feature encoder adopts a dual-level attention mechanism comprising a first level attention module and a second level attention module, wherein the first level attention module is configured to extract a diseased location-aware attention map and the second level attention module is configured to extract a pixel-to-pixel inter-dependence attention map. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the GNN includes a high-order GNN. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the anatomical structure includes an anatomical tree structure or an anatomical graph structure. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the local feature encoder includes a first CNN combined with an attention mechanism for localization, followed by a first self-attention block combined with an attention mechanism for pixel-to-pixel correlation. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the global feature encoder includes a second CNN combined with an attention mechanism for localization, followed by a RNN, which is then followed by a second self-attention block combined with an attention mechanism for pixel-to-pixel correlation, wherein the RNN is configured to accumulate information from a root of the anatomical structure to the current sampling point. ### Claim 7 (depends on 5) The computer-implemented method of claim 5, wherein the attention mechanism for localization includes a residual attention, and the first self-attention block includes a nonlocal network. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein obtaining an embed feature based on both the local feature and the global feature further comprises concatenating the local feature and the global feature to obtain the embed feature. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the GNN includes a graph convolution based neural network or a graph gated based neural network. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the anatomical structure includes a vessel structure or an airway structure. ### Claim 11 A system for predicting disease quantification parameters for an anatomical structure, which comprises: an interface configured to receive a medical image containing the anatomical structure; and a processor configured to: extract a centerline structure based on the medical image; and predict the disease quantification parameter for each sampling point on the extracted centerline structure by using a graph neural network (GNN), wherein each node of the GNN corresponds to a sampling point on the extracted centerline structure and each edge of the GNN corresponds to a spatial constraint relationship between two sampling points, wherein to predict the disease quantification parameter for each sampling point, the processor is further configured to: extract a local feature based on an image patch for the sampling point by using a local feature encoder; extract a global feature by using a global feature encoder based on a set of image patches for a set of sampling points, which include the sampling point and have a spatial constraint relationship defined by the centerline structure; and obtain an embed feature based on both the local feature and the global feature and inputting the embed feature to the node of the GNN corresponding to the sampling point. ### Claim 12 (depends on 11) The system of claim 11, wherein each of the local feature encoder and the global feature encoder adopts a dual-level attention mechanism comprising a first level attention module and a second level attention module, wherein the first level attention module is configured to extract diseased location-aware attention map and the second level attention module is configured to extract pixel-to-pixel inter-dependence attention map. ### Claim 13 (depends on 11) The system of claim 11, wherein the GNN includes a high-order GNN. ### Claim 14 (depends on 11) The system of claim 11, wherein the anatomical structure includes an anatomical tree structure or an anatomical graph structure. ### Claim 15 (depends on 11) The system of claim 11, wherein the local feature encoder includes a first CNN combined with an attention mechanism for localization, followed by a first self-attention block combined with an attention mechanism for pixel-to-pixel correlation. ### Claim 16 (depends on 11) The system of claim 11, wherein the global feature encoder includes a second CNN combined with an attention mechanism for localization, followed by a RNN, which is then followed by a second self-attention block combined with an attention mechanism for pixel-to-pixel correlation, wherein the RNN is configured to accumulate the information from a root of the anatomical structure to the current sampling point. ### Claim 17 (depends on 11) The system of claim 11, wherein to obtain an embed feature based on both the local feature and the global feature, the processor is further configured to concatenate the local feature and the global feature to obtain the embed feature. ### Claim 18 (depends on 11) The system of claim 11, wherein the GNN includes a graph convolution based neural network or a graph gated based neural network. ### Claim 19 (depends on 11) The system of claim 11, wherein the anatomical structure includes a vessel structure or an airway structure. ### Claim 20 A non-transitory computer-readable storage medium, with computer executable instructions stored thereon, which, when being performed by a processor, perform a method of predicting disease quantification parameters for an anatomical structure, the method comprising: receiving a medical image containing the anatomical structure; extracting a centerline structure based on the medical image; and predicting the disease quantification parameter for each sampling point on the extracted centerline structure by using a graph neural network (GNN) where each node of the GNN corresponds to a sampling point on the extracted centerline structure and each edge of the GNN corresponds to a spatial constraint relationship between two sampling points, wherein predicting the disease quantification parameter for each sampling point further comprises: extracting a local feature based on an image patch for the sampling point by using a local feature encoder; extracting a global feature by using a global feature encoder based on a set of image patches for a set of sampling points, which include the sampling point and have a spatial constraint relationship defined by the centerline structure; and obtaining an embed feature based on both the local feature and the global feature and inputting the embed feature to the node of the GNN corresponding to the sampling point.