# US 2019/0318484 A1 · Apparatuses and methods for navigation in and local segmentation extension of anatomical treelike structures > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318484 A1 - Document ID: US20190318484A1 - Status: Published application - Kind: A1 - Application number: 15/953,150 - Filed: 2018-04-13 - Published: 2019-10-17 - Earliest claimed: 2018-04-12 - Assignees: Veran Medical Technologies, Inc. - Applicants: Veran Medical Technologies, Inc. - Inventors: Andrew DOUGHERTY; Mark HUNTER - Web page: https://patentgrep.com/patent/US20190318484A1 ## CPC - G06T 7/187: involving region growing; involving region merging; involving connected component labelling - G06T 7/0012: {Biomedical image inspection} - G06T 7/11: Region-based segmentation - G06T 7/155: involving morphological operators - G06T 7/162: involving graph-based methods - G06T 7/215: Motion-based segmentation - G06T 7/62: of area, perimeter, diameter or volume - G06T 7/64: of convexity or concavity - G06T 7/73: using feature-based methods - G06T 15/08: Volume rendering - G06T 17/005: {Tree description, e.g. octree, quadtree} - G06T 2200/04: involving 3D image data - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/10132: Ultrasound image - G06T 2207/20104: Interactive definition of region of interest [ROI] - G06T 2207/20156: Automatic seed setting - G06T 2207/30061: Lung - G06T 2207/30064: Lung nodule - G06T 2207/30096: Tumor; Lesion - G06T 2207/30101: Blood vessel; Artery; Vein; Vascular - G06T 2207/30172: Centreline of tubular or elongated structure - G06T 2207/30204: Marker - G06T 2207/30241: Trajectory - G06T 2210/41: Medical - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method of extending a segmentation of an image using navigated image data from a navigation system includes tracking, with the navigation system, at least one of a traveled path and a position of an imaging device relative to an initial segmentation of 3D image data including an initial treelike structure. Navigated image data including image data including at least one 2D or 3D image is captured with the imaging device. A point from the navigated image data corresponding to a potential airway structure is obtained by the navigation system. The initial segmentation of 3D image data is extended by the navigation system using the point obtained from the navigated image data. ## Claims ### Claim 1 A method of extending a segmentation of an image using navigated image data from a navigation system, the method comprising: tracking, with the navigation system, at least one of a traveled path and a position of an imaging device relative to an initial segmentation of 3D image data including an initial treelike structure; capturing, with the imaging device, navigated image data comprising image data including at least one 2D or 3D image; obtaining, by the navigation system, a point from the navigated image data corresponding to a potential airway structure; and extending, by the navigation system, the initial segmentation of 3D image data using the point obtained from the navigated image data. ### Claim 2 (depends on 1) The method of claim 1, further comprising registering the navigated image data to the 3D image data. ### Claim 3 (depends on 1) The method of claim 1, further comprising constructing a 3D volume from the navigated image data. ### Claim 4 (depends on 3) The method of claim 3, wherein obtaining a point comprises automatically segmenting the constructed 3D volume to obtain a point from the navigated image data. ### Claim 5 (depends on 1) The method of claim 1, wherein obtaining a point comprises selecting the point using an I/O component of the navigation system. ### Claim 6 (depends on 1) The method of claim 1, wherein obtaining a point comprises obtaining a point cloud. ### Claim 7 (depends on 1) The method of claim 1, wherein extending the initial segmentation comprises: extracting a region of interest from the 3D image data to create a sub-image; extracting connected components based on spillage-constrained region growing using seed points from the sub-image; pruning the extracted components to discard components not likely to be connected to the initial treelike structure and keep only candidate components likely to be a valid sub-tree of the initial treelike structure; and connecting the candidate components to the initial treelike structure. ### Claim 8 (depends on 7) The method of claim 7, wherein extending the initial segmentation using the point obtained from the navigated image data comprises determining the region of interest in the 3D image data based on the point obtained from the navigated image data. ### Claim 9 (depends on 7) The method of claim 7, wherein extending the initial segmentation using the point obtained from the navigated image data further comprises performing the spillage-constrained region growing using the point obtained from the navigated image data as a seed point. ### Claim 10 (depends on 7) The method of claim 7, wherein pruning comprises: discarding components based on whether they have genus zero and are relatively convex, including determining whether a ratio of a volume of the extracted component before morphological closing to a volume of the extracted component after morphological closing is greater than a threshold; discarding or modifying components based on the components curving more than a threshold amount, including determining whether a dot product of a direction of the extracted component and a direction of a segment of the extracted component is below a threshold; discarding components based on a shape analysis of the shape of the extracted component, including comparing shape analysis metrics including at least one of a minor axis length, an elongation, and an eccentricity of the extracted component to target metrics; and discarding components based on a cylindrical score of the components, including determining a circle equivalent diameter of the extracted component. ### Claim 11 (depends on 10) The method of claim 10, wherein extending the initial segmentation using the point obtained from the navigated image data comprises relaxing the thresholds and analysis metrics in the pruning step based on the point obtained from the navigated image data. ### Claim 12 (depends on 7) The method of claim 7, wherein connecting comprises: determining whether the candidate component has boundary voxels that are 4-connected to the initial segmentation and if so filling in voxels to fully connect the candidate component to the initial segmentation; determining whether the radius of a candidate component is compatible with a terminal segment of the initial segmentation, and filling in voxels to connect the candidate component to the terminal segment if they are compatible; determining if the candidate component includes a path of seed points to the initial segmentation, and if so identifying potential new growth based on a gradient ball method. ### Claim 13 (depends on 12) The method of claim 12, wherein extending the initial segmentation using the point obtained from the navigated image data comprises relaxing the determinations in the connecting step based on the point obtained from the navigated image data. ### Claim 14 (depends on 1) The method of claim 1, wherein capturing comprises capturing 2D ultrasound image data with the imaging device. ### Claim 15 A method of extending a segmentation of an image using navigation data from a navigation system, the method comprising: tracking, with the navigation system, at least one of a traveled path, a position, and a trajectory of a navigated instrument relative to an initial segmentation of 3D image data including an initial treelike structure to create navigation data; and extending the initial segmentation of the 3D image data using the navigation data, wherein extending the initial segmentation comprises: extracting a region of interest from the 3D image data to create a sub-image; extracting connected components based on spillage-constrained region growing using seed points from the sub-image; pruning the extracted components to discard components not likely to be connected to the initial treelike structure and keep only candidate components likely to be a valid sub-tree of the initial treelike structure; and connecting the candidate components to the initial treelike structure. ### Claim 16 (depends on 15) The method of claim 15, wherein extending the initial segmentation further comprises determining a region of interest based on the navigation data. ### Claim 17 (depends on 15) The method of claim 15, wherein extending the initial segmentation further comprises performing the spillage-constrained region growing using the navigation data of the navigated instrument as seed points. ### Claim 18 (depends on 15) The method of claim 15, wherein pruning comprises: discarding components based on whether they have genus zero and are relatively convex, including determining whether a ratio of a volume of the extracted component before morphological closing to a volume of the extracted component after morphological closing is greater than a threshold; discarding or modifying components based on the components curving more than a threshold amount, including determining whether a dot product of a direction of the extracted component and a direction of a segment of the extracted component is below a threshold; discarding components based on a shape analysis of the shape of the extracted component, including comparing shape analysis metrics including at least one of a minor axis length, an elongation, and an eccentricity of the extracted component to target metrics; and discarding components based on a cylindrical score of the components, including determining a circle equivalent diameter of the extracted component. ### Claim 19 (depends on 18) The method of claim 18, wherein extending the initial segmentation comprises relaxing the thresholds and analysis metrics in the pruning step based on the navigation data. ### Claim 20 (depends on 15) The method of claim 15, wherein connecting comprises: determining whether the candidate component has boundary voxels that are 4-connected to the initial segmentation and if so filling in voxels to fully connect the candidate component to the initial segmentation; determining whether the radius of a candidate component is compatible with a terminal segment of the initial segmentation, and filling in voxels to connect the candidate component to the terminal segment if they are compatible; determining if the candidate component includes a path of seed points to the initial segmentation, and if so identifying potential new growth based on a gradient ball method. ### Claim 21 (depends on 20) The method of claim 20, wherein extending the initial segmentation comprises relaxing the determinations in the connecting step based on the navigation data.