# US 12,182,928 B2 · System and methods for generating a 3D model of a pathology sample > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,928 B2 - Document ID: US12182928B2 - Status: Granted - Kind: B2 - Application number: 17/761,576 - Filed: 2020-09-16 - Published: 2022-12-01 - Granted: 2024-12-31 - Earliest claimed: 2019-09-18 - Assignees: Inveox GmbH - Applicants: Inveox GmbH - Inventors: Maria Sievert; Dominik Sievert - Web page: https://patentgrep.com/patent/US12182928B2 ## CPC - G01B 11/24: for measuring contours or curvatures - G06T 15/08: Volume rendering - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06V 10/457: {by analysing connectivity, e.g. edge linking, connected component analysis or slices} - G01B 2210/54: Revolving an optical measuring instrument around a body - G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A system and a method for generating a combined 3D model (95) of a sample comprising a sample imaging system (1) configured to generate a first 3D model (25) of the sample, a slice imaging system (2) configured to generate a second 3D model (615) of the sample, and a combiner engine (90) configured to generate a combined 3D model (95) based on the first 3D model and the second 3D model of the sample. ## Claims ### Claim 1 A system comprising: a sample imaging system configured to: acquire, prior to slicing the sample, a plurality of first sensor data sets related to an outer surface of the sample and generate a first 3D model based on the plurality of first sensor data sets; a slice imaging system configured to: acquire, after slicing the sample into slices, a plurality of second sensor data sets related to the slices of the sample and generate a second 3D model based on the plurality second data sets; and a combiner engine configured to generate a combined 3D model of the sample based on the first 3D model of the sample and the second 3D model of the sample, wherein the combiner engine is configured to calculate a matching score configured to indicate a fitness level of the at least one cross-sectional view on the first 3D model, wherein the matching score is calculated based on a fitness between edges of the at least one cross-sectional view and edges of the first 3D model and wherein the combiner engine is configured to fit the at least one cross-sectional view to the first model such that said matching score can be maximized. ### Claim 2 (depends on 1) The system of claim 1, wherein the first 3D model comprises a boundary model of the sample representing the outer surface of the sample and the second 3D model comprises a solid model of the sample comprising at least one cross sectional view of the sample representing the internal structure of the sample. ### Claim 3 (depends on 2) The system according to claim 2, wherein the combiner engine is configured to generate the combined 3D model by fitting the at least one cross-sectional view of the sample obtained from the second 3D model to the first 3D model. ### Claim 4 (depends on 1) The system according to claim 1, wherein the combiner engine is configured to generate the combined 3D model by executing an image registration algorithm during which the plurality of second sensor data sets of the slices of the sample are registered using the first 3D model as a reference, wherein registering the plurality of second sensor data sets comprises bringing them into spatial alignment. ### Claim 5 (depends on 1) The system according to claim 1, wherein the combiner engine is configured to generate the combined 3D model based on at least one sample change parameter, and wherein the sample change parameter indicates a change of the physical and/or chemical structure of the sample, such as, a change caused to the sample during histological processing, and wherein the sample change parameter can comprise a shrinking parameter and/or a color change parameter and/or a shape change parameter. ### Claim 6 (depends on 1) The system of claim 1, wherein the sample imaging system comprises: at least one sensor device configured to acquire the plurality of first sensor data sets and a data processing device configured to process the plurality of first sensor data sets to generate the first 3D model of the sample. ### Claim 7 (depends on 6) The system of claim 6, wherein the data processing device is configured to extract a respective shape of at least one face of the sample from a respective first sensor data set of the plurality of first sensor data sets. ### Claim 8 (depends on 6) The system of claim 6, wherein the sample imaging system comprises a sample positioning device configured to move the sample, such that multiple sides of the sample are positioned within the field of view of the at least one sensor device and/or a sensor positioning device configured to handle or transport or move or rotate the at least one sensor device, such that the at least one sensor device can be positioned in multiple poses relative to the at least one sample and/or a plurality of sensor devices with different viewing angles toward the sample. ### Claim 9 (depends on 6) The system of claim 6, wherein the at least one sensor device comprises at least one visual sensor configured to capture visual images of the sample and/or at least one depth sensor configured to capture distance images of the sample. ### Claim 10 (depends on 1) The system of claim 1, wherein the slice imaging system comprises: a slice imaging device configured to acquire the plurality of second sensor data sets and a three-dimensional rendering engine configured to process the plurality of second sensor data sets to generate a second 3D model of the sample. ### Claim 11 (depends on 10) The system of claim 10, wherein the slice imaging system is configured to receive the plurality of slices of the sample, wherein a slice of the sample is a cross section of the sample and wherein the slice of the sample is generated by cutting the sample with a sectioning device configured to cut thin slices of a material, such as, a microtome. ### Claim 12 (depends on 10) The system of claim 10, wherein the slice imaging device comprises at least one visual sensor configured to capture images of the slices of the sample. ### Claim 13 (depends on 10) The system of claim 10, wherein the three-dimensional rendering engine is configured to extract a respective shape of at least one slice of the sample from a respective second sensor data set of the plurality of second sensor data sets and generate the second 3D model based on the respective shape of the at least one slice of the sample. ### Claim 14 (depends on 10) The system of claim 10, wherein the 3D rendering engine is configured to generate the second 3D model based on additional information indicating features of the slices of the sample, wherein the additional information comprises at least one of the thickness of the slices; the position of the slices on the sample, such as, the order of the slices; and/or the orientation of the slices relative to each other or any combination thereof. ### Claim 15 (depends on 1) The system according to claim 1, wherein the sample is at least one of a histological, pathological, forensic pathology, medical, biological, veterinary, agricultural tissue and/or biopsy sample. ### Claim 16 A method for generating a combined 3D model of a sample, the method comprising the steps of: acquiring, prior to slicing the sample, a plurality of first sensor data sets related to an outer surface of the sample via at least one sensor device; processing with a data processing device the plurality of first sensor data sets to extract general structure data related to the sample; acquiring, after slicing the sample into slices, a plurality of second sensor data sets related to the slices of the sample via a slice imaging device; processing with a 3D rendering engine the plurality of second sensor data sets to extract cross-sectional data related to the sample; and combining with a combiner engine the general structure data and the cross-sectional data to generate a combined 3D model of the sample; wherein the combiner engine calculates a matching score configured to indicate a fitness level of the at least one cross-sectional view on the first 3D model, wherein the matching score is calculated based on a fitness between edges of the at least one cross-sectional view and edges of the first 3D model and wherein the combiner engine fits the at least one cross-sectional view to the first model such that said matching score can be maximized. ### Claim 17 (depends on 16) The method of claim 16, wherein the general structure data comprises a boundary model of the sample representing the surface of the sample and the cross-sectional data comprises at least one cross sectional view of the sample. ### Claim 18 (depends on 16) The method of claim 16, wherein combining with a combiner engine the general structure data and the cross-sectional data comprises the combiner engine executing an image registration algorithm during which the plurality of second sensor data sets of the slices of the sample are registered using the general structure data as a reference, wherein registering the plurality of second sensor data sets comprises bringing them into spatial alignment.