# US 2022/0284580 A1 · Image analysis in pathology > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0284580 A1 - Document ID: US20220284580A1 - Status: Published application - Kind: A1 - Application number: 17/752,298 - Filed: 2022-05-24 - Published: 2022-09-08 - Earliest claimed: 2019-09-26 - Assignees: Aiforia Technologies Oyj - Applicants: Aiforia Technologies Oyj - Inventors: Juha REUNANEN; Liisa-Maija KEINÄNEN; Tuomas ROPPONEN - Web page: https://patentgrep.com/patent/US20220284580A1 ## CPC - G06T 7/0014: {using an image reference approach} - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06T 7/0012: {Biomedical image inspection} - G06V 10/764: using classification, e.g. of video objects - G06V 10/7788: {the supervisor being a human, e.g. interactive learning with a human teacher} - G06V 10/82: using neural networks - G06V 20/695: {Preprocessing, e.g. image segmentation} - G06V 20/698: {Matching; Classification} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/10056: Microscopic image - G06T 2207/20021: Dividing image into blocks, subimages or windows - G06T 2207/20076: Probabilistic image processing - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/20104: Interactive definition of region of interest [ROI] - G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro - G06T 2207/30096: Tumor; Lesion - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method of facilitating image analysis in pathology involves receiving a sample image representing a sample for analysis, the sample image including sample image elements, causing one or more functions to be applied to the sample image to determine a plurality of property specific confidence related scores, each associated with a sample image element and a respective sample property and representing a level of confidence that the associated element represents the associated sample property, sorting a set of elements based at least in part on the confidence related scores, producing signals for causing one or more of the set of elements to be displayed to a user in an order based on the sorting, for each of the one or more elements displayed, receiving user input, and causing the user input to be used to update the one or more functions. Other methods, systems, and computer-readable media are disclosed. ## Claims ### Claim 1 A method of facilitating image analysis in pathology, the method comprising: receiving a sample image representing a sample for analysis in pathology, the sample image including a plurality of sample image elements; causing one or more functions to be applied to the sample image to determine a plurality of property specific confidence related scores, each of the plurality of property specific confidence related scores associated with one of the plurality of sample image elements and a respective one of one or more sample properties, the property specific confidence related score representing a level of confidence that the associated sample image element represents the associated sample property; sorting a set of sample image elements of the plurality of sample image elements based at least in part on the property specific confidence related scores associated with the set of sample image elements; producing signals for causing one or more of the set of sample image elements to be displayed to a user in an order based on the sorting; for each of the one or more sample image elements displayed, receiving user input; and causing the user input to be used to update the one or more functions. ### Claim 2 (depends on 1) The method of claim 1 wherein the user input includes a user-provided indication that the sample image element represents one of the one or more sample properties. ### Claim 3 (depends on 1) The method of claim 1 wherein the set of sample image elements includes two or more sample image elements and the one or more of the set of sample image elements displayed includes two or more of the set of sample image elements. ### Claim 4 (depends on 1) The method of claim 1 wherein producing signals for causing the one or more of the set of sample image elements to be displayed to the user comprises, for at least one of the one or more of the set of sample image elements to be displayed to the user, producing signals for causing at least one contextual sample image element proximate to the sample image element to be displayed to the user with the sample image element. ### Claim 5 (depends on 4) The method of claim 4 further comprising: for each of the contextual sample image elements, receiving contextual user input including a user-provided indication that the contextual sample image element represents one of the one or more sample properties; and causing the contextual user input to be used to update the one or more functions. ### Claim 6 (depends on 5) The method of claim 5 wherein sorting the set of sample image elements comprises, for at least one of the set of sample image elements: identifying at least one contextual sample image element to be displayed with the sample image element; and determining a group score based at least in part on the property specific confidence related score of the identified at least one contextual sample image element and the sample image element; and wherein sorting the set of sample image elements comprises sorting based at least in part on the determined group scores. ### Claim 7 (depends on 6) The method of claim 6 wherein determining the group score comprises comparing respective property specific confidence related scores associated with each of the identified at least one contextual sample image element and the sample image element. ### Claim 8 (depends on 1) The method of claim 1 wherein causing the one or more functions to be applied to the sample image to determine the property specific confidence related scores comprises causing the one or more functions to determine more than one related property specific confidence related score associated with each of the sample image elements. ### Claim 9 (depends on 8) The method of claim 8 wherein the more than one related property specific confidence related scores associated with each of the sample image elements are associated with properties that are mutually exclusive. ### Claim 10 (depends on 8) The method of claim 8 wherein sorting the set of sample image elements comprises identifying, for each sample image element of the set of sample image elements, a largest property specific confidence related score of the more than one related property specific confidence related scores associated with the sample image element and sorting based at least in part on the largest property specific confidence related scores. ### Claim 11 (depends on 10) The method of claim 10 wherein sorting based at least in part on the largest property specific confidence related scores comprises sorting in ascending order of the largest property specific confidence related scores associated with each of the sample image elements and wherein producing signals for causing the one or more sample image elements to be displayed to the user comprises producing signals for causing sample image elements associated with lower largest property specific confidence related scores to be displayed first. ### Claim 12 (depends on 8) The method of claim 8 wherein sorting the set of sample image elements comprises identifying, for each sample image element of the set of sample image elements, a second largest property specific confidence related score of the more than one related property specific confidence related scores associated with the sample image element and sorting based at least in part on the second largest property specific confidence related scores. ### Claim 13 (depends on 12) The method of claim 12 wherein sorting based at least in part on the second largest property specific confidence related scores comprises sorting in descending order of the second largest property specific confidence related scores associated with each of the sample image elements and wherein producing signals for causing the one or more sample image elements to be displayed to the user comprises producing signals for causing sample image elements associated with higher second largest property specific confidence related scores to be displayed first. ### Claim 14 (depends on 1) The method of claim 1 wherein sorting the set of sample image elements comprises determining, for each of the set of sample image elements, a difference between the property specific confidence related score associated with the sample image element and a property specific confidence related threshold and sorting the set of sample image elements based at least in part on the determined differences. ### Claim 15 (depends on 1) The method of claim 1 wherein sorting the set of sample image elements comprises sorting the set of sample image elements into a plurality of sorted lists and generating a final sorted list from the plurality of sorted lists, wherein generating the final sorted list comprises adding sample image elements to the final sorted list from each of the plurality of sorted lists. ### Claim 16 (depends on 15) The method of claim 15 wherein adding the sample image elements to the final sorted list from each of the plurality of sorted lists comprises adding the sample image elements alternatingly from each of the plurality of sorted lists. ### Claim 17 (depends on 1) The method of claim 1 comprising determining, for each of the set of sample image elements, whether the sample image element is within a threshold proximity of at least one earlier sorted sample image element. ### Claim 18 (depends on 1) The method of claim 1 wherein applying the one or more functions to the sample image to determine the plurality of property specific confidence related scores comprises, for each of the sample image elements, applying at least one of the one or more functions to a respective subimage included in the sample image and including the sample image element, and disregarding the rest of the sample image, each of the subimages having a width and height less than a width and height of the sample image. ### Claim 19 (depends on 18) The method of claim 18 wherein producing signals for causing the one or more sample image elements to be displayed to the user comprises producing signals for causing the one or more sample image elements to be displayed in one or more subimage size based review images showing a portion of the sample image, each of the one or more subimage size based review images having a width about 1 to 10 times the width of the subimages and having a height about 1 to 10 times the height of the subimages. ### Claim 20 (depends on 1) The method of claim 1 wherein producing signals for causing the one or more sample image elements to be displayed to the user comprises producing signals for causing at least one of the one or more sample image elements to be displayed in a sample image element size based review image showing a portion of the sample image having a width and a height about 10 to 20 times a diameter of the at least one of the one or more sample image elements. ### Claim 21 (depends on 1) The method of claim 1 wherein the one or more functions include one or more neural network functions and wherein causing the user input to be used to update the one or more functions comprises causing the user input to be used to train the one or more neural network functions. ### Claim 22 (depends on 1) The method of claim 1 wherein the sample image elements comprise sets of adjacent pixels. ### Claim 23 (depends on 1) The method of claim 1 wherein the sample image elements comprise groups of pixels identified as objects. ### Claim 24 A system for facilitating image analysis in pathology, the system comprising at least one processor configured to: receive a sample image representing a sample for analysis in pathology, the sample image including a plurality of sample image elements; cause one or more functions to be applied to the sample image to determine a plurality of property specific confidence related scores, each of the plurality of property specific confidence related scores associated with one of the plurality of sample image elements and a respective one of one or more sample properties, the property specific confidence related score representing a level of confidence that the associated sample image element represents the associated sample property; sort a set of sample image elements of the plurality of sample image elements based at least in part on the property specific confidence related scores associated with the set of sample image elements; produce signals for causing one or more of the set of sample image elements to be displayed to a user in an order based on the sorting; for each of the one or more sample image elements displayed, receive user input; and cause the user input to be used to update the one or more functions. ### Claim 25 (depends on 24) The system of claim 24 wherein the at least one processor is configured to, for at least one of the one or more of the set of sample image elements to be displayed to the user, produce signals for causing at least one contextual sample image element proximate to the sample image element to be displayed to the user with the sample image element and wherein the at least one processor is configured to: for each of the contextual sample image elements, receive contextual user input including a user-provided indication that the contextual sample image element represents one of the one or more sample properties; and cause the contextual user input to be used to update the one or more functions. ### Claim 26 (depends on 24) The system of claim 24 wherein the at least one processor is configured to cause the one or more functions to determine more than one related property specific confidence related score associated with each of the sample image elements. ### Claim 27 (depends on 26) The system of claim 26 wherein the at least one processor is configured to identify, for each sample image element of the set of sample image elements, a largest property specific confidence related score of the more than one related property specific confidence related scores associated with the sample image element and sort based at least in part on the largest property specific confidence related scores. ### Claim 28 (depends on 26) The system of claim 26 wherein the at least one processor is configured to identify, for each sample image element of the set of sample image elements, a second largest property specific confidence related score of the more than one related property specific confidence related scores associated with the sample image element and sort based at least in part on the second largest property specific confidence related scores. ### Claim 29 A non-transitory computer-readable medium having stored thereon codes that when executed by at least one processor cause the at least one processor to: receive a sample image representing a sample for analysis in pathology, the sample image including a plurality of sample image elements; cause one or more functions to be applied to the sample image to determine a plurality of property specific confidence related scores, each of the plurality of property specific confidence related scores associated with one of the plurality of sample image elements and a respective one of one or more sample properties, the property specific confidence related score representing a level of confidence that the associated sample image element represents the associated sample property; sort a set of sample image elements of the plurality of sample image elements based at least in part on the property specific confidence related scores associated with the set of sample image elements; produce signals for causing one or more of the set of sample image elements to be displayed to a user in an order based on the sorting; for each of the one or more sample image elements displayed, receive user input; and cause the user input to be used to update the one or more functions. ### Claim 30 A system for facilitating image analysis in pathology, the system comprising: means for receiving a sample image representing a sample for analysis in pathology, the sample image including a plurality of sample image elements; means for causing one or more functions to be applied to the sample image to determine a plurality of property specific confidence related scores, each of the plurality of property specific confidence related scores associated with one of the plurality of sample image elements and a respective one of one or more sample properties, the property specific confidence related score representing a level of confidence that the associated sample image element represents the associated sample property; means for sorting a set of sample image elements of the plurality of sample image elements based at least in part on the property specific confidence related scores associated with the set of sample image elements; means for producing signals for causing one or more of the set of sample image elements to be displayed to a user in an order based on the sorting; means for, for each of the one or more sample image elements displayed, receiving user input; and means for causing the user input to be used to update the one or more functions.