# US 2020/0401854 A1 · Method and system for image segmentation and identification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401854 A1 - Document ID: US20200401854A1 - Status: Published application - Kind: A1 - Application number: 16/448,252 - Filed: 2019-06-21 - Published: 2020-12-24 - Earliest claimed: 2019-06-21 - Assignees: Straxcorp Pty Ltd - Applicants: StraxCorp Pty. Ltd. - Inventors: Yu PENG - Web page: https://patentgrep.com/patent/US20200401854A1 ## CPC - G06T 7/11: Region-based segmentation - G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade} - G06F 18/2155: {characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling} - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/10081: Computed x-ray tomography [CT] - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30004: Biomedical image processing - G06T 2207/30008: Bone - G06V 2201/03: Recognition of patterns in medical or anatomical images - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An image segmentation method system, the system comprising: a training subsystem configured to train a segmentation machine learning model using annotated training data comprising images associated with respective segmentation annotations, so as to generate a trained segmentation machine learning model; a model evaluator; and a segmentation subsystem configured to perform segmentation of a structure or material in an image using the trained segmentation machine learning model. The model evaluator is configured to evaluate the segmentation machine learning model by (i) controlling the segmentation subsystem to segment at least one evaluation image associated with an existing segmentation annotation using the segmentation machine learning model and thereby generate a segmentation of the annotated evaluation image, and (ii) forming a comparison of the segmentation of the annotated evaluation image and the existing segmentation annotation. The method includes deploying the trained segmentation machine learning model for use if the comparison indicates that the segmentation machine learning model is satisfactory. ## Claims ### Claim 1 An image segmentation system, comprising: a training subsystem configured to train a segmentation machine learning model using annotated training data comprising images associated with respective segmentation annotations, so as to generate a trained segmentation machine learning model; a model evaluator; and a segmentation subsystem configured to perform segmentation of a structure or material in an image using the trained segmentation machine learning model; wherein the model evaluator is configured to evaluate the segmentation machine learning model by (i) controlling the segmentation subsystem to segment at least one evaluation image associated with an existing segmentation annotation using the segmentation machine learning model and thereby generate a segmentation of the annotated evaluation image, and (ii) forming a comparison of the segmentation of the annotated evaluation image and the existing segmentation annotation; and deploying or releasing the trained segmentation machine learning model for use if the comparison indicates that the segmentation machine learning model is satisfactory. ### Claim 2 (depends on 1) A system as claimed in claim 1, wherein the system is configured to deploy or release the model for use if the segmentation of the annotated evaluation image and the existing annotation agree to within a predefined threshold. ### Claim 3 (depends on 1) A system as claimed in claim 1, wherein the system is configured to continue training the model if the segmentation of the annotated evaluation image and the existing annotation do not agree to within a predefined threshold. ### Claim 4 (depends on 1) A system as claimed in claim 1, wherein the training subsystem is configured to receive (i) an image and an annotation for the image, and (ii) a score associated with the annotation, the score being indicative of a degree of success or failure of the segmentation machine learning model in segmenting the image, wherein a higher score is indicative of failure and a lower weighting is indicative of success; and retrain or refine the segmentation machine learning model using the image and the annotation, including weighting the annotation of the image according to the score. ### Claim 5 (depends on 1) A system as claimed in claim 1, wherein the training subsystem generates the segmentation machine learning model by refining or modifying an existing segmentation machine learning model. ### Claim 6 (depends on 1) A system as claimed in claim 1, including an annotation subsystem configured to form at least one of the annotated training images from an unannotated or partially annotated image by generating one or more candidate image annotations for the unannotated or partially annotated image, receiving inputs identifying one or more portions of the one or more of the candidate image annotations, and forming the annotated training image from at least the one or more portions. ### Claim 7 (depends on 1) A system as claimed in claim 1, further comprising an identification subsystem, wherein the annotated training data further comprise identification annotations, and a) the segmentation machine learning model is a segmentation and identification machine learning model; or b) the training subsystem is further configured to train an identification machine learning model using (i) the annotated training data once the respective images have been segmented by the segmentation machine learning model and (ii) the identification annotations. ### Claim 8 (depends on 1) A system as claimed in claim 1, wherein the training subsystem includes a model trainer configured to employ machine learning to train the segmentation machine learning model to determine a class of each pixel/voxel on the image. ### Claim 9 A computer-implemented image segmentation method, comprising: training a segmentation machine learning model using annotated training data comprising respective images and segmentation annotations, so as to generate a trained segmentation machine learning model; evaluating the segmentation machine learning model by (i) segmenting at least one evaluation image associated with an existing annotation using the segmentation machine learning model and thereby generating a segmentation of the annotated evaluation image, and (ii) forming a comparison of the segmentation of the annotated evaluation image and the existing annotation; and deploying or releasing the trained segmentation machine learning model for use when the comparison indicates that the segmentation machine learning model is satisfactory. ### Claim 10 (depends on 9) A method as claimed in claim 9, including (a) deploying or releasing the model for use if the segmentation of the annotated evaluation image and the existing annotation agree to within a predefined threshold; and/or (b) continuing to train the model if the segmentation of the annotated evaluation image and the existing annotation do not agree to within a predefined threshold. ### Claim 11 (depends on 9) A method as claimed in claim 9, wherein said training includes receiving (i) an annotated image for use in retraining or refining the segmentation machine learning model, and (ii) a score associated with the annotated image, the score being indicative of a degree of success or failure of the segmentation machine learning model in segmenting the annotated image, wherein a higher score is indicative of failure and a lower weighting is indicative of success; and weighting the annotated image according to the score in the retraining or refining of the segmentation machine learning model. ### Claim 12 (depends on 9) A method as claimed in claim 9, including generating the segmentation machine learning model by refining or modifying an existing segmentation machine learning model. ### Claim 13 (depends on 9) A method as claimed in claim 9, including forming at least one of the annotated training images from an unannotated or partially annotated image by generating one or more candidate image annotations for the unannotated image, receiving inputs identifying one or more portions of the one or more of the candidate image annotations, and forming the annotated training image from at least the one or more portions. ### Claim 14 (depends on 9) A method as claimed in claim 9, wherein (a) the annotated training data further comprise identification data, and the method includes training the segmentation machine learning model as a segmentation and identification machine learning model; or (b) the annotated training data further comprise identification annotations and the method includes training an identification machine learning model using (i) the annotated training data once the respective images have been segmented by the segmentation machine learning model and (ii) the identification annotations. ### Claim 15 (depends on 9) A method as claimed in claim 9, including employing machine learning to train the segmentation machine learning model to determine a class of each pixel/voxel on the image. ### Claim 16 (depends on 1) An image annotation and segmentation system, comprising: a) an input for receiving at least one unannotated or partially annotated image; and b) an annotator configured to form a respective annotated training image from the at least one unannotated or partially annotated image by generating one or more candidate image annotations for the unannotated or partially annotated image, receiving inputs identifying one or more portions of the one or more of the candidate image annotations, and forming the annotated training image from at least the one or more portions; and c) an image segmentation system as claimed in claim 1; wherein the training subsystem is configured to train the segmentation machine learning model using the at least one annotated training image. ### Claim 17 (depends on 9) An image annotation and segmentation method, comprising: a) receiving or accessing at least one unannotated or partially annotated image; and b) forming a respective annotated training image from the unannotated or partially annotated image by generating one or more candidate image annotations for the unannotated image, receiving inputs identifying one or more portions of the one or more of the candidate image annotations, and forming the annotated training image from at least the one or more portions; and c) performing a segmentation method as claimed in claim 9, including training the segmentation machine learning model using the at least one annotated training image. ### Claim 18 (depends on 9) Computer program code configured, when executed by one or more processors, to implement the method of claim 9. ### Claim 19 (depends on 17) Computer program code configured, when executed by one or more processors, to implement the method of claim 17. ### Claim 20 (depends on 18) A computer readable medium, comprising the computer program code of claim 18.