Surgical simulation for training detection and classification neural networks

A system comprising:

• one or more data processors;
• and
• a non-transitory computer readable storage medium containing instructions which when executed on the one or more data processors, cause the one or more data processors to perform actions including:
• identifying a set of states represented in a procedural workflow;
• for each state of the set of states:
• accessing one or more base images that corresponds to the state;
• and
• generating, for each base image of the one or more base images, first image-segmentation data that indicates a presence and/or location of each of one or more objects within the base image;
• identifying a set of target rendering specifications, wherein the set of target rendering specifications include, for each image-parameter variable of one or more image-parameter variables, multiple different variable values for the image-parameter variable;
• generating a set of virtual images based on the set of target rendering specifications and the one or more base images, wherein, for each of the set of states, the set of virtual images includes at least one virtual image based on the base image that corresponds to the state;
• generating, for each virtual image of the set of virtual images, corresponding data that includes:
• an indication of the state of the set of states with which the virtual image is associated;
• and
• second image-segmentation data that indicates a presence and/or position of each of one or more objects within the virtual image;
• training a machine-learning model using the set of virtual images and corresponding data to define a set of parameter values;
• accessing a real image;
• processing the real image via execution of the trained machine-learning model using the set of parameter values, wherein the processing includes identifying third image-segmentation data that indicates a presence and/or position of each of one or more objects within the real image;
• generating an output based on the third image-segmentation data;
• and
• presenting or transmitting the output.

A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform actions including:

• identifying a set of states represented in a procedural workflow;
• for each state of the set of states:
• accessing one or more base images that corresponds to the state;
• and
• generating, for each base image of the one or more base images, first image-segmentation data that indicates a presence and/or location of each of one or more objects within the base image;
• identifying a set of target rendering specifications, wherein the set of target rendering specifications include, for each image-parameter variable of one or more image-parameter variables, multiple different variable values for the image-parameter variable;
• generating a set of virtual images based on the set of target rendering specifications and the one or more base images, wherein, for each of the set of states, the set of virtual images includes at least one virtual image based on the base image that corresponds to the state;
• generating, for each virtual image of the set of virtual images, corresponding data that includes:
• an indication of the state of the set of states with which the virtual image is associated;
• and
• second image-segmentation data that indicates a presence and/or position of each of one or more objects within the virtual image;
• training a machine-learning model using the set of virtual images and corresponding data to define a set of parameter values;
• accessing a real image;
• processing the real image via execution of the trained machine-learning model using the set of parameter values, wherein the processing includes identifying third image-segmentation data that indicates a presence and/or position of each of one or more objects within the real image;
• generating an output based on the third image-segmentation data;
• and
• presenting or transmitting the output.