# US 2023/0260413 A1 · Surgical procedure labeling and teaching system and method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0260413 A1
- Document ID: US20230260413A1
- Status: Published application
- Kind: A1
- Application number: 17/673,748
- Filed: 2022-02-16
- Published: 2023-08-17
- Earliest claimed: 2022-02-16
- Assignees: Smart Surgery
- Applicants: Smart Surgery
- Inventors: Yu-Chieh LEE; Yi-Ta SHEN; Hsin-Man CHIANG
- Web page: https://patentgrep.com/patent/US20230260413A1

## CPC

- G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06T 7/0014: {using an image reference approach}
- G09B 5/06: with both visual and audible presentation of the material to be studied
- G09B 5/065: {Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems}
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)}
- G16H 30/40: for processing medical images, e.g. editing
- G16H 70/20: relating to practices or guidelines
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20092: Interactive image processing based on input by user
- G06T 2207/30004: Biomedical image processing
- G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A surgical procedure labeling and teaching system and a method thereof are disclosed. When one of names of to-be-labeled surgical images on a surgical image selection area of a user interface is selected, the to-be-labeled surgical image corresponding to the selected name is displayed on a surgical image labeling area of the user interface; after one of surgery label blocks on a surgery label selection area of the user interface is selected, a user can circle a surgery labeling area to establish a surgery feature label on the surgical image labeling area. A teaching interface can map the labeled surgical image to the to-be-labeled surgical image, so as to provide a surgical procedure teaching corresponding to the to-be-labeled surgical image. The technical effect of providing surgical procedure teaching by using the labeling-reference surgical image to assist in labeling and selecting of the to-be-labeled surgical image is achieved.

## Claims

### Claim 1

A surgical procedure labeling and teaching system, executed in an electronic device and comprising:
a user interface module, the user interface module is configured to display a user interface, wherein the user interface comprises a surgical image selection area, a surgical image labeling area, a label record area, a surgery video playback area, an image-labeling reference area and a surgery label selection area;
a video playback module, the video playback module is configured to play a surgery video on the surgery video playback area;
a display module, wherein when one of names of to-be-labelled surgical images on the surgical image selection area is selected, the display module displays the to-be-labelled surgical image corresponding to the selected name on the surgical image labelling area, and displays the labelling-reference surgical image corresponding to the selected name on the image-labelling reference area;
a labeling module, wherein when one of the surgery label block on the surgery label selection area is selected, the labeling module circles a surgery labelling area on the surgical image labelling area, and fills the surgery labeling area with a color corresponding to the selected surgery label block on the surgical image labeling area;
a label establishing module, the label establishing module is configured to establish and display a background label on the label record area, wherein after the surgery labelling area is circled on the surgical image labelling area, the label establishing module establishes a name label on the label record area based on a name corresponding to the selected surgery label block;
a storage module, the storage module is configured to store the labelled to-be-labelled surgical image as a labelled surgical image, wherein the labelled surgical image comprises surgery labeling areas filled with colors; and
a teaching module, the teaching module is configured to display a teaching interface comprising a to-be-labeled surgical image area, a labelled surgical image area and a surgery label display area, wherein the teaching module displays the to-be-labelled surgical images on the to-be-labelled surgical image area, and displays the labelled surgical image on the labelled surgical image area, and provides a surgical procedure teaching corresponding to the to-be-labeled surgical image by the surgery labelling areas of the labelled surgical image filled with colors and mapped to the to-be-labelled surgical image, and an auxiliary content corresponding to the surgery labelling area on the surgery label display area.

### Claim 2 (depends on 1)

The surgical procedure labelling and teaching system according to claim 1, wherein when one of the names of to-be-labelled surgical images on the surgical image selection area is selected, the video playback module captures a part of the surgery video in a period from a video time point minus a preset time length to the video time point plus the time length, to generate a captured surgery video, based on the video time point of the selected to-be-labeled surgical image, and video playback module plays the captured surgery video.

### Claim 3 (depends on 1)

The surgical procedure labelling and teaching system according to claim 1, wherein the display module displays the name of the surgical image labelled already, the name of the surgical image being labeled, and the name of surgical image not labelled yet, with different background patterns on the surgical image selection area.

### Claim 4 (depends on 1)

The surgical procedure labelling and teaching system according to claim 1, wherein when one of the surgery label blocks displayed on the surgery label selection area is selected by the labelling module, all of the surgery label blocks on the surgery label selection area are not permitted to select, and all of the surgery label blocks are displayed with the same effect.

### Claim 5 (depends on 1)

The surgical procedure labelling and teaching system according to claim 1, wherein the surgery label blocks have different display priorities, the surgery labelling area filled with the surgery label block having a high display priority is always displayed over the surgery labelling area filled with the surgery label block having a low display priority.

### Claim 6

A surgical procedure labelling and teaching method, comprising:
displaying a user interface comprising a surgical image selection area, a surgical image labeling area, a label record area, a surgery video playback area, an image-labeling reference area, and a surgery label selection area, by an electronic device;
playing a surgery video on the surgery video playback area, by the electronic device;
when one of names of to-be-labelled surgical images displayed on the surgical image selection area is selected, displaying the to-be-labelled surgical image corresponding to the selected name on the surgical image labelling area;
when one of the names of the to-be-labelled surgical images displayed on the surgical image selection area is selected, displaying a labelling-reference surgical image corresponding to the selected name on the image-labelling reference area;
establishing and displaying a background label on the label record area, by the electronic device;
when one of surgery label blocks displayed on the surgery label selection area is selected, circling a surgery labelling area on the surgical image labelling area;
filling the surgery labeling area with a color corresponding to the selected surgery label block on the surgical image labeling area, and establishing a name label based on a name corresponding to the selected surgery label block on the label record area, by the electronic device;
storing the to-be-labelled surgical image, which is labeled already, as a labelled surgical image, by the electronic device, wherein the labelled surgical image comprises surgery labeling areas filled with colors;
displaying a teaching interface comprising a to-be-labeled surgical image area, a labelled surgical image area and a surgery label display area, by an electronic device;
displaying the to-be-labelled surgical image on the to-be-labelled surgical image area, and displaying the labelled surgical image on the labelled surgical image area, by the electronic device; and
displaying the surgery labelling areas of the labelled surgical image filled with colors and mapped to the to-be-labelled surgical image, and displaying an auxiliary content corresponding to the surgery labelling area on the surgery label display area, so as to provide a surgical procedure teaching corresponding to the to-be-labeled surgical image.

### Claim 7 (depends on 6)

The surgical procedure labelling and teaching method according to claim 6, further comprising:
when one of the names of to-be-labelled surgical images on the surgical image selection area is selected, capturing a part of the surgery video in a period from a video time point minus a preset time length to the video time point plus the time length, to generate a captured surgery video, based on the video time point of the selected to-be-labeled surgical image, and playing the captured surgery video, by the electronic device.

### Claim 8 (depends on 6)

The surgical procedure labelling and teaching method according to claim 6, further comprising:
displaying the name of the surgical image labelled already, the name of the surgical image being labeled, and the name of surgical image not labelled yet, with different background patterns on the surgical image selection area, by the electronic device.

### Claim 9 (depends on 6)

The surgical procedure labelling and teaching method according to claim 6, wherein when one of the surgery label blocks displayed on the surgery label selection area is selected by the labelling module, all of the surgery label blocks on the surgery label selection area are not permitted to select, and all of the surgery label blocks are displayed with the same effect.

### Claim 10 (depends on 6)

The surgical procedure labelling and teaching method according to claim 6, wherein the surgery label blocks have different display priorities, the electronic device always displays the surgery labelling area filled with the surgery label block having a high display priority over the surgery labelling area filled with the surgery label block having a low display priority.
