# US 2023/0065764 A1 · Method for operating a surgical system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0065764 A1 - Document ID: US20230065764A1 - Status: Published application - Kind: A1 - Application number: 17/899,915 - Filed: 2022-08-31 - Published: 2023-03-02 - Earliest claimed: 2021-09-02 - Assignees: Olympus Winter & Ibe GmbH - Applicants: OLYMPUS Winter & Ibe GmbH - Inventors: Hendrik TIMMERMANN; Andreas MUECKNER - Web page: https://patentgrep.com/patent/US20230065764A1 ## CPC - A61B 1/000096: {using artificial intelligence} - A61B 1/0676: {at distal tip of an endoscope} - A61B 1/00154: {using guiding arrangements for insertion} - A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion - A61B 1/0655: {Control therefor} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - A61B 1/0008: {characterised by distal tip features} - A61B 90/30: Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure - G06N 20/00: Machine learning - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method including: upon or during the insertion of an endoscope shaft into a trocar sleeve, detecting first brightness values at a distal end of the endoscope shaft by an image sensor over time and detecting a first temporal brightness profile depending on the detected first brightness values and the time, and/or upon or during the withdrawal of the endoscope shaft from the trocar sleeve, detecting second brightness values at the distal end of the endoscope shaft by the image sensor over time and detecting a second temporal brightness profile depending on the detected second brightness values and the time. ## Claims ### Claim 1 A method comprising one or more of: upon or during insertion of an endoscope shaft into a trocar sleeve, detecting first brightness values at a distal end of the endoscope shaft by an image sensor over time and detecting a first temporal brightness profile depending on the detected first brightness values and the time, and upon or during withdrawal of the endoscope shaft from the trocar sleeve, detecting second brightness values at the distal end of the endoscope shaft by the image sensor over time and detecting a second temporal brightness profile depending on the detected second brightness values and the time. ### Claim 2 (depends on 1) The method according to claim 1, further comprising determining a passage of the distal end of the endoscope shaft through a distal opening of the trocar sleeve based on the first temporal brightness profile. ### Claim 3 (depends on 2) The method according to claim 2, wherein the determining of the passage of the distal end of the endoscope shaft through the distal opening of the trocar sleeve comprises completely determining the passage. ### Claim 4 (depends on 2) The method according to claim 2, further comprising, when determining the passage of the distal end of the endoscope shaft through the distal opening of the trocar sleeve, generating a time stamp for the passage of the distal end of the endoscope shaft. ### Claim 5 (depends on 1) The method according to claim 1, further comprising, determining a withdrawal of the distal end of the endoscope shaft from a proximal opening of the trocar sleeve based on the second temporal brightness profile. ### Claim 6 (depends on 5) The method according to claim 5, further comprising, when determining the withdrawal of the distal end of the endoscope shaft from the proximal opening of the trocar sleeve, generating a time stamp for the withdrawal of the distal end of the endoscope shaft. ### Claim 7 (depends on 5) The method according to claim 5, wherein, during or after determining the withdrawal of the distal end of the endoscope shaft from the proximal opening of the trocar sleeve, turning off or dimming the light generated by the light source. ### Claim 8 (depends on 1) The method according to claim 1, further comprising, when inserting the distal end of the endoscope shaft into the trocar sleeve, controlling the light strength of the light source based on the first temporal brightness. ### Claim 9 (depends on 8) The method according to claim 8, wherein the controlling is performed using a closed loop processor. ### Claim 10 (depends on 1) The method according to claim 1, further comprising ascertaining a reduction of the degree of transmission of the optical waveguide based on one or more of: the first brightness values by a comparison with a first brightness setpoint at a first predetermined position of the distal end of the endoscope shaft in the trocar sleeve; and the second brightness values by a comparison with a second brightness setpoint at a second predetermined position of the distal end of the endoscope shaft. ### Claim 11 (depends on 10) The method according to claim 10, wherein the ascertaining of the reduction of the degree of transmission of the optical waveguide is performed automatically. ### Claim 12 (depends on 10) The method according to claim 10, wherein the first brightness setpoint is a predetermined first brightness setpoint and the second brightness setpoint is a predetermined second brightness setpoint ### Claim 13 (depends on 1) The method according to claim 1, further comprising ascertaining a reduction of the degree of transmission of the optical waveguide (based on one or more of: the first temporal brightness profile by a comparison with a first brightness setpoint; and the second temporal brightness profile by comparing with a second brightness setpoint at a second predetermined position of the distal end of the endoscope shaft. ### Claim 14 (depends on 1) The method according to claim 1, wherein the image sensor comprises one of a CMOS sensor and a CCD sensor and digital brightness values of at least one pixel or an array of pixels or all pixels of the image sensors are detected by the image sensor. ### Claim 15 (depends on 1) The method according to claim 1, further comprising one or more of: first processing the first brightness values when inserting the distal end of the endoscope shaft and the first temporal brightness profile by a processor and second processing the second brightness values when withdrawing the distal end of the endoscope shaft and the second temporal brightness profile by the processor. ### Claim 16 (depends on 15) The method according to claim 15, wherein one or more of the first and second processing is performed using an artificial neural network or a machine learning algorithm. ### Claim 17 (depends on 1) The method according to claim 1, wherein one or more of the detecting of the first and second temporal brightness profile is performed automatically. ### Claim 18 A surgical system comprising: a processor comprising hardware, the processor being configured to one or more of: upon or during insertion of an endoscope shaft into a trocar sleeve, detecting first brightness values at a distal end of the endoscope shaft by an image sensor over time and detecting a first temporal brightness profile depending on the detected first brightness values and the time, and upon or during withdrawal of the endoscope shaft from the trocar sleeve, detecting second brightness values at the distal end of the endoscope shaft by the image sensor over time and detecting a second temporal brightness profile depending on the detected second brightness values and the time. ### Claim 19 Non-transitory computer-readable storage medium storing instructions that cause a computer to one of: upon or during the insertion of the endoscope shaft into the trocar sleeve, detect first brightness values at the distal end of the endoscope shaft by an image sensor over time and detect a first temporal brightness profile depending on the detected first brightness values and the time, and upon or during the withdrawal of the endoscope shaft from the trocar sleeve, detect second brightness values at the distal end of the endoscope shaft by the image sensor over time and detect a second temporal brightness profile depending on the detected second brightness values and the time.