# US 11,998,293 B2 · Systems and methods for entering and exiting a teleoperational state > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,998,293 B2 - Document ID: US11998293B2 - Status: Granted - Kind: B2 - Application number: 18/061,357 - Filed: 2022-12-02 - Published: 2023-04-13 - Granted: 2024-06-04 - Earliest claimed: 2017-03-06 - Assignees: Intuitive Surgical Operations, Inc. - Applicants: Intuitive Surgical Operations, Inc. - Inventors: Govinda Payyavula; Simon P. Dimaio; John William Henrie; Brandon D. Itkowitz; Anthony M. Jarc - Web page: https://patentgrep.com/patent/US11998293B2 ## CPC - A61B 34/35: for telesurgery - A61B 34/30: Surgical robots - A61B 34/74: {Manipulators with manual electric input means} - B25J 9/1689: {Teleoperation} - A61B 2017/00216: {with eye tracking or head position tracking control} - A61B 2034/2055: {Optical tracking systems} - A61B 2034/741: {Glove like input devices, e.g. "data gloves"} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) ## Abstract A teleoperational medical system includes a teleoperational control system, a teleoperational manipulator configured for operating a medical instrument in a medical environment, and an operator controller. The teleoperational control system includes a processing unit including one or more processors. The processing unit is configured to determine whether a head portion of an operator of the operator controller is directed toward or away from a display region of a display device based on at least one marker on the operator that is configured to remain substantially fixed relative to the head portion of the operator during a movement of the head portion. The processing unit is further configured to, based on a determination that the head portion is directed away from the display region for at least a threshold time period, enter a standby mode such that movement of the operator controller does not provide corresponding movement to the teleoperational manipulator. ## Claims ### Claim 1 A teleoperational medical system comprising: a teleoperational control system; a teleoperational manipulator configured for operating a medical instrument in a medical environment; and an operator controller in communication with the teleoperational control system, wherein the teleoperational control system includes a processing unit including one or more processors, and wherein the processing unit is configured to: determine whether a head portion of an operator of the operator controller is directed toward or away from a display region of a display device based on at least one marker on the operator, wherein the at least one marker is configured to remain substantially fixed relative to the head portion of the operator during a movement of the head portion; and based on a determination that the head portion is directed away from the display region for at least a threshold time period, enter a standby mode such that movement of the operator controller does not provide corresponding movement to the teleoperational manipulator. ### Claim 2 (depends on 1) The teleoperational medical system of claim 1, wherein the processing unit is configured to prevent initiation of an operator following mode responsive to a determination that the head portion is not directed toward the display region. ### Claim 3 (depends on 1) The teleoperational medical system of claim 1, wherein the processing unit is configured to prevent initiation of an operator following mode responsive to a determination that the operator is not present at the operator controller. ### Claim 4 (depends on 1) The teleoperational medical system of claim 1, wherein the at least one marker includes a natural landmark on a face of the operator. ### Claim 5 (depends on 1) The teleoperational medical system of claim 1, wherein the processing unit is further configured to: determine whether the head portion is positioned in a predefined region in the medical environments; and initiate an operator following mode based on a determination that the head portion is positioned in the predefined region. ### Claim 6 (depends on 1) The teleoperational medical system of claim 1, wherein the processing unit is further configured to: determine whether the operator has performed an activation gesture; and initiate an operator following mode based on a determination that the operator has performed the activation gesture. ### Claim 7 (depends on 6) The teleoperational medical system of claim 6, wherein the activation gesture is a matching hand grip, that matches a displayed configuration of the medical instrument on the display device, or a movement of a pedal device. ### Claim 8 (depends on 1) The teleoperational medical system of claim 1, wherein the processing unit is further configured to: identify the operator; and initiate an operator following mode based on a determination that the identified operator is permitted to operate the teleoperational medical system. ### Claim 9 (depends on 1) The teleoperational medical system of claim 1, wherein the threshold time period is a first threshold time period for a first direction that the head portion is directed away from the display region, wherein the threshold time period is a second threshold time period for a second direction that the head portion is directed away from the display region, wherein the first threshold time period differs from the second threshold time period, and wherein the first direction differs from the second direction. ### Claim 10 (depends on 1) The teleoperational medical system of claim 1, wherein determining whether the operator of the operator controller has the head portion directed toward or away from the display region of the display device includes determining an operator viewing vector and determining whether the operator viewing vector intersects a display plane within the display region. ### Claim 11 (depends on 1) The teleoperational medical system of claim 1, wherein the at least one marker comprises, or is coupled to, an item worn by the operator, the item selected from the group consisting of: glasses, a headband, and an earpiece. ### Claim 12 (depends on 1) The teleoperational medical system of claim 1, wherein the processing unit is further configured to: determine that the head portion has been directed away from the display region of the display device for less than the threshold time period and has returned to be directed toward the display region; and remain in an operator following mode based on the determination that the head portion has been directed away from the display region for less than the threshold time period. ### Claim 13 A method for operating a teleoperational system including a teleoperational manipulator and an operator controller, the method comprising: determining whether a head portion of an operator is directed toward or away from a display region of a display device based on at least one marker on the operator, wherein the at least one marker remains substantially fixed relative to the head portion of the operator during a movement of the head portion; and based on a determination that the head portion is directed away from the display region for at least a threshold period of time, enter a standby mode such that movement of the operator controller does not provide corresponding movement to the teleoperational manipulator. ### Claim 14 (depends on 13) The method of claim 13, further comprising: preventing initiation of an operator following mode responsive to a determination that the head portion is not directed toward the display region or that the operator is not present at the operator controller. ### Claim 15 (depends on 13) The method of claim 13, further comprising: determining whether the head portion is positioned in a predefined region; and initiating an operator following mode based on a determination that the head portion is positioned in the predefined region. ### Claim 16 (depends on 13) The method of claim 13, further comprising: determining whether the operator has performed an activation gesture; and initiating an operator following mode based on a determination that the operator has performed the activation gesture. ### Claim 17 (depends on 13) The method of claim 13, further comprising: determining that the head portion has been directed away from the display region of the display device for less than the threshold period of time and has returned to be directed toward the display region; and remaining in an operator following mode based on the determination that the head portion has been directed away from the display region for less than the threshold period of time. ### Claim 18 A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which when executed by one or more processors associated with a medical device are adapted to cause the one or more processors to perform a method comprising: determining whether a head portion of an operator is directed toward or away from a display region of a display device based on at least one marker on the operator, wherein the at least one marker remains substantially fixed relative to the head portion of the operator during a movement of the head portion; and based on a determination that the head portion is directed away from the display region for at least a threshold period of time, enter a standby mode such that the movement of an operator controller does not provide corresponding movement to a teleoperational manipulator.