# US 11,998,279 B2 · Method and system of tracking patient position in operation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,998,279 B2 - Document ID: US11998279B2 - Status: Granted - Kind: B2 - Application number: 17/263,855 - Filed: 2019-07-31 - Published: 2021-10-07 - Granted: 2024-06-04 - Earliest claimed: 2019-07-31 - Assignees: Brain Navi Biotechnology Co., Ltd. - Applicants: Brain Navi Biotechnology Co., Ltd - Inventors: Chieh Hsiao Chen; Kuan Ju Wang - Web page: https://patentgrep.com/patent/US11998279B2 ## CPC - A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 34/30: Surgical robots - A61B 90/36: {Image-producing devices or illumination devices not otherwise provided for} - A61B 90/39: {Markers, e.g. radio-opaque or breast lesions markers} - A61B 2017/00694: {with means correcting for movement of or for synchronisation with the body} - A61B 2034/107: {Visualisation of planned trajectories or target regions} - A61B 2034/2055: {Optical tracking systems} - A61B 2090/364: {Correlation of different images or relation of image positions in respect to the body} - A61B 2090/3983: {Reference marker arrangements for use with image guided surgery} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract Embodiments of the present invention set forth a method to update an operation pathway for a robotic arm assembly in response to a movement of a patient. The method includes processing a two-dimensional image associated with a tag having a spatial relationship with the patient. A corresponding movement of the tag in response to the movement of the patient is determined based on the spatial relationship. The tag includes a first point and a second point and the two-dimensional image includes a first point image and a second point image. The method also includes associating the first point image with the first point and the second point image with the second point and updating the operation pathway based on a conversion matrix of the first point and the second point, and the first point image and the second point image. ## Claims ### Claim 1 A method to update an operation pathway for a robotic arm assembly to perform an operation on a patient having a predetermined spatial relationship with a frame including a tag in response to a movement of the patient, comprising: processing a two-dimensional image associated with the tag having the spatial relationship with the patient, wherein a corresponding movement of the tag in response to the movement of the patient is determined based on the spatial relationship, and wherein the tag includes a first point and a second point, and the two-dimensional image includes a first point image and a second point image; associating the first point image with the first point and the second point image with the second point; and updating the operation pathway based on a conversion matrix of the first point and the second point, and the first point image and the second point image; wherein the updating the operation pathway further includes: obtaining a first projection point associated with the first point and a second projection point associated with the second point in the two-dimensional coordinate system based on the conversion matrix; obtaining a third distance between the first projection point and the first point image in the two-dimensional coordinate system; and obtaining a fourth distance between the second projection point and the second point image in the two-dimensional coordinate system; wherein the method further comprises in response to a sum of the third distance and the fourth distance greater than a second threshold, identifying the movement of the patient and updating the operation pathway. ### Claim 2 (depends on 1) The method of claim 1, wherein the processing the two-dimensional image further includes identifying the first point image and the second point image having intensities greater than a first threshold based on an enlargement factor and a shrinkage factor. ### Claim 3 (depends on 1) The method of claim 1, wherein the processing two-dimensional image further includes assigning a two-dimensional coordinate system to the two-dimensional image and identifying coordinates associated with the first point image and the second point image in the two-dimensional coordinate system. ### Claim 4 (depends on 1) The method of claim 1, wherein the associating the first point image with the first point further comprises: obtaining a geometric center associated with the first point image and the second point image in the two-dimensional coordinate system; obtaining a first distance between the first point image and the geometric center and a second distance between the second point image and the geometric center; and selecting the first point image as a base point image based on the first distance and the second distance. ### Claim 5 (depends on 1) The method of claim 1, wherein the associating the second point image with the second point further comprises: obtaining a first vector from the geometric center to the first point image in the two-dimensional coordinate system; obtaining a second vector from the geometric center to the second point image in the two-dimensional coordinate system; and associating the second point image with the second point based on a cross product of the first vector and the second vector and a dot project of the first vector and the second vector. ### Claim 6 (depends on 1) The method of claim 1, the method further comprising: in response to identifying the movement of the patient, moving the robotic arm assembly to a safety point; and moving the robotic arm assembly from the safety point to an operation target of the patient based on the updated operation pathway. ### Claim 7 (depends on 1) The method of claim 1, the method further comprising verifying the updated operation pathway to identify a collision associated with the robotic arm assembly. ### Claim 8 (depends on 1) The system of claim 1, wherein the processor is further configured to verify the updated operation pathway to identify a collision associated with the robotic arm assembly. ### Claim 9 A system to update an operation pathway for a robotic arm assembly to perform an operation on a patient having a predetermined spatial relationship with a frame including a tag in response to a movement of the patient, comprising: an optical apparatus; and a processor configured to: process a two-dimensional image associated with the tag having the spatial relationship with the patient, wherein a corresponding movement of the tag in response to the movement of the patient is determined based on the spatial relationship, and wherein the tag includes a first point and a second point, and the two-dimensional image includes a first point image and a second point image; associate the first point image with the first point and the second point image with the second point; and update the operation pathway based on a conversion matrix of the first point and the second point, and the first point image and the second point image; wherein the processor is further configured to: obtain a first projection point associated with the first point and a second projection point associated with the second point in the two-dimensional coordinate system based on the conversion matrix; obtain a third distance between the first projection point and the first point image in the two-dimensional coordinate system; obtain a fourth distance between the second projection point and the second point image in the two-dimensional coordinate system; and identify the movement of the patient and update the operation pathway in response to a sum of the third distance and the fourth distance greater than a second threshold. ### Claim 10 (depends on 9) The system of claim 9, wherein the processor is further configured to identify the first point image and the second point image having intensities greater than a first threshold based on an enlargement factor and a shrinkage factor. ### Claim 11 (depends on 9) The system of claim 9, wherein the processor is further configured to assign a two-dimensional coordinate system to the two-dimensional image and identify coordinates associated with the first point image and the second point image in the two-dimensional coordinate system. ### Claim 12 (depends on 9) The system of claim 9, wherein the processor is further configured to: obtain a geometric center associated with the first point image and the second point image in the two-dimensional coordinate system; obtain a first distance between the first point image and the geometric center and a second distance between the second point image and the geometric center; and select the first point image as a base point image based on the first distance and the second distance. ### Claim 13 (depends on 9) The system of claim 9, wherein the processor is further configured to: obtain a first vector from the geometric center to the first point image in the two-dimensional coordinate system; obtain a second vector from the geometric center to the second point image in the two-dimensional coordinate system; and associate the second point image with the second point based on a cross product of the first vector and the second vector and a dot product of the first vector and the second vector. ### Claim 14 (depends on 9) The system of claim 9, wherein the processor is further configured to issue a first command to move the robotic arm assembly to a safety point in response to identifying the movement of the patient and a second command to move the robotic arm assembly from the safety point to an operation target of the patient based on the updated operation pathway. ### Claim 15 A non-transitory computer-readable storage medium containing a set of executable instructions which, in response to execution by a processor, cause the processor to perform a method to update an operation pathway for a robotic arm assembly to perform an operation on a patient having a predetermined spatial relationship with a frame including a tag in response to a movement of the patient, the method comprising: processing a two-dimensional image associated with the tag having the spatial relationship with the patient, wherein a corresponding movement of the tag in response to the movement of the patient is determined based on the spatial relationship, and wherein the tag includes a first point and a second point, and the two-dimensional image includes a first point image and a second point image; associating the first point image with the first point and the second point image with the second point; and updating the operation pathway based on a conversion matrix of the first point and the second point, and the first point image and the second point image.