# US 11,992,362 B2 · Ultrasound-guided alignment and insertion of percutaneous cannulating instruments > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,992,362 B2 - Document ID: US11992362B2 - Status: Granted - Kind: B2 - Application number: 17/284,018 - Filed: 2019-10-09 - Published: 2021-12-09 - Granted: 2024-05-28 - Earliest claimed: 2018-10-09 - Assignees: Rutgers, The State University of New Jersey - Applicants: Rutgers, The State University of New Jersey - Inventors: Martin L. Yarmush; Alvin Chen; Max Balter; Joshua Leipheimer - Web page: https://patentgrep.com/patent/US11992362B2 ## CPC - A61B 8/0841: {for locating instruments} - A61B 5/15003: {for venous or arterial blood} - A61B 5/150748: {Having means for aiding positioning of the piercing device at a location where the body is to be pierced (means for aiding positioning of devices for bringing media into the body A61M5/427, locating blood vessels A61B5/489)} - A61B 5/1545: {comprising means for indicating vein or arterial entry} - A61B 8/06: Measuring blood flow - A61B 8/12: in body cavities or body tracts, e.g. by using catheters - A61B 8/4218: {characterised by articulated arms} - A61B 8/466: {adapted to display 3D data} - A61B 17/3403: {Needle locating or guiding means} - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61M 5/427: {Locating point where body is to be pierced, e.g. vein location means using ultrasonic waves, injection site templates} - A61B 8/0891: {for diagnosis of blood vessels} - A61B 8/429: {characterised by determining or monitoring the contact between the transducer and the tissue} - A61B 2017/3413: {guided by ultrasound} - A61B 2034/2048: {using an accelerometer or inertia sensor} - A61B 2034/2051: {Electromagnetic tracking systems} - A61B 2034/2059: {Mechanical position encoders} - A61B 2034/2065: {Tracking using image or pattern recognition} - A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - A61M: DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY (introducing media into or onto the bodies of animals A61D7/00; means for inserting tampons A61F13/26; devices for administering food or medicines orally A61J; containers for collecting, storing or administering blood or medical fluids A61J1/05); DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY (surgery A61B; chemical aspects of surgical articles A61L); DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR {(Electrotherapy, e.g. producing anaesthesia by the use of alternating or intermittent currents A61N1/36021)} ## Abstract A device and method for directing a cannula toward a target location under a patient's skin. The device is handheld and the device operations may be automated. The device includes an imaging probe for imaging the target location, a positioning unit for manipulating a cannula towards the target location, and a processor. The processor receives imaging data from the imaging probe, cannula pose data from at least one cannula position sensor, and device pose data from at least one device position sensor. The target location is identified from the imaging data, and a trajectory for manipulating the cannula towards the target location is determined based on the imaging data, the cannula pose data, and the device pose data. The processor may determine that the trajectory becomes misaligned with the target position, and may update to a corrected trajectory based on the imaging data, cannula pose data, and the device pose data. ## Claims ### Claim 1 A device comprising: an imaging probe for continuously generating imaging data of a target location under a patient's skin, wherein the target location is a center of a target vessel; a positioning unit including: a plurality of motors configured to manipulate a cannula attached to the positioning unit toward the target location, at least one cannula position sensor configured to generate cannula pose data indicating a position and orientation of the cannula based on movement of the corresponding motors; and at least one device position sensor configured to generate device pose data indicating an orientation and position of the device; and a processor configured to: receive imaging data from the imaging probe; receive cannula pose data from the at least one cannula position sensor; receive device pose data from the at least one device position sensor; identify the target location from the imaging data; determine a trajectory for manipulating the cannula towards the target location based on the imaging data, the cannula pose data, and the device pose data; instruct the positioning unit to manipulate the cannula according to the determined trajectory; identify the determined trajectory becoming misaligned with the target location based on a detected motion of the target location; update the determined trajectory to a corrected trajectory based on the imaging data, the cannula pose data, and the device pose data; and instruct the positioning unit to manipulate the cannula according to the corrected trajectory. ### Claim 2 (depends on 1) The device of claim 1, wherein the imaging probe is an ultrasound probe, and wherein the imaging data generated by the imaging probe is one of: two-dimensional imaging data, volumetric three-dimensional imaging data, and biplanar imaging data. ### Claim 3 (depends on 2) The device of claim 2, wherein the device is adapted to obtain imaging data of the target location from a plurality of angles, wherein the processor is further configured to perform at least one of: constructing three-dimensional imaging data from the received imaging data obtained from the plurality of angles; and determining a three-dimensional pose of the target location under the patient's skin from the imaging data obtained from the plurality of angles. ### Claim 4 (depends on 1) The device of claim 1, wherein the imaging data includes video data of the target location obtained at different times, and wherein the processor is configured to determine the trajectory for manipulating the cannula towards the target pose based on a comparison between previous and subsequent frames of the video data. ### Claim 5 (depends on 1) The device of claim 1, wherein the positioning unit is configured to manipulate the cannula in at least two degrees of freedom. ### Claim 6 (depends on 1) The device of claim 1, wherein the processor is configured to identify the determined trajectory becoming misaligned with the target location based on device pose data received from the at least one device position sensor. ### Claim 7 (depends on 6) The device of claim 6, wherein the at least one device position sensor comprises at least one of: a camera, an electromagnetic sensor, a gyroscope and an accelerometer. ### Claim 8 (depends on 1) The device of claim 1, further comprising a force sensor configured to generate force data indicating a measured amount of force exerted on the cannula, wherein the processor is configured to: receive the force data from the force sensor and to update the cannula pose data to a corrected position and orientation based on the received force data; and instruct the cannula positioning unit to manipulate the cannula according to the corrected position and orientation. ### Claim 9 (depends on 8) The device of claim 8, further comprising a blood flash sensor configured to generate blood flash data indicating a flow of blood into the cannula, wherein the processor is configure to: receive the blood flash data from the blood flash sensor; and instruct the cannula positioning unit to manipulate the cannula based further on the blood flash data. ### Claim 10 (depends on 1) The device of claim 1, further comprising a force sensor configured to generate force data indicating a measured amount of force exerted on the cannula, wherein the processor is configured to receive the force data from the force sensor and to: obtain a force profile from the force data; classify the obtained force profile as one of a successful cannulation event, vessel rolling event, undershoot event, and overshoot event; and in an absence of a successful cannulation event, instruct the positioning unit to manipulate the cannula in response to the classified event. ### Claim 11 (depends on 1) The device of claim 1, wherein the at least one cannula position sensor comprises a plurality of motor sensors, each motor sensor associated with a corresponding one of the plurality of motors to generate cannula pose data. ### Claim 12 A method executed by a processor coupled to an imaging probe for continuously generating imaging data of a target location under a patient's skin, wherein the target location is a center of a target vessel, and a positioning unit including a plurality of motors configured to manipulate a cannula attached to the positioning unit toward the target location, at least one cannula position sensor configured to generate cannula pose data indicating a position and orientation of the cannula based on movement of the corresponding motors, and at least one device position sensor configured to generate device pose data indicating an orientation and position of the device, wherein the method comprises: receiving imaging data from the imaging probe; receiving cannula pose data from the at least one cannula position sensor; receiving device pose data from the at least one device position sensor; identifying the target location from the imaging data; determining a trajectory for manipulating the cannula towards the target location based on the imaging data, the cannula pose data, and the device pose data; instructing the positioning unit to manipulate the cannula according to the determined trajectory; identifying the determined trajectory becoming misaligned with the target location based on a detected motion of the target location; updating the determined trajectory to a corrected trajectory based on the imaging data, the cannula pose data, and the device pose data; and instructing the positioning unit to manipulate the cannula according to the corrected trajectory. ### Claim 13 (depends on 12) The method of claim 12, wherein the method further comprises: identifying and determining a pose of a target vessel under the patient's skin based on the imaging data; and instructing the positioning unit to manipulate the cannula based on the determined pose of the target vessel. ### Claim 14 (depends on 12) The method of claim 12, wherein receiving imaging data from the probe comprises receiving imaging data from a plurality of angles, and wherein the method further comprises at least one of: constructing three-dimensional imaging data from the received imaging data obtained from the plurality of angles; and determining the pose of the target location under the patient's skin from the imaging data obtained from the plurality of angles. ### Claim 15 (depends on 12) The method of claim 12, wherein instructing the positioning unit to manipulate the cannula comprises providing instructions for manipulating each of the positioning unit's two or more degrees of freedom. ### Claim 16 (depends on 12) The method of claim 12, wherein identifying the determined trajectory becoming misaligned with the target location is based on the device pose data received from the device position sensor. ### Claim 17 (depends on 12) The method of claim 12, wherein determining the cannula pose data is based on the imaging data using at least one of a computer vision algorithm or an electromagnetic sensor. ### Claim 18 (depends on 12, 13, 14) The method of any one of claims 12-14, wherein the processor is further coupled to a force sensor for generating force data indicating a measured amount of force exerted on the cannula, and wherein the method further comprises: receiving the force data from the force sensor; updating the cannula positioning data to a corrected position and orientation based on the received force data; and instructing the cannula positioning unit to manipulate the cannula according to the corrected position and orientation. ### Claim 19 (depends on 12) The method of claim 12, wherein the processor is further coupled to a force sensor for generating force data indicating a measured amount of force exerted on the cannula, and wherein the method further comprises: obtaining a force profile from the force data; classifying the obtained force profile as one of a successful cannulation event, vessel rolling event, undershoot event, and overshoot event; and in an absence of a successful cannulation event, instructing the positioning unit to manipulate the cannula in response to the classified event.