# US 2024/0197286 A1 · Automated Needle Entry Detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0197286 A1 - Document ID: US20240197286A1 - Status: Published application - Kind: A1 - Application number: 18/431,256 - Filed: 2024-02-02 - Published: 2024-06-20 - Earliest claimed: 2018-12-05 - Assignees: Fujifilm Sonosite, Inc. - Applicants: FUJIFILM SonoSite, Inc. - Inventors: Edward J. Patton; Tong Li - Web page: https://patentgrep.com/patent/US20240197286A1 ## CPC - A61B 8/0841: {for locating instruments} - A61B 5/7267: {involving training the classification device} - A61B 8/463: {characterised by displaying multiple images or images and diagnostic data on one display} - A61B 8/54: {Control of the diagnostic device} - A61B 8/5246: {combining images from the same or different imaging techniques, e.g. color Doppler and B-mode} - A61B 2017/3413: {guided by ultrasound} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract In one embodiment, an ultrasound imaging system is configured to receive a set of ultrasound images of a target anatomical region. The set of ultrasound images is combined to create a composite tissue frame. The ultrasound imaging system determines whether an interventional instrument is present within the ultrasound images based on a set of trained classification algorithms based on the set of ultrasound images and the composite tissue frame. If an interventional instrument is detected, the ultrasound imaging system further determines whether an additional ultrasound frame should be captured to image the interventional instrument, and the steer angle to be used for the additional ultrasound image. The ultrasound imaging system determines a linear structure corresponding to the interventional instrument, and creates a blended image showing the interventional instrument and the composite tissue frame. ## Claims ### Claim 1 An ultrasound imaging system comprising an image processor, wherein the ultrasound imaging system is configured to: capture a first set of ultrasound image frames of a target region; capture a second set of ultrasound image frames of the target region, wherein: the second set of ultrasound image frames is captured by the ultrasound imaging system subsequent to the first set of ultrasound image frames; at least one of the second set of ultrasound image frames is taken at a first side having a positive steer angle, wherein the positive steer angle is determined relative to a zero angle where a transmit beam of the ultrasound imaging system is perpendicular to an ultrasound transducer of the ultrasound imaging system; and at least another one of the second set of ultrasound image frames is taken from a second side having a negative steer angle relative to the zero angle; determine whether an interventional instrument is present in the target region based on at least the second set of ultrasound image frames; in response to determining that an interventional instrument is present, determine a side corresponding to a direction of entry of the interventional instrument depicted by an image frame, the side determined to be the first side or the second side; and in response to determining the side, create a composite tissue frame depicting a tissue of the target region and the interventional instrument based on at least the first set of ultrasound image frames and a third set of ultrasound image frames taken from a first steer angle on the determined side. ### Claim 2 (depends on 1) The ultrasound imaging system of claim 1, wherein the third set of ultrasound image frames comprises one or more ultrasound image frames from the second set of ultrasound image frames. ### Claim 3 (depends on 1) The ultrasound imaging system of claim 1, wherein the third set of ultrasound image frames is additionally captured by the ultrasound imaging system in response to determining the side. ### Claim 4 (depends on 1) The ultrasound imaging system of claim 1, wherein the ultrasound imaging system is further configured to, in response to the determining the side, for subsequent second sets of ultrasound image frames, only capture ultrasound image frames taken at the first steer angle on the determined side. ### Claim 5 (depends on 1) The ultrasound imaging system of claim 1, wherein the ultrasound imaging system is further configured to, in response to the determining the side, select, from a plurality of steer angles on the determined side, a second steer angle for depicting the interventional instrument, wherein the second steer angle is selected based on a detection that the first steer angle is nearly perpendicular to an orientation of the interventional instrument. ### Claim 6 (depends on 5) The ultrasound imaging system of claim 5, wherein the ultrasound imaging system is further configured to, in response to selecting the second steer angle, use the second steer angle for capturing subsequent second sets of ultrasound image frames. ### Claim 7 (depends on 1) The ultrasound imaging system of claim 1, wherein the ultrasound imaging system is further configured to adjust one or more steer angles of subsequent second sets of ultrasound image frames for subsequent images. ### Claim 8 (depends on 1) The ultrasound imaging system of claim 1, wherein the ultrasound imaging system is further configured to cease capturing frames on an opposite side that is opposite the determined side. ### Claim 9 (depends on 8) The ultrasound imaging system of claim 8, wherein: the determined side corresponds to a left-side entry of the interventional instrument relative to the ultrasound transducer and the opposite side corresponds to a right-side entry of the interventional instrument relative to the ultrasound transducer; or the determined side corresponds to the right-side entry of the interventional instrument and the opposite side corresponds to the left-side entry of the interventional instrument. ### Claim 10 (depends on 1) The ultrasound imaging system of claim 1, wherein the side corresponding to the direction of entry of the interventional instrument is determined based on detection of tissue warping in the second set of ultrasound image frames. ### Claim 11 One or more computer-readable non-transitory storage media storing instructions that, when executed by one or more processors of an ultrasound imaging system, cause the one or more processors to: capture a first set of ultrasound image frames of a target region; capture a second set of ultrasound image frames of the target region, wherein: the second set of ultrasound image frames is captured by the ultrasound imaging system subsequent to the first set of ultrasound image frames; at least one of the second set of ultrasound image frames is taken at a first side having a positive steer angle, wherein the positive steer angle is determined relative to a zero angle where a transmit beam of the ultrasound imaging system is perpendicular to an ultrasound transducer of the ultrasound imaging system; and at least another one of the second set of ultrasound image frames is taken from a second side having a negative steer angle relative to the zero angle; determine whether an interventional instrument is present in the target region based on at least the second set of ultrasound image frames; in response to determining that an interventional instrument is present, determine a side corresponding to a direction of entry of the interventional instrument depicted by an image frame, the side determined to be the first side or the second side; and in response to determining the side, create a composite tissue frame depicting a tissue of the target region and the interventional instrument based on at least the first set of ultrasound image frames and a third set of ultrasound image frames taken from a first steer angle on the determined side. ### Claim 12 (depends on 11) The ultrasound imaging system of claim 11, wherein the third set of ultrasound image frames comprises one or more ultrasound image frames from the second set of ultrasound image frames. ### Claim 13 (depends on 11) The ultrasound imaging system of claim 11, wherein the third set of ultrasound image frames is additionally captured by the ultrasound imaging system in response to determining the side. ### Claim 14 (depends on 11) The ultrasound imaging system of claim 11, wherein the ultrasound imaging system is further configured to, in response to the determining the side, for subsequent second sets of ultrasound image frames, only capture ultrasound image frames taken at the first steer angle on the determined side. ### Claim 15 (depends on 11) The ultrasound imaging system of claim 11, wherein the ultrasound imaging system is further configured to, in response to the determining the side, select, from a plurality of steer angles on the determined side, a second steer angle for depicting the interventional instrument, wherein the second steer angle is selected based on a detection that the first steer angle is nearly perpendicular to an orientation of the interventional instrument. ### Claim 16 (depends on 15) The ultrasound imaging system of claim 15, wherein the ultrasound imaging system is further configured to, in response to selecting the second steer angle, use the second steer angle for capturing subsequent second sets of ultrasound image frames. ### Claim 17 (depends on 11) The ultrasound imaging system of claim 11, wherein the ultrasound imaging system is further configured to adjust one or more steer angles of subsequent second sets of ultrasound image frames for subsequent images. ### Claim 18 (depends on 11) The ultrasound imaging system of claim 11, wherein the ultrasound imaging system is further configured to cease capturing frames on an opposite side that is opposite the determined side. ### Claim 19 (depends on 18) The ultrasound imaging system of claim 18, wherein: the determined side corresponds to a left-side entry of the interventional instrument relative to the ultrasound transducer and the opposite side corresponds to a right-side entry of the interventional instrument relative to the ultrasound transducer; or the determined side corresponds to the right-side entry of the interventional instrument and the opposite side corresponds to the left-side entry of the interventional instrument. ### Claim 20 (depends on 11) The ultrasound imaging system of claim 11, wherein the side corresponding to the direction of entry of the interventional instrument is determined based on detection of tissue warping in the second set of ultrasound image frames.