# US 11,896,425 B2 · Guiding instrument insertion > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,896,425 B2 - Document ID: US11896425B2 - Status: Granted - Kind: B2 - Application number: 17/239,323 - Filed: 2021-04-23 - Published: 2022-10-27 - Granted: 2024-02-13 - Earliest claimed: 2021-04-23 - Assignees: Fujifilm Sonosite, Inc. - Applicants: Fujifilm Sonosite, Inc. - Inventors: Davinder S. Dhatt; Christopher Aleksandr White; Adam Benjamin Pely; Thomas Michael Duffy; Paul Tomotaro Danset; Diku Pranav Mandavia - Web page: https://patentgrep.com/patent/US11896425B2 ## CPC - A61B 8/0841: {for locating instruments} - A61B 8/085: {for locating body or organic structures, e.g. tumours, calculi, blood vessels, nodules} - A61B 8/0891: {for diagnosis of blood vessels} - A61B 8/12: in body cavities or body tracts, e.g. by using catheters - A61B 8/4427: {Device being portable or laptop-like} - A61B 8/445: {Details of catheter construction} - A61B 8/4472: {Wireless probes} - A61B 8/461: {Displaying means of special interest} - A61B 8/467: {characterised by special input means} - A61B 8/5207: {involving processing of raw data to produce diagnostic data, e.g. for generating an image} - A61B 8/5223: {for extracting a diagnostic or physiological parameter from medical diagnostic data (for algorithms to analyse biomedical images G06T7/0012)} - G06N 3/02: Neural networks - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06N 3/045: Combinations of networks - G06N 3/08: Learning methods - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method and apparatus for identifying blood vessels in ultrasound images and displaying blood vessels in ultrasound images are described. In some embodiments, the method is implemented by a computing device and includes receiving an ultrasound image that includes one or more blood vessels, and determining, with a neural network implemented at least partially in hardware of the computing device, diameters of the one or more blood vessels in the ultrasound image. The method includes receiving a user selection of an instrument size, and indicating, in the ultrasound image, at least one blood vessel of the one or more blood vessels based on the instrument size and the diameters of the one or more blood vessels. ## Claims ### Claim 1 A method implemented by a computing device for aiding a clinician in selecting an appropriate vein for catheterization, the method comprising: receiving ultrasound echo signals that are reflected from an object in response to ultrasound signals that are sent to the object by an ultrasound probe; receiving a time series of ultrasound images that include one or more blood vessels that are generated based on the ultrasound echo signals, the time series of ultrasound images including a current ultrasound image including a first blood vessel and one or more additional ultrasound images preceding the current ultrasound image; determining, with a neural network implemented at least partially in hardware of the computing device, diameters of each of the one or more blood vessels in the time series of ultrasound images, wherein the first blood vessel in the current ultrasound image has a first diameter caused by a first probe pressure; determining that the first blood vessel exists in the one or more additional ultrasound images preceding the current ultrasound image, wherein the first blood vessel in the one or more additional ultrasound images has a second diameter caused by a second probe pressure; setting a maximum diameter of the first blood vessel based on the first diameter and the second diameter; receiving a user selection of an instrument size; and indicating, in the ultrasound image, the first blood vessel of the one or more blood vessels based on the instrument size and the maximum diameter of the first blood vessel, wherein the indicating includes displaying an outline of the first blood vessel; assigning, with the neural network, one of a vein classification or an artery classification to the one or more blood vessels; and determining a confidence level for the vein classification or the artery classification assigned to the one or more blood vessels, wherein the displaying the outline of the first blood vessel includes displaying the outline with an opacity based on the confidence level determined for the first blood vessel. ### Claim 2 (depends on 1) The method as described in claim 1, further comprising: determining ratios of the instrument size to the diameters of the one or more blood vessels; and comparing the ratios to a threshold ratio; wherein the indicating the first blood vessel is based on the comparing. ### Claim 3 (depends on 1) The method as described in claim 1, wherein the displaying the outline of the first blood vessel includes displaying the outline in a color associated with the instrument size. ### Claim 4 (depends on 1) The method as described in claim 1, wherein the displaying the outline of the first blood vessel includes displaying the outline with an opacity based on a distance of the first blood vessel to an edge of the ultrasound image. ### Claim 5 (depends on 1) The method as described in claim 1, wherein the one or more additional ultrasound images in the time series include at least three ultrasound images. ### Claim 6 (depends on 1) The method as described in claim 1, further comprising generating the one or more additional ultrasound images based on an imaging parameter that is set to a value that is different than the value of the imaging parameter used to generate the current ultrasound image. ### Claim 7 (depends on 5) The method as described in claim 5, further comprising generating at least one of the one or more additional ultrasound images based on a beamform angle that is different than an additional beamform angle used to generate the current ultrasound image. ### Claim 8 (depends on 1) The method as described in claim 1, wherein the one or more additional ultrasound images include an additional image that indicates locations corresponding to the one or more blood vessels; wherein the determining the diameters with the neural network is based on the current ultrasound image and the additional image. ### Claim 9 (depends on 8) The method as described in claim 8, wherein the additional image indicates the locations corresponding to the one or more blood vessels with one or more bounding containers. ### Claim 10 A system to aid a clinician in selecting an appropriate vein for catheterization, the system comprising: a memory to store one or more ultrasound images; a neural network implemented at least partially in hardware coupled to the memory; and a processor coupled to the neural network and configured to: receive ultrasound echo signals that are reflected from an object in response to ultrasound signals that are sent to the object by an ultrasound probe; receive a time series of ultrasound images that includes one or more blood vessels that are generated based on the ultrasound echo signals, the time series of ultrasound images including a current ultrasound image including a first blood vessel and one or more additional ultrasound images preceding the current ultrasound image; determine, using the neural network, diameters of each of the one or more blood vessels in the time series of ultrasound images, wherein the first blood vessel in the current ultrasound image has a first diameter caused by a first probe pressure; determine that the first blood vessel exists in the one or more additional ultrasound images preceding the current ultrasound image, wherein the first blood vessel in the one or more additional ultrasound images has a second diameter caused by a second probe pressure; set a maximum diameter of the first blood vessel based on the first diameter and the second diameter; receive a user selection of an instrument size; and indicate, in the ultrasound image, the first blood vessel of the one or more blood vessels based on the instrument size and the maximum diameter of the first blood vessel, wherein the indicating includes displaying an outline of the first blood vessel; assign, using the neural network, one of a vein classification or an artery classification to the one or more blood vessels; and determine a confidence level for the vein classification or the artery classification assigned to the one or more blood vessels, wherein the displaying the outline of the first blood vessel includes displaying the outline with an opacity based on the confidence level determined for the first blood vessel. ### Claim 11 (depends on 10) The system as described in claim 10, wherein the processor is further configured to: determine ratios of the instrument size to the diameters of the one or more blood vessels; and compare the ratios to a threshold ratio; wherein the indicating the first blood vessel is based on the comparing. ### Claim 12 (depends on 10) The system as described in claim 10, wherein the outline of the first blood vessel is displayed in a color associated with the instrument size. ### Claim 13 (depends on 10) The system as described in claim 10, wherein the displaying the outline of the first blood vessel includes displaying the outline with an opacity based on a distance of the first blood vessel to an edge of the ultrasound image. ### Claim 14 (depends on 10) The system as described in claim 10, wherein the one or more additional ultrasound images are generated based on an imaging parameter that is set to a value that is different than the value of the imaging parameter used to generate the current ultrasound image. ### Claim 15 (depends on 10) The system as described in claim 10, wherein at least one of the one or more additional ultrasound images is generated based on a beamform angle that is different than an additional beamform angle used to generate the ultrasound image. ### Claim 16 (depends on 10) The system as described in claim 10, wherein one or more additional ultrasound images include an additional image that indicates locations corresponding to the one or more blood vessels; wherein the determining the diameters with the neural network is based on the current ultrasound image and the additional image. ### Claim 17 (depends on 16) The system as described in claim 16, wherein the additional image indicates the locations corresponding to the one or more blood vessels with one or more bounding containers.