# US 2019/0167148 A1 · Systems And Methods For Visualizing Anatomy, Locating Medical Devices, Or Placing Medical Devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0167148 A1 - Document ID: US20190167148A1 - Status: Published application - Kind: A1 - Application number: 16/209,601 - Filed: 2018-12-04 - Published: 2019-06-06 - Earliest claimed: 2017-12-04 - Assignees: Bard Access Systems, Inc. - Applicants: Bard Access Systems, Inc. - Inventors: Tyler L. Durfee; Eddie K. Burnside - Web page: https://patentgrep.com/patent/US20190167148A1 ## CPC - A61B 5/062: {using magnetic field} - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 34/25: {User interfaces for surgical systems} - A61B 90/36: {Image-producing devices or illumination devices not otherwise provided for} - A61B 2017/00216: {with eye tracking or head position tracking control} - A61B 2034/2051: {Electromagnetic tracking systems} - A61B 2034/301: {for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes} - A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image} - A61B 2090/378: {using ultrasound} - A61B 2562/0223: Magnetic field sensors - G02B 27/017: {Head mounted} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract A medical device-placing system includes an ultrasound probe, a medical-device detector, a console, and an alternative-reality headset. The ultrasound probe is configured to emit ultrasound signals into a patient's limb and receive echoed ultrasound signals from the patient's limb. The medical-device detector is configured for placement about the patient's limb. The console is configured to transform the echoed ultrasound signals to produce ultrasound-image segments corresponding to anatomical structures of the patient's limb, as well as transform magnetic-sensor signals from the medical-device detector into location information for a magnetized medical device within the patient's limb. The alternative-reality headset includes a display screen through which a wearer of the alternative-reality headset can see the patient's limb. The display screen is configured to display over the patient's limb a virtual medical device per the location information for the medical device within objects of virtual anatomy corresponding to the ultrasound-image segments. ## Claims ### Claim 1 A medical device-placing system, comprising: an ultrasound probe configured to emit ultrasound signals into a limb of a patient and receive echoed ultrasound signals from the patient's limb by way of a piezoelectric sensor array; an array of magnetic sensors embedded within a housing configured for placement about the patient's limb; a console having electronic circuitry including memory and a processor configured to: transform the echoed ultrasound signals to produce ultrasound-image segments corresponding to anatomical structures of the patient's limb, and transform magnetic-sensor signals from the array of magnetic sensors into location information for a magnetized medical device within the patient's limb when the housing is placed about the patient's limb; and an alternative-reality headset, including: a frame having electronic circuitry including memory and a processor; and a display screen coupled to the frame through which a wearer of the alternative-reality headset can see the patient's limb, the display screen configured to display over the patient's limb a virtual medical device in accordance with the location information for the medical device within objects of virtual anatomy corresponding to the ultrasound-image segments. ### Claim 2 (depends on 1) The medical device-placing system according to claim 1, wherein the ultrasound probe is configured with a pulsed-wave Doppler imaging mode for emitting and receiving the ultrasound signals, and wherein the console is configured to capture ultrasound-imaging frames in accordance with the pulsed-wave Doppler imaging mode, stitch the ultrasound-imaging frames together with a stitching algorithm, and segment the ultrasound-imaging frames or the stitched ultrasound-imaging frames into the ultrasound-image segments with an image segmentation algorithm. ### Claim 3 (depends on 1) The medical device-placing system according to claim 1, wherein the console is configured to transform the ultrasound-image segments into the objects of virtual anatomy with a virtualization algorithm and send both the virtual medical device and the objects of virtual anatomy to the alternative-reality headset for display over the patient's limb. ### Claim 4 (depends on 1) The medical device-placing system according to claim 1, wherein the alternative-reality headset is configured to anchor the virtual medical device and the objects of virtual anatomy to the patient's limb over which the virtual medical device and the objects of virtual anatomy are displayed. ### Claim 5 (depends on 1) The medical device-placing system according to claim 1, the alternative-reality headset further comprising one or more eye-tracking cameras coupled to the frame configured to capture eye movements of the wearer, the processor of the alternative-reality headset configured to process the eye movements with an eye-movement algorithm to identify a focus of the wearer for selecting or enhancing the objects of virtual anatomy, the virtual medical device, or both corresponding to the focus of the wearer. ### Claim 6 (depends on 1) The medical device-placing system according to claim 1, the alternative-reality headset further comprising one or more patient-facing cameras coupled to the frame configured to capture gestures of the wearer, the processor of the alternative-reality headset configured to process the gestures with a gesture-command algorithm to identify gesture-based commands issued by the wearer for execution thereof by the alternative-reality headset. ### Claim 7 (depends on 1) The medical device-placing system according to claim 1, the alternative-reality headset further comprising one or more microphones coupled to the frame configured to capture audio of the wearer, the processor of the alternative-reality headset configured to process the audio with an audio-command algorithm to identify audio-based commands issued by the wearer for execution thereof by the alternative-reality headset. ### Claim 8 (depends on 1) The medical device-placing system according to claim 1, wherein the housing is a rigid frame, each magnetic sensor of the array of magnetic sensors embedded within the frame having a fixed spatial relationship to another magnetic sensor available to the console for transforming the magnetic-sensor signals from the array of magnetic sensors into the location information for the medical device. ### Claim 9 (depends on 1) The medical device-placing system according to claim 1, wherein the housing is a drape, each magnetic sensor of the array of magnetic sensors embedded within the drape having a variable spatial relationship to another magnetic sensor depending upon how the drape is placed about the limb of the patient. ### Claim 10 (depends on 9) The medical device-placing system according to claim 9, further comprising a magnetic-field generator configured to generate a magnetic field, the console configured to determine the spatial relationship of each magnetic sensor of the array of magnetic sensors to another magnetic sensor from the magnetic-sensor signals produced by the array of magnetic sensors while in the presence of the generated magnetic field, thereby making the determined spatial relationship available to the console for subsequently transforming the magnetic-sensor signals from the array of magnetic sensors into the location information for the medical device. ### Claim 11 An anatomy-visualizing system, comprising: an ultrasound-imaging system, including: an ultrasound probe configured to emit ultrasound signals into a patient and receive echoed ultrasound signals from the patient by way of a piezoelectric sensor array, and a console having electronic circuitry including memory and a processor configured to transform the echoed ultrasound signals to produce ultrasound-image segments corresponding to anatomical structures of the patient; and an alternative-reality headset, including: a frame having electronic circuitry including memory and a processor; and a display screen coupled to the frame through which display screen a wearer of the alternative-reality headset can see the patient, the display screen configured to display objects of virtual anatomy over the patient corresponding to the ultrasound-image segments. ### Claim 12 (depends on 11) The anatomy-visualizing system according to claim 11, wherein the ultrasound probe is configured with a pulsed-wave Doppler imaging mode for emitting and receiving the ultrasound signals, and wherein the console is configured to capture ultrasound-imaging frames in accordance with the pulsed-wave Doppler imaging mode, stitch the ultrasound-imaging frames together with a stitching algorithm, and segment the ultrasound-imaging frames or the stitched ultrasound-imaging frames into the ultrasound-image segments with an image segmentation algorithm. ### Claim 13 (depends on 11) The anatomy-visualizing system according to claim 11, wherein the console is configured to transform the ultrasound-image segments into the objects of virtual anatomy with a virtualization algorithm and send the objects of virtual anatomy to the alternative-reality headset for display over the patient. ### Claim 14 (depends on 11) The anatomy-visualizing system according to claim 11, wherein the alternative-reality headset is configured to anchor the objects of virtual anatomy to the patient over which the objects of virtual anatomy are displayed. ### Claim 15 (depends on 11) The anatomy-visualizing system according to claim 11, the alternative-reality headset further comprising one or more eye-tracking cameras coupled to the frame configured to capture eye movements of the wearer, the processor of the alternative-reality headset configured to process the eye movements with an eye-movement algorithm to identify a focus of the wearer for selecting or enhancing the objects of virtual anatomy corresponding to the focus of the wearer. ### Claim 16 (depends on 11) The anatomy-visualizing system according to claim 11, the alternative-reality headset further comprising one or more patient-facing cameras coupled to the frame configured to capture gestures of the wearer, the processor of the alternative-reality headset configured to process the gestures with a gesture-command algorithm to identify gesture-based commands issued by the wearer for execution thereof by the alternative-reality headset. ### Claim 17 (depends on 11) The anatomy-visualizing system according to claim 11, the alternative-reality headset further comprising one or more microphones coupled to the frame configured to capture audio of the wearer, the processor of the alternative-reality headset configured to process the audio with an audio-command algorithm to identify audio-based commands issued by the wearer for execution thereof by the alternative-reality headset. ### Claim 18 A medical device-locating system, comprising: an array of magnetic sensors embedded within a housing configured for placement about a limb of a patient; and a console having electronic circuitry including memory and a processor configured to transform magnetic-sensor signals from the array of magnetic sensors into location information for a magnetized medical device within the limb of the patient when the housing is placed about the limb of the patient. ### Claim 19 (depends on 18) The medical device-locating system according to claim 18, wherein the housing is a rigid frame, each magnetic sensor of the array of magnetic sensors embedded within the frame having a fixed spatial relationship to another magnetic sensor available to the console for transforming the magnetic-sensor signals from the array of magnetic sensors into the location information for the medical device. ### Claim 20 (depends on 18) The medical device-locating system according to claim 18, wherein the housing is a drape, each magnetic sensor of the array of magnetic sensors embedded within the drape having a variable spatial relationship to another magnetic sensor depending upon how the drape is placed about the limb of the patient. ### Claim 21 (depends on 20) The medical device-locating system according to claim 20, further comprising a magnetic-field generator configured to generate a magnetic field, the console configured to determine the spatial relationship of each magnetic sensor of the array of magnetic sensors to another magnetic sensor from the magnetic-sensor signals produced by the array of magnetic sensors while in the presence of the generated magnetic field, thereby making the determined spatial relationship available to the console for subsequently transforming the magnetic-sensor signals from the array of magnetic sensors into the location information for the medical device. ### Claim 22 (depends on 18) The medical device-locating system according to claim 18, further comprising a display screen configured to depict the medical device within the limb of the patient in accordance with the location information for the medical device. ### Claim 23 (depends on 22) The medical device-locating system according to claim 22, wherein the display screen is a see-through display screen coupled to a frame of an alternative-reality headset configured to receive from the console a virtual object corresponding to the medical device for depicting the medical device within the limb of the patient in accordance with the location information for the medical device. ### Claim 24 A method of a medical device-placing system, comprising: emitting ultrasound signals into a limb of a patient and receiving echoed ultrasound signals from the patient's limb by way of a piezoelectric sensor array of an ultrasound probe; transforming the echoed ultrasound signals with a console having electronic circuitry including memory and a processor to produce ultrasound-image segments corresponding to anatomical structures of the patient's limb; transforming magnetic-sensor signals from an array of magnetic sensors embedded within a housing placed about the patient's limb with the console into location information for a magnetized medical device within the patient's limb; and displaying over the patient's limb on a see-through display screen of an alternative-reality headset having electronic circuitry including memory and a processor in a frame coupled to the display screen a virtual medical device in accordance with the location information for the medical device within objects of virtual anatomy corresponding to the ultrasound-image segments. ### Claim 25 (depends on 24) The method according to claim 24, further comprising: capturing in the memory of the console ultrasound-imaging frames in accordance with a pulsed-wave Doppler imaging mode of the ultrasound probe while emitting and receiving the ultrasound signals; stitching the ultrasound-imaging frames together with a stitching algorithm; and segmenting the ultrasound-imaging frames or the stitched ultrasound-imaging frames into the ultrasound-image segments with an image segmentation algorithm. ### Claim 26 (depends on 24) The method according to claim 24, further comprising: transforming the ultrasound-image segments into the objects of virtual anatomy with a virtualization algorithm; and sending both the virtual medical device and the objects of virtual anatomy to the alternative-reality headset for display over the patient's limb. ### Claim 27 (depends on 26) The method according to claim 26, further comprising anchoring the virtual medical device and the objects of virtual anatomy to the patient's limb over which the virtual medical device and the objects of virtual anatomy are displayed. ### Claim 28 (depends on 24) The method according to claim 24, further comprising: capturing in the memory of the console eye movements of the wearer using one or more eye-tracking cameras coupled to the frame of the alternative-reality headset; and processing the eye movements with an eye-movement algorithm to identify a focus of the wearer for selecting or enhancing the objects of virtual anatomy corresponding to the focus of the wearer. ### Claim 29 (depends on 24) The method according to claim 24, further comprising: capturing in the memory of the console gestures of the wearer using one or more patient-facing cameras coupled to the frame of the alternative-reality headset; and processing the gestures with a gesture-command algorithm to identify gesture-based commands issued by the wearer for execution thereof by the alternative-reality headset. ### Claim 30 (depends on 24) The method according to claim 24, further comprising: capturing in the memory of the console audio of the wearer using one or more microphones coupled to the frame of the alternative-reality headset; and processing the audio with an audio-command algorithm to identify audio-based commands issued by the wearer for execution thereof by the alternative-reality headset. ### Claim 31 (depends on 24) The method according to claim 24, wherein each magnetic sensor of the array of magnetic sensors is embedded within a rigid frame for the housing, the magnetic sensors having a fixed spatial relationship to each other. ### Claim 32 (depends on 24) The method according to claim 24, wherein each magnetic sensor of the array of magnetic sensors is embedded within a drape for the housing, the magnetic sensors having a variable spatial relationship to each other depending upon how the drape is placed about the limb of the patient. ### Claim 33 (depends on 32) The method according to claim 32, further comprising: generating a magnetic field with a magnetic-field generator; and determining the spatial relationship of each magnetic sensor of the array of magnetic sensors to another magnetic sensor from the magnetic-sensor signals produced by the array of magnetic sensors while in the presence of the generated magnetic field.