# US 2026/0224197 A1 · Devices and methods for freehand multimodality imaging

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0224197 A1
- Document ID: US20260224197A1
- Status: Published application
- Kind: A1
- Application number: 19/150,269
- Filed: 2024-01-23
- Published: 2026-08-06
- Earliest claimed: 2023-01-23
- Assignees: Trustees of Tufts College
- Applicants: Trustees of Tufts College
- Inventors: Srivalleesha Mallidi; Deeksha Sankepalle
- Web page: https://patentgrep.com/patent/US20260224197A1

## CPC

- A61B 8/4254: {using sensors mounted on the probe}
- A61B 5/0062: {Arrangements for scanning}
- A61B 5/0095: {by applying light and detecting acoustic waves, i.e. photoacoustic measurements}
- A61B 8/4444: {related to the probe}
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/89: for mapping or imaging
- A61B 8/08: Clinical applications (A61B8/02, A61B8/04, A61B8/06 take precedence)
- A61B 8/4218: {characterised by articulated arms}
- A61B 8/4245: {involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient}
- A61B 8/4263: {using sensors not mounted on the probe, e.g. mounted on an external reference frame}
- A61B 8/4455: {Features of the external shape of the probe, e.g. ergonomic aspects}
- A61B 8/4488: {the transducer being a phased array}
- A61B 8/483: {involving the acquisition of a 3D volume of data}
- A61B 8/5215: {involving processing of medical diagnostic data}
- A61B 8/54: {Control of the diagnostic device}
- A61B 2034/2048: {using an accelerometer or inertia sensor}
- A61B 2034/2055: {Optical tracking systems}
- G01S 15/8936: {using transducers mounted for mechanical movement in three dimensions}
- G01S 15/899: {Combination of imaging systems with ancillary equipment}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

An apparatus and method for integrated ultrasound and photoacoustic imaging (USPAI) and visual odometry (VO) three-dimensional (3D) USPA imaging reconstruction is described. The apparatus is configured for freehand scanning. A processor is configured to synchronize the imaging scans acquired by the USPA probe and odometer data from the at least one optical sensor and inertial measurement unit (IMU) of the visual odometer, whereby the data from their respective components is timestamped. Alignment and smoothing steps enable accurate 3D reconstruction of the two-dimensional UPSA imaging scans from the linear translational motion of the USPA probe.

## Claims

### Claim 1

An apparatus for three-dimensional (3D) image reconstruction, the apparatus comprising:
a housing configured for freehand movement within a 3D reference frame defined by an XW-axis, YW-axis, and ZW-axis, the housing comprising:
a laser source configured to transmit an electromagnetic (EM) wave into a sample to produce a photoacoustic effect therein;
an ultrasound probe configured to generate scans of the sample;
a visual odometer configured to track movement of the ultrasound probe in the 3D reference frame to generate odometer data; and
a processor in communication with the ultrasound probe and the visual odometer, configured to:
receive the scans and odometer data, wherein the scans and odometer data each include timestamps,
synchronize the scans and odometer data, and
construct a 3D image of the scans based on the synchronization.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the ultrasound probe includes one or more one-dimensional (1D) or two-dimensional (2D) array transducers.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the visual odometer includes at least one optical sensor and an inertial measurement unit (IMU).

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the at least one optical sensor includes a visible light camera, infrared camera, or light detecting and ranging (LiDAR) camera.

### Claim 5 (depends on 3)

The apparatus of claim 3, wherein the at least one optical sensor includes a fisheye camera.

### Claim 6 (depends on 3)

The apparatus of claim 3, wherein the optical sensor is configured to be directed toward the sample or away from the sample.

### Claim 7 (depends on 3)

The apparatus of claim 3, wherein the IMU includes a tri-axial gyroscope, a tri-axial accelerometer, and optionally a tri-axial magnetometer.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the processor is further configured to fuse data acquired by the optical sensor, gyroscope, accelerometer, and optional magnetometer into the output data including a six degrees of freedom (6 DOF) pose and an orientation of the optical sensor.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the processor is further configured to identify a motion of a feature in the scans.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the feature includes a speckle pattern.

### Claim 11 (depends on 8)

The apparatus of claim 8, wherein the processor is further configured to trim the odometer data to match the scans.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the processor is further configured to apply a smoothing filter to the output data to generate smooth data with reduced noise.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the processor is further configured to divide the smoothed output data by a spatial resolution of the ultrasound probe in each of the x-axis, y-axis, and z-axis to determine a pixel shift.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the processor is further configured to up-sample the scans to match a frame rate of the optical sensor.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the processor is further configured to spatially align the scans in a 3D space based on a transformation, wherein the transformation includes the pixel shift.

### Claim 16

A method for three-dimensional (3D) image reconstruction, the method comprising:
receiving, using a processor, photoacoustic ultrasound scans from an ultrasound probe within a housing configured for freehand movement within a 3D reference frame defined by an XW-axis, YW-axis, and ZW-axis;
receiving, using the processor, odometer data from a visual odometer within the housing configured to track movement of the ultrasound probe in the 3D reference frame;
synchronizing the scans and the odometer data based on timestamps associated with each of the scans and the odometer data;
constructing a 3D image of the scans based on the synchronization.

### Claim 17 (depends on 16)

The method of claim 16, wherein the ultrasound probe includes one or more one-dimensional (1D) or two-dimensional (2D) array transducers.

### Claim 18 (depends on 16)

The method of claim 16, wherein the visual odometer includes at least one optical sensor and an inertial measurement unit (IMU).

### Claim 19 (depends on 18)

The method of claim 18, wherein the at least one optical sensor includes a visible light camera, infrared camera, or light detecting and ranging (LiDAR) camera.

### Claim 20 (depends on 18)

The method of claim 18, wherein the at least one optical sensor includes a fisheye camera.

### Claim 21 (depends on 18)

The method of claim 18, wherein the optical sensor is configured to be directed toward the sample or away from the sample.

### Claim 22 (depends on 18)

The method of claim 18, wherein the IMU includes a tri-axial gyroscope, a tri-axial accelerometer, and optionally a tri-axial magnetometer.

### Claim 23 (depends on 22)

The method of claim 22, further comprising, using the processor, fusing data acquired by the optical sensor, gyroscope, accelerometer, and optional magnetometer into the output data including a six degrees of freedom (6 DOF) pose and an orientation of the optical sensor.

### Claim 24 (depends on 23)

The method of claim 23, further comprising, using the processor, identifying a translational motion of a feature in the scans.

### Claim 25 (depends on 24)

The method of claim 24, wherein the feature includes a speckle pattern.

### Claim 26 (depends on 23)

The method of claim 23, further comprising, using the processor, trimming the odometer data to match the scans.

### Claim 27 (depends on 26)

The method of claim 26, further comprising, using the processor, applying a smoothing filter to the odometer data to generate smooth data with reduced noise.

### Claim 28 (depends on 27)

The method of claim 27, further comprising, using the processor, dividing the smoothed odometer data by a spatial resolution of the ultrasound probe in each of the x-axis, y-axis, and z-axis to determine a pixel shift.

### Claim 29 (depends on 28)

The method of claim 28, further comprising, using the processor, up-sampling the scans to match a frame rate of the optical sensor.

### Claim 30 (depends on 29)

The method of claim 29, further comprising, using the processor, spatially aligning the scans in a 3D space based on a transformation, wherein the transformation includes the pixel shift.
