# US 2024/0197433 A1 · Real-time 3d anatomical mapping of the eye

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0197433 A1
- Document ID: US20240197433A1
- Status: Published application
- Kind: A1
- Application number: 18/387,494
- Filed: 2023-11-07
- Published: 2024-06-20
- Earliest claimed: 2022-12-20
- Assignees: Johnson & Johnson Surgical Vision, Inc.
- Applicants: Johnson & Johnson Surgical Vision, Inc.
- Inventors: Assaf Govari
- Web page: https://patentgrep.com/patent/US20240197433A1

## CPC

- A61B 90/37: {Surgical systems with images on a monitor during operation}
- A61B 3/10: Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B 90/00: Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/514: from specularities
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06T 7/55: from multiple images
- G06T 7/73: using feature-based methods
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- A61B 2090/371: {with simultaneous use of two cameras}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10056: Microscopic image
- G06T 2207/30041: Eye; Retina; Ophthalmic
- G06T 2210/41: Medical
- G06T 2210/56: Particle system, point based geometry or rendering
- G06T 2219/2016: Rotation, translation, scaling
- G06T 2219/2021: Shape modification
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A medical visualization apparatus includes two or more imaging devices and a processor. The two or more imaging devices are configured to acquire speckle images of an organ of a patient from multiple different directions. The processor is configured to (a) generate from the speckle images a three-dimensional (3D) point cloud of locations, (b) generate from the 3D point cloud of locations a 3D map of an anatomical surface of the organ, and (c) display the 3D map of the anatomical surface according to a requested gazing direction relative to the organ.

## Claims

### Claim 1

A medical visualization apparatus, comprising:
two or more imaging devices configured to acquire speckle images of an organ of a patient from multiple different directions; and
a processor, which is configured to:
generate from the speckle images a three-dimensional (3D) point cloud of locations;
generate from the 3D point cloud of locations a 3D map of an anatomical surface of the organ; and
display the 3D map of the anatomical surface according to a requested gazing direction relative to the organ.

### Claim 2 (depends on 1)

The apparatus according to claim 1, wherein the processor is configured to generate a video stream of the organ and overlay the video stream on the 3D map.

### Claim 3 (depends on 1)

The apparatus according to claim 1, wherein the processor is configured to generate the 3D map of the anatomical surface by performing at least the steps of:
estimating lateral coordinates of speckles in the speckle images;
performing depth analysis of the speckles to estimate depth coordinates of the speckles;
aggregating estimated spatial locations of the speckles into a 3D point cloud data set;
dividing the 3D point cloud data set into unit areas, and estimate respective orientations of the unit areas; and
using the orientations of the unit areas, reconstructing from the 3D point cloud a mesh approximation of a variably curved surface in 3D space.

### Claim 4 (depends on 3)

The apparatus according to claim 3, wherein the processor is further configured to graphically indicate different anatomical portions of the variably curved surface.

### Claim 5 (depends on 3)

The apparatus according to claim 3, wherein the mesh approximation is a triangular mesh.

### Claim 6 (depends on 3)

The apparatus according to claim 3, wherein edges of the mesh approximation are locations of a subset of the data points of the 3D point cloud.

### Claim 7 (depends on 1)

The apparatus according to claim 1, wherein the imaging devices are configured to acquire a stream of the speckle images at a video rate, and wherein the processor is configured to generate and display the 3D map at the video rate.

### Claim 8 (depends on 1)

The apparatus according to claim 1, wherein the processor is further configured to, in response to a request to change the gazing direction, display the 3D map of the anatomical surface as viewed from a new direction.

### Claim 9 (depends on 8)

The apparatus according to claim 8, wherein the processor is configured to respond to the request at the video rate.

### Claim 10 (depends on 1)

The apparatus according to claim 1, wherein the imaging devices comprise digital microscopes.

### Claim 11 (depends on 1)

The apparatus according to claim 1, wherein the organ is an eye.

### Claim 12

A medical visualization method, comprising:
acquiring speckle images of an organ of a patient from multiple different directions using two or more imaging devices;
generating from the speckle images a three-dimensional (3D) point cloud of locations;
generating from the 3D point cloud of locations a 3D map of an anatomical surface of the organ; and
displaying the 3D map of the anatomical surface according to a requested gazing direction relative to the organ.

### Claim 13 (depends on 12)

The method according to claim 12, and comprising generating a video stream of the organ and overlaying the video stream on the 3D map.

### Claim 14 (depends on 12)

The method according to claim 12, wherein generating the 3D map of the anatomical surface comprises performing at least the steps of:
estimating lateral coordinates of speckles in the speckle images;
performing depth analysis of the speckles to estimate depth coordinates of the speckles;
aggregating estimated spatial locations of the speckles into a 3D point cloud data set;
dividing the 3D point cloud data set into unit areas, and estimate respective orientations of the unit areas; and
using the orientations of the unit areas, reconstructing from the 3D point cloud a mesh approximation of a variably curved surface in 3D space.

### Claim 15 (depends on 14)

The method according to claim 14, and comprising graphically indicating different anatomical portions of the variably curved surface.

### Claim 16 (depends on 14)

The method according to claim 14, wherein the mesh approximation is a triangular mesh.

### Claim 17 (depends on 14)

The method according to claim 14, wherein edges of the mesh approximation are locations of a subset of the data points of the 3D point cloud.

### Claim 18 (depends on 12)

The method according to claim 12, wherein the imaging devices are configured to acquire a stream of the speckle images at a video rate, and wherein the processor is configured to generate and display the 3D map at the video rate.

### Claim 19 (depends on 12)

The method according to claim 12, and comprising, in response to a request to change the gazing direction, displaying the 3D map of the anatomical surface as viewed from a new direction.

### Claim 20 (depends on 19)

The method according to claim 19, and comprising responding to the request at the video rate.

### Claim 21 (depends on 12)

The method according to claim 12, wherein the imaging devices comprise digital microscopes.

### Claim 22 (depends on 12)

The method according to claim 12, wherein the organ is an eye.
