# US 2024/0187560 A1 · Three-dimensional annotation rendering system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0187560 A1
- Document ID: US20240187560A1
- Status: Published application
- Kind: A1
- Application number: 18/258,459
- Filed: 2021-12-14
- Published: 2024-06-06
- Earliest claimed: 2020-12-25
- Inventors: Tokio GOTO
- Web page: https://patentgrep.com/patent/US20240187560A1

## CPC

- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- G02B 30/25: using polarisation techniques
- G06T 7/593: from stereo images
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T)
- G09G 5/36: characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- G09G 5/377: for mixing or overlaying two or more graphic patterns (G09G5/02, G09G5/397 take precedence)
- H04N 5/64: Constructional details of receivers, e.g. cabinets or dust covers (furniture aspects {of television cabinets} A47B81/06)
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N 13/156: Mixing image signals
- H04N 13/293: Generating mixed stereoscopic images; Generating mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
- H04N 13/361: Reproducing mixed stereoscopic images; Reproducing mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A three-dimensional annotation rendering system includes a calculation device receiving signals captured by right-eye and left-eye cameras, a background image generation unit generating right-eye and left-eye background images, a pointer depth position information generation unit, a pointer longitudinal and lateral position information generation unit, an annotation start/end information generation unit, an annotation-related information storage unit storing depth position information and longitudinal and lateral position information on the pointer from a recording start to a recording end on the annotation, a pointer image generation unit generating right-eye and left-eye pointer images, an annotation-related image generation unit generating right-eye and left-eye annotation-related images, and a background annotation image synthesis unit combining the right-eye and left-eye background images, pointer images, and annotation-related images to generate final right-eye and left-eye images.

## Claims

### Claim 1

A three-dimensional annotation rendering system implemented by a calculation device,
the calculation device comprising:
a camera image reception unit configured to receive a right-eye imaging signal of a subject imaged by a right-eye camera and a left-eye imaging signal of the subject imaged by a left-eye camera;
a background image generation unit configured to generate a right-eye background image on a basis of the right-eye imaging signal and a left-eye background image on a basis of the left-eye imaging signal;
a pointer longitudinal and lateral position information generation unit configured to generate longitudinal and lateral position information on a pointer, on a basis of an operation signal transmitted from an annotation input device for operating the pointer;
a pointer depth position information generation unit configured to generate depth position information on the pointer;
an annotation start/end information generation unit configured to generate recording start information and recording end information on an annotation, on a basis of the operation signal;
an annotation-related information storage unit configured to store depth position information on the pointer during a period from time of generating the recording start information to time of generating the recording end information as the depth position information on the annotation and to store the longitudinal and lateral position information on the pointer during the period from the time of generating the recording start information to the time of generating the recording end information as the longitudinal and lateral position information on the annotation;
a pointer image generation unit configured to generate a right-eye pointer image and a left-eye pointer image by referring to at least the longitudinal and lateral position information on the pointer;
an annotation-related image generation unit configured to generate a right-eye annotation-related image and a left-eye annotation-related image by referring to the depth position information on the annotation and the longitudinal and lateral position information on the annotation; and
a background annotation image synthesis unit configured to combine the right-eye background image, the right-eye pointer image, and the right-eye annotation-related image to generate a right-eye final image, and to combine the left-eye background image, the left-eye pointer image, and the left-eye annotation-related image to generate a left-eye final image.

### Claim 2 (depends on 1)

The three-dimensional annotation rendering system according to claim 1, wherein the calculation device further comprises a left and right image synthesis unit configured to generate a three-dimensional final image by superimposing the right-eye final image and the left-eye final image over each other.

### Claim 3 (depends on 1)

The three-dimensional annotation rendering system according to claim 1, wherein the pointer depth position information generation unit generates the depth position information on the pointer on a basis of the operation signal signifying depth movement that is transmitted from the annotation input device.

### Claim 4 (depends on 1)

The three-dimensional annotation rendering system according to claim 1, wherein:
the calculation device comprises a subject depth position information calculation unit configured to calculate depth position information on the subject on a basis of the right-eye background image and the left-eye background image; and
the pointer depth position information generation unit generates the depth position information on the pointer on a basis of the depth position information on the subject corresponding to the longitudinal and lateral position information on the pointer.

### Claim 5 (depends on 1)

The three-dimensional annotation rendering system according to claim 1, wherein the pointer image generation unit generates the right-eye pointer image and the left-eye pointer image including a parallax based on the depth position information on the pointer.
