# US 2020/0005543 A1 · Apparatus and method for calibrating augmented-reality image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0005543 A1
- Document ID: US20200005543A1
- Status: Published application
- Kind: A1
- Application number: 16/447,847
- Filed: 2019-06-20
- Published: 2020-01-02
- Earliest claimed: 2018-07-02
- Assignees: Electronics and Telecommunications Research Institute
- Applicants: Electronics and Telecommunications Research Institute
- Inventors: Jae-Hean KIM; Bon-Ki KOO; Chang-Joon PARK
- Web page: https://patentgrep.com/patent/US20200005543A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 7/73: using feature-based methods
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Disclosed herein are an apparatus and method for calibrating an augmented-reality image. The apparatus includes a camera unit for capturing an image and measuring 3D information pertaining to the image, an augmented-reality image calibration unit for generating an augmented-reality image using the image and the 3D information and for calibrating the augmented-reality image, and a display unit for displaying the augmented-reality image.

## Claims

### Claim 1

An apparatus for calibrating an augmented-reality image, comprising:
a camera unit for capturing an image and measuring 3D information pertaining to the image;
an augmented-reality image calibration unit for generating an augmented-reality image using the image and the 3D information and calibrating the augmented-reality image; and
a display unit for displaying the augmented-reality image.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the camera unit selects a feature point in the image and extracts 3D coordinates of the feature point.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the augmented-reality image calibration unit generates a patch, which is a predetermined range surrounding the feature point extracted from the image.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the augmented-reality image calibration unit generates the patch, which includes a marker line for indicating a position of the feature point.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the augmented-reality image calibration unit displays the image on which the patch is overlaid at an arbitrary position through the display unit.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the augmented-reality image calibration unit fixes the patch at the arbitrary position and performs a matching process for generating calibration information including 2D coordinates of the patch and 3D coordinates of the feature point when a part of the image overlaps a same part in the patch depending on movement of a user.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the augmented-reality image calibration unit generates multiple patches and performs the matching process for the multiple patches, thereby generating the calibration information.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the augmented-reality image calibration unit calculates a calibration relationship between the 2D coordinates of the patch and the 3D coordinates of the feature point using the calibration information.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the augmented-reality image calibration unit calculates the calibration relationship using the calibration information, a first relationship between a reference coordinate system of the camera unit and a reference coordinate system of the user, and a second relationship between a reference coordinate system of the display unit and the reference coordinate system of the user.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the augmented-reality image calibration unit transforms 3D coordinates of a virtual object in a 3D space into the 2D coordinates using the calibration relationship and displays the augmented-reality image including the virtual object through the display unit.

### Claim 11

A method for calibrating an augmented-reality image, performed by an apparatus for calibrating the augmented-reality image, comprising:
capturing, by a camera unit, an image, and measuring, by the camera unit, 3D information pertaining to the image;
generating, by an augmented-reality image calibration unit, an augmented-reality image using the image and the 3D information, and calibrating, by the augmented-reality image calibration unit, the augmented-reality image; and
displaying, by a display unit, the augmented-reality image.

### Claim 12 (depends on 11)

The method of claim 11, wherein measuring the 3D information is configured to select a feature point in the image and extract 3D coordinates of the feature point.

### Claim 13 (depends on 12)

The method of claim 12, wherein calibrating the augmented-reality image is configured to generate a patch that is a predetermined range surrounding the feature point cut from the image.

### Claim 14 (depends on 13)

The method of claim 13, wherein calibrating the augmented-reality image is configured to generate the patch that includes a marker line for indicating a position of the feature point.

### Claim 15 (depends on 14)

The method of claim 14, wherein calibrating the augmented-reality image is configured to display the image on which the patch is overlaid at an arbitrary position through the display unit.

### Claim 16 (depends on 15)

The method of claim 15, wherein calibrating the augmented-reality image is configured to fix the patch at the arbitrary position and perform a matching process for generating calibration information including 2D coordinates of the patch and 3D coordinates of the feature point when a part of the image overlaps a same part in the patch depending on movement of a user.

### Claim 17 (depends on 16)

The method of claim 16, wherein calibrating the augmented-reality image is configured to generate multiple patches and perform the matching process for the multiple patches, thereby generating the calibration information.

### Claim 18 (depends on 17)

The method of claim 17, wherein calibrating the augmented-reality image is configured to calculate a calibration relationship between the 2D coordinates of the patch and the 3D coordinates of the feature point using the calibration information.

### Claim 19 (depends on 18)

The method of claim 18, wherein calibrating the augmented-reality image is configured to calculate the calibration relationship using the calibration information, a first relationship between a reference coordinate system of the camera unit and a reference coordinate system of the user, and a second relationship between a reference coordinate system of the display unit and the reference coordinate system of the user.

### Claim 20 (depends on 19)

The method of claim 19, wherein calibrating the augmented-reality image is configured to transform 3D coordinates of a virtual object in a 3D space into the 2D coordinates using the calibration relationship and to display the augmented-reality image including the virtual object through the display unit.
