# US 2023/0410451 A1 · Augmented reality implement apparatus and method using mobile scanned object model scaling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410451 A1
- Document ID: US20230410451A1
- Status: Published application
- Kind: A1
- Application number: 18/248,026
- Filed: 2021-10-05
- Published: 2023-12-21
- Earliest claimed: 2020-10-05
- Inventors: Jun Pyo HONG
- Web page: https://patentgrep.com/patent/US20230410451A1

## CPC

- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04N 13/361: Reproducing mixed stereoscopic images; Reproducing mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Disclosed are a method and device for realizing augmented reality via mobile scan object model scaling. According to one embodiment of the present invention, an augmented reality realization method performed in an augmented reality realization device via mobile scan object modeling may comprise the steps of: capturing a background image by means of an image-capturing unit; displaying the background image in real time via a display unit, and creating, by means of an AR drive unit, an AR space corresponding to the background image according to an augmented reality start command from a user; loading an object model selected by the user onto the AR space; matching a model coordinate system of the object model with an AR coordinate system of the AR space; scaling the object model according to a result of the matching; and overlaying the scaled object model onto the AR space, and displaying same.

## Claims

### Claim 1

A method of implementing an augmented reality performed on an apparatus of implementing the augmented reality using a mobile scanned object model, comprising:
capturing a background image by an image capturing unit;
when the background image is displayed on a display in real-time, generating an augmented reality space corresponding to the background image by an AR running unit according to an augmented reality start command from a user;
loading an object model selected by the user within the augmented reality space;
matching an augmented reality coordinate system for the augmented reality space with a model coordinate system for the object model;
scaling the object model according to the matching result; and
displaying the scaled object model overlappingly within the augmented reality space.

### Claim 2 (depends on 1)

The method of claim 1, wherein the matching the augmented reality coordinate system with the model coordinate system comprises:
estimating a plurality of first camera locations in the augmented reality coordinate system;
estimating a plurality of second camera locations corresponding to the plurality of first camera locations in the model coordinate system; and
calculating a scale factor for the model coordinate system to match the plurality of first camera locations with the plurality of second camera locations,
wherein the object model is scaled by applying the scale factor in the scaling the object model.

### Claim 3 (depends on 2)

The method of claim 2, wherein the calculating the scale factor is calculating a ratio of a first unit length between the plurality of first cameral locations and a second unit length between the plurality of second cameral locations as the scale factor.

### Claim 4 (depends on 2)

The method of claim 2, wherein the calculating the scale factor comprises:
calculating a ratio of a length between two among the plurality of first cameral locations and a length between two among the plurality of second cameral locations as partial scale factors; and
analyzing a numerical distribution of the partial scale factors and setting an average of a cluster having the largest distribution as the scale factor.

### Claim 5 (depends on 2)

The method of claim 2, wherein the first camera location is estimated by SLAM (Simultaneous Localization and Mapping).

### Claim 6 (depends on 1)

The method of claim 1, wherein the object model that is a mobile scanned object model being scanned by a camera provided in a mobile device is a 3D model that is generated by a photogrammetry with a plurality of 2D images captured by the camera.

### Claim 7

An apparatus of implementing an augmented reality using a mobile scanned object model, comprising:
an image capturing unit, configured for capturing an image that is a background to implement the augmented reality;
a display unit, configured for displaying the image captured by the image capturing unit on a screen in real-time;
an augmented reality running unit, configured for running the image as an augmented reality space according to a user input and loading an object module selected by a user within the augmented reality space; and
a controller, configured for controlling operation of the image capturing unit, the display, and the augmented reality running unit,
wherein the object model is a mobile scanned object model that is scanned by a camera provided in a mobile device.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the augmented reality running unit comprises:
an augmented reality core unit, configured for managing an augmented reality coordinate system for the augmented reality space; and
a coordinate matching unit, configured for matching a model coordinate system that is included in the object model with the augmented reality coordinate system to scale the object model to correspond to the augmented reality space.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the coordinate matching unit estimates a plurality of first camera locations in the augmented reality coordinate system, estimates a plurality of second camera locations corresponding to the plurality of first camera locations in the model coordinate system, calculates a scale factor for the model coordinate system to match the plurality of first camera locations with the plurality of second camera locations, and then scales the object model by applying the scale factor to the object model.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the coordinate matching unit calculates a ratio of a first unit length between the plurality of first cameral locations and a second unit length between the plurality of second cameral locations as the scale factor.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein the coordinate matching unit calculates a ratio of a length between two among the plurality of first cameral locations and a length between two among the plurality of second cameral locations as partial scale factors, analyzes a numerical distribution of the partial scale factors, and sets an average of a cluster having the largest distribution as the scale factor.

### Claim 12 (depends on 9)

The apparatus of claim 9, wherein the first camera location is estimated by SLAM (Simultaneous Localization and Mapping).
