# US 2019/0172250 A1 · Method and apparatus for estimating parameter of virtual screen

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0172250 A1
- Document ID: US20190172250A1
- Status: Published application
- Kind: A1
- Application number: 16/103,416
- Filed: 2018-08-14
- Published: 2019-06-06
- Earliest claimed: 2017-12-06
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Dong Kyung NAM; Hyoseok HWANG
- Web page: https://patentgrep.com/patent/US20190172250A1

## CPC

- G06T 15/20: Perspective computation
- G06T 3/60: Rotation of whole images or parts thereof
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/74: {involving reference images or patches}
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 2207/30208: Marker matrix
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method and an apparatus for estimating a parameter of a virtual screen are disclosed. The method includes determining first transformation parameters of virtual cameras relative to a reference camera of the virtual cameras based on transformation parameters of the virtual cameras relative to a physical pattern, determining second transformation parameters of the virtual cameras relative to the reference camera based on transformation parameters of the virtual cameras relative to a virtual pattern, and estimating a size parameter of a virtual screen based on the first transformation parameters and the second transformation parameters.

## Claims

### Claim 1

A method of estimating a parameter of a virtual screen, the method comprising:
acquiring, from a physical camera, a plurality of captured images that represent a reflector reflecting a virtual pattern displayed on a virtual screen and a physical pattern displayed on a physical plane;
estimating, for each reflection angle of the reflector, a respective one from among a plurality of virtual cameras that correspond to the physical camera;
determining, for each respective one from among the plurality of virtual cameras, at least one first transformation parameter relative to a reference camera of the plurality of virtual cameras based on physical pattern transformation parameters of the plurality of virtual cameras that relate to the physical pattern;
determining, for each respective one from among the plurality of virtual cameras, at least one second transformation parameter relative to the reference camera based on virtual pattern transformation parameters of the plurality of virtual cameras that relate to the virtual pattern; and
estimating a size parameter of the virtual screen based on each of the at least one first transformation parameter and each of the at least one second transformation parameter.

### Claim 2 (depends on 1)

The method of claim 1, wherein the estimating the size parameter comprises estimating a value that corresponds to a minimum difference between a respective one from among the at least one first transformation parameter and a corresponding one from among the at least one second transformation parameter.

### Claim 3 (depends on 1)

The method of claim 1, wherein the estimating the size parameter comprises:
determining each of the at least one second transformation parameter based on a candidate size parameter;
determining a respective difference between each respective one from among the at least first transformation parameter and each corresponding one from among the at least one second transformation parameter determined based on the candidate size parameter;
comparing each determined respective difference to a threshold; and
estimating the candidate size parameter to be the size parameter based on a result of the comparing.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
estimating a third transformation parameter of the virtual screen relative to the physical camera based on the estimated size parameter.

### Claim 5 (depends on 4)

The method of claim 4, wherein the estimating the third transformation parameter of the virtual screen comprises:
estimating geometric relationships between the virtual pattern and the reflector based on the plurality of captured images;
calculating a projection error between at least one virtual image of the virtual pattern in the plurality of captured images and the virtual pattern based on the geometric relationships; and
estimating the third transformation parameter of the virtual screen relative to the physical camera such that the projection error is minimized.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
estimating a third transformation parameter of the virtual screen relative to the physical camera based on at least one fourth transformation parameter of the plurality of virtual cameras relative to the virtual screen and a fifth transformation parameter of the physical camera relative to the plurality of virtual cameras.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
estimating the fifth transformation parameter of the physical camera relative to the plurality of virtual cameras based on the plurality of captured images.

### Claim 8 (depends on 7)

The method of claim 7, wherein the estimating the fifth transformation parameter of the physical camera relative to the plurality of virtual cameras comprises:
estimating geometric relationships between the physical pattern and the reflector based on the plurality of captured images;
calculating a projection error between at least one virtual image of the physical pattern in the plurality of captured images and the physical pattern based on the geometric relationships; and
estimating the fifth transformation parameter of the physical camera relative to the plurality of virtual cameras such that the projection error is minimized.

### Claim 9 (depends on 6)

The method of claim 6, further comprising:
estimating the at least one fourth transformation parameter of the plurality of virtual cameras relative to the virtual screen based on each of the at least one first transformation parameter and each of the at least one second transformation parameter.

### Claim 10 (depends on 1)

The method of claim 1, wherein each of the plurality of captured images is acquired by adjusting a reflection angle of the reflector.

### Claim 11

A method of estimating a parameter of a virtual screen, the method comprising:
acquiring, from a physical camera, a plurality of captured images that represent a reflector that includes a physical pattern and that reflects a virtual pattern displayed on a virtual screen;
estimating, for each reflection angle of the reflector, a respective one from among a plurality of virtual cameras that correspond to the physical camera;
determining, for each respective one from among the plurality of virtual cameras, at least one first transformation parameter relative to a reference camera of the plurality of virtual cameras based on physical pattern transformation parameters of the plurality of virtual cameras that relate to the physical pattern;
determining, for each respective one from among the plurality of virtual cameras, at least one second transformation parameter relative to the reference camera based on virtual pattern transformation parameters of the plurality of virtual cameras that relate to the virtual pattern; and
estimating a size parameter of the virtual screen based on each of the at least one first transformation parameter and each of the at least one second transformation parameter.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
estimating a third transformation parameter of the virtual screen relative to the physical camera based on at least one fourth transformation parameter of the plurality of virtual cameras relative to the virtual screen and a fifth transformation parameter of the physical camera relative to the plurality of virtual cameras.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
estimating the fifth transformation parameter of the physical camera relative to the plurality of virtual cameras based on a geometric relationship between the physical camera and the reflector.

### Claim 14 (depends on 12)

The method of claim 12, further comprising:
estimating the at least one fourth transformation parameter of the plurality of virtual cameras relative to the virtual screen based on each of the at least one first transformation parameter and each of the at least one second transformation parameter.

### Claim 15 (depends on 1)

A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1.

### Claim 16

An apparatus for estimating a parameter of a virtual screen, the apparatus comprising:
a processor; and
a memory that includes an instruction to be read by the processor,
wherein the processor is configured to execute the instruction to:
acquire, from a physical camera, a plurality of captured images that represent a reflector reflecting a virtual pattern displayed on a virtual screen and a physical pattern displayed on a physical plane;
estimate, for each reflection angle of the reflector, a respective one from among a plurality of virtual cameras that correspond to the physical camera;
determine, for each respective one from among the plurality of virtual cameras, at least one first transformation parameter relative to a reference camera of the plurality of virtual cameras based on physical pattern transformation parameters of the plurality of virtual cameras that relate to the physical pattern;
determine, for each respective one from among the plurality of virtual cameras, at least one second transformation parameter relative to the reference camera based on virtual pattern transformation parameters of the plurality of virtual cameras that relate to the virtual pattern; and
estimate a size parameter of the virtual screen based on each of the at least one first transformation parameter and each of the at least one second transformation parameter.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the processor is further configured to estimate a value that corresponds to a minimum difference between a respective one from among the at least one first transformation parameter and a corresponding one from among the at least one second transformation parameter.

### Claim 18 (depends on 16)

The apparatus of claim 16, wherein the processor is further configured to estimate a third transformation parameter of the virtual screen relative to the physical camera based on the estimated size parameter.

### Claim 19 (depends on 16)

The apparatus of claim 16, wherein the processor is further configured to estimate a third transformation parameter of the virtual screen relative to the physical camera based on at least one fourth transformation parameter of the plurality of virtual cameras relative to the virtual screen and a fifth transformation parameter of the physical camera relative to the plurality of virtual cameras.

### Claim 20 (depends on 16)

The apparatus of claim 16, wherein each of the plurality of captured images is acquired by adjusting a reflection angle of the reflector.