# US 2026/0073644 A1 · Motion matching device and method for heterogeneous 3d models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073644 A1
- Document ID: US20260073644A1
- Status: Published application
- Kind: A1
- Application number: 18/924,251
- Filed: 2024-10-23
- Published: 2026-03-12
- Earliest claimed: 2024-09-06
- Assignees: Korea Electronics Technology Institute
- Applicants: Korea Electronics Technology Institute
- Inventors: Jin Young LEE; Sang Shin LEE; Won Gi CHOI; Ju Kyung YOO; Ankhzaya BAATARBILEG; Yeonu JUNG
- Web page: https://patentgrep.com/patent/US20260073644A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 7/251: {involving models}
- G06T 7/75: {involving models}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/30196: Human being; Person
- G06T 2210/32: Image data format
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Proposed is a motion matching method. The method may include collecting real-time images taken in real time and non-real-time images taken previously and acquiring real-time motion data on a first object contained in the real-time images and non-real-time motion data on a second object contained in the non-real-time images. The method may also include operating a real-time 3D model by applying the real-time motion data to the real-time 3D model and operating a non-real-time 3D model by applying the non-real-time motion data to the non-real-time 3D model, and setting a start point by finding a pose of the non-real-time 3D model that matches a pose of the real-time 3D model. The method may further include matching motions of the real-time 3D model and the non-real-time 3D model by operating the real-time and non-real-time 3D models based on the start point.

## Claims

### Claim 1

A method for motion matching of heterogeneous three-dimensional (3D) models, performed by a motion matching device, the method comprising:
collecting real-time images taken in real time and non-real-time images taken previously;
acquiring real-time motion data on a first object contained in the real-time images and non-real-time motion data on a second object contained in the non-real-time images;
operating a real-time 3D model by applying the real-time motion data to the real-time 3D model and operating a non-real-time 3D model by applying the non-real-time motion data to the non-real-time 3D model;
setting a start point by finding a pose of the non-real-time 3D model that matches a pose of the real-time 3D model; and
matching motions of the real-time 3D model and the non-real-time 3D model by operating the real-time 3D model and the non-real-time 3D model based on the start point.

### Claim 2 (depends on 1)

The method of claim 1, wherein the real-time images comprise real-time streaming images taken with a stereo camera, and the non-real-time images comprise a previously taken video.

### Claim 3 (depends on 2)

The method of claim 2, wherein upon acquiring the real-time motion data, the motion matching device:
acquires real-time motion capture data by performing a real-time motion capture for the first object contained in the real-time streaming images; and
acquires the real-time motion data by converting the real-time motion capture data into a biovision hierarchical data (BVH) format.

### Claim 4 (depends on 3)

The method of claim 3, wherein upon acquiring the non-real-time motion data, the motion matching device:
converts a camera coordinate system of the previously taken video from a local coordinate system to a world coordinate system;
acquires non-real-time motion capture data by performing a non-real-time motion capture for the second object contained in the previously taken video in the world coordinate system; and
acquires the non-real-time motion data by converting the non-real-time motion capture data into a BVH format.

### Claim 5 (depends on 4)

The method of claim 4, wherein upon acquiring the non-real-time motion capture data, the motion matching device performs the non-real-time motion capture through pose estimation for the second object contained in the previously taken video.

### Claim 6 (depends on 1)

The method of claim 1, wherein upon setting the start point, the motion matching device sets the start point by finding the non-real-time motion data that matches the real-time motion data corresponding to a pose of the real-time 3D model.

### Claim 7 (depends on 6)

The method of claim 6, wherein upon setting the start point, the motion matching device finds a pose of the non-real-time 3D model that matches a specific pose within a given time from a time point at which the real-time 3D model starts motion.

### Claim 8 (depends on 7)

The method of claim 7, wherein upon setting the start point, the motion matching device:
extracts a reference pose from motions of the real-time 3D model within the given time;
extracts a matching pose that matches the reference pose from motions of the non-real-time 3D model; and
sets, as the start point, a time point when the reference pose of the real-time 3D model and the matching pose of the non-real-time 3D model are taken.

### Claim 9 (depends on 8)

The method of claim 8, wherein upon extracting the matching pose, in order to correct a deviation caused by a size difference between the real-time 3D model and the non-real-time 3D model, the motion matching device corrects a vector of the reference pose with a correction value according to the size difference between the real-time 3D model and the non-real-time 3D model, and then extracts a vector of the matching pose that matches the corrected vector of the reference pose.

### Claim 10

A motion matching device of heterogeneous three-dimensional (3D) models, the device comprising:
an image collecting processor configured to collect real-time images taken in real time and non-real-time images taken previously;
a motion data acquiring processor configured to acquire real-time motion data on a first object contained in the real-time images and non-real-time motion data on a second object contained in the non-real-time images;
a 3D model operating processor configured to operate a real-time 3D model by applying the real-time motion data to the real-time 3D model and operates a non-real-time 3D model by applying the non-real-time motion data to the non-real-time 3D model;
a start point setting processor configured to set a start point by finding a pose of the non-real-time 3D model that matches a pose of the real-time 3D model; and
a motion matching processor configured to match motions of the real-time 3D model and the non-real-time 3D model by operating the real-time 3D model and the non-real-time 3D model based on the start point.

### Claim 11

A motion matching system of heterogeneous three-dimensional (3D) models, the system comprising:
a real-time image providing device configured to provide real-time images taken in real time;
a non-real-time image providing device configured to provide non-real-time images that have already been taken; and
a motion matching device configured to match motions of a real-time 3D model operating based on the real-time images and a non-real-time 3D model operating based on the non-real-time images, and expresses the matched motions in a virtual world,
the motion matching device including:
an image collecting processor configured to collect the real-time images and the non-real-time images;
a motion data acquiring processor configured to acquire real-time motion data on a first object contained in the real-time images and non-real-time motion data on a second object contained in the non-real-time images;
a 3D model operating processor configured to operate a real-time 3D model with the real-time motion data applied and operates a non-real-time 3D model with the non-real-time motion data applied;
a start point setting processor configured to set a start point by finding a pose of the non-real-time 3D model that matches a pose of the real-time 3D model; and
a motion matching processor configured to match motions of the real-time 3D model and the non-real-time 3D model by operating the real-time 3D model and the non-real-time 3D model based on the start point.
