# US 12,112,444 B2 · Electronic device and graphic object control method of electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,444 B2
- Document ID: US12112444B2
- Status: Granted
- Kind: B2
- Application number: 17/426,538
- Filed: 2020-01-22
- Published: 2022-04-21
- Granted: 2024-10-08
- Earliest claimed: 2019-01-28
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Yejin Kim; Yoonjeong Kang; Iseul Yu; Heeyul Kim; Sanghyun Park; Jongil Jeong
- Web page: https://patentgrep.com/patent/US12112444B2

## CPC

- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 15/205: {Image-based rendering}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 20/30: in albums, collections or shared content, e.g. social network photos or video
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/169: {Holistic features and representations, i.e. based on the facial image taken as a whole}
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/30201: Face
- G06T 2210/16: Cloth
- G06T 2219/2008: Assembling, disassembling
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V 40/16: Human faces, e.g. facial parts, sketches or expressions
- G06V 40/172: {Classification, e.g. identification}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An electronic device includes a camera, a display, and a processor, wherein the processor is configured to acquire an image using the camera, determine a 3D graphic object corresponding to an object included in the acquired image, and apply the determined 3D graphic object to a 3D avatar and display the same.

## Claims

### Claim 1

An electronic device comprising:
a camera;
a display;
a processor; and
memory storing instructions that, when executed by the processor, cause the electronic device to:
obtain an image using the camera;
analyze an object comprised in the obtained image to detect category information and attribute information about the object to identify a type of the object;
determine a two-dimensional (2D) icon indicating the type of the object corresponding to the category information;
determine a first three-dimensional (3D) graphic object corresponding to the attribute information;
simultaneously display the 2D icon indicating the type of the object and the obtained image;
apply the first 3D graphic object to a 3D avatar to display the 3D avatar;
while the 3D avatar is being displayed on the display, obtain the image using the camera when a personalization item generation is selected with the displayed 3D avatar:
store the first 3D graphic object without the 3D avatar, as a personalization item of the 3D avatar;
analyze context information that can be identified in connection with a plurality of applications of the electronic device;
determine, as a recommended item, a second 3D graphic object related to the context information among personalization items of the 3D avatar;
display the second 3D graphic object determined as the recommended item:
apply the second 3D graphic object determined as the recommended item to the 3D avatar to display the 3D avatar; and
change and display a background screen of the 3D avatar based on the context information.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed by the processor, cause the electronic device to:
detect a 2D graphic image from an image comprising a body part when obtaining the image using the camera; and
apply the 2D graphic image to a portion of the 3D avatar corresponding to the body part to display the 3D avatar.

### Claim 3 (depends on 2)

The electronic device of claim 2, wherein the instructions, when executed by the processor, cause the electronic device to:
display at least one recommended graphic image related to the 2D graphic image; and
apply a recommended graphic image selected by a user to the portion of the 3D avatar corresponding to the body part to display the 3D avatar.

### Claim 4 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed by the processor, cause the electronic device to:
recognize a face when obtaining the image using the camera, wherein the image comprises a whole body;
determine the 3D avatar corresponding to the recognized face;
determine a plurality of objects in the image and position information about each of the plurality of objects;
determine a plurality of 3D graphic objects corresponding to the plurality of objects; and
apply the plurality of 3D graphic objects to a position of the 3D avatar corresponding to the position information about each of the plurality of objects to display the 3D avatar.

### Claim 5 (depends on 4)

The electronic device of claim 4, wherein the instructions, when executed by the processor, cause the electronic device to store the plurality of 3D graphic objects as the personalization items of the 3D avatar.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein the processor is further configured to:
identify the object comprised in the obtained image based on a selection by a user.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the first 3D graphic object applied to the 3D avatar is displayed at a location of the 3D avatar that is selected by a user.

### Claim 8

A graphic object control method of an electronic device, the method comprising:
obtaining an image using a camera;
analyzing an object comprised in the obtained image to detect category information and attribute information about the object to identify a type of the object;
determining a two-dimensional (2D) icon indicating the type of the object corresponding to the category information;
determining a first three-dimensional (3D) graphic object corresponding to the attribute information;
simultaneously displaying the 2D icon indicating the type of the object and the obtained image;
applying the determined 3D graphic object to a 3D avatar to display the 3D avatar;
while the 3D avatar is being displayed on the display, obtaining the image using the camera when a personalization item generation is selected with the displayed 3D avatar:
storing the 3D graphic object without the 3D avatar, as a personalization item of the 3D avatar;
analyzing context information that can be identified in connection with a plurality of applications of the electronic device;
determining, as a recommended item, a second 3D graphic object related to the context information among personalization items of the 3D avatar;
displaying the second 3D graphic object determined as the recommended item;
applying the second 3D graphic object determined as the recommended item to the 3D avatar to display the 3D avatar; and
changing and displaying a background screen of the 3D avatar based on the context information.

### Claim 9 (depends on 8)

The graphic object control method of claim 8, further comprising:
detecting a (2D) graphic image from an image comprising a body part when obtaining the image using the camera; and
applying the 2D graphic image to a portion of the 3D avatar corresponding to the body part to display the 3D avatar.

### Claim 10 (depends on 9)

The graphic object control method of claim 9, further comprising:
displaying at least one recommended graphic image related to the 2D graphic image; and
applying a recommended graphic image selected by a user to the portion of the 3D avatar corresponding to the body part to display the 3D avatar.

### Claim 11 (depends on 8)

The graphic object control method of claim 8, further comprising:
recognizing a face when obtaining the image using the camera, wherein the image comprises a whole body;
determining the 3D avatar corresponding to the recognized face;
determining a plurality of objects in the image and position information about each of the plurality of objects;
determining a plurality of 3D graphic objects corresponding to the plurality of objects;
applying the plurality of 3D graphic objects to a position of the 3D avatar corresponding to the position information about each of the plurality of objects to display the 3D avatar; and
storing the plurality of 3D graphic objects as the personalization items of the 3D avatar.

### Claim 12 (depends on 8)

The graphic object control method of claim 8, further comprising:
storing the plurality of 3D graphic objects as the personalization items of the 3D avatar.

### Claim 13 (depends on 8)

The graphic object control method of claim 8, further comprising:
identifying the object comprised in the obtained image based on a selection by a user.

### Claim 14 (depends on 8)

The graphic object control method of claim 8, wherein the first 3D graphic object applied to the 3D avatar is displayed at a location of the 3D avatar that is selected by a user.