# US 12,118,683 B2 · Content creation in augmented reality environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,683 B2
- Document ID: US12118683B2
- Status: Granted
- Kind: B2
- Application number: 18/313,188
- Filed: 2023-05-05
- Published: 2023-08-31
- Granted: 2024-10-15
- Earliest claimed: 2019-08-22
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Varun Nigam; Veenu Pandey
- Web page: https://patentgrep.com/patent/US12118683B2

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/03545: {Pens or stylus}
- G06F 3/0383: {Signal control means within the pointing device}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 21/10: Protecting distributed programs or content, e.g. vending or licensing of copyrighted material (protection in video systems or pay television H04N7/16) {; Digital rights management [DRM]}
- G06F 21/44: Program or device authentication
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods for creating content in augmented reality (AR) environment. The content is created by a combination of at least a handheld device and an AR device, and involves initiation and establishment of a session between the handheld device and the AR device, command and data generation at the handheld device, transmission of the command and data from the handheld device to the AR device, processing of the command and data at the AR device to build context from the command and/or data and to create content from the context, saving and sharing of the created content, retrieval of the created content, and optionally modification of the retrieved content.

## Claims

### Claim 1

An operation method of an electronic device, the method comprising:
establishing a communication link with an external electronic device;
receiving a command related to an object from the external electronic device via the communication link, wherein the command includes information related to a selection of the object among a plurality of objects located in a real world within a field of view of a camera of the electronic device;
recognizing the object indicated to be selected by the command, which is located in the real world within the field of view of the camera of the electronic device, based on the received command;
determining a context related to the recognized object, wherein determining the context includes:
capturing at least one parameter selected from a global context, a local context, and one or more other objects located in the real world within the field of view of the camera of the electronic device when recognizing the object;
identifying a profile of a content creator;
identifying the context of the object based on a combination of the command, data related to the recognized object received from the external electronic device via the communication link, the at least one captured parameter and the identified profile of the content creator, wherein the data is generated based on movement of the external electronic device in three-dimensional (3D) space relative to the recognized object;
rendering an action list and a menu for generating an augmented reality (AR) content based on the identified context;
analyzing the rendered action list and the menu; and
building the context based on the analyzed rendered action list, the menu, and the identified context;
generating the AR content based on the determined context and the data; and
displaying the generated AR content in a display of the electronic device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
locking the recognized object to establish a session.

### Claim 3 (depends on 2)

The method of claim 2, wherein locking the recognized object includes:
locking the recognized object upon receiving the command related to the object from the external electronic device.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
the global context includes data belonging to the content creator stored on a server, and
the local context is selected from one or more of date and time of recognizing the object, geolocation of the object and physical conditions of the object.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating the AR content includes:
generating a subsequent context-aware action list and a subsequent context-aware menu, based on the built context and the data; and
rendering the subsequent context-aware action-list and the subsequent context-aware menu for generating the AR content.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
authenticating a content recipient;
authenticating at least one condition set by a content creator for delivery of a content generated by the content creator;
notifying, by the electronic device to the content recipient, generation of the AR content by the content creator upon successful authentication of the at least one condition;
retrieving the generated AR content; and
rendering, for the content recipient, the generated AR content.

### Claim 7 (depends on 6)

The method of claim 6, wherein the at least one condition includes at least one of location of the object, physical conditions of the object, or confirming a presence of other real world objects in a vicinity of the object.

### Claim 8 (depends on 1)

The method of claim 1, wherein:
the external electronic device includes a stylus or a pen.

### Claim 9

An electronic device comprising:
a display;
a camera;
a communication circuit; and
a processor configured to be operatively connected to the display and the communication circuit,
wherein the processor is configured to:
establish a communication link with an external electronic device via the communication circuit;
receive a command related to an object from the external electronic device via the communication link, wherein the command includes information related to a selection of the object among a plurality of objects located in a real world within a field of view of the camera;
recognize the object indicated to be selected by the command, which is located in the real world within the field of view of the camera based on the received command;
determine a context related to the recognized object;
capture at least one parameter selected from a global context, a local context, and one or more other objects located in the real world within the field of view of the camera when recognizing the object;
identify a profile of a content creator;
identify the context of the object based on a combination of the command, data related to the recognized object received from the external electronic device via the communication link, the at least one captured parameter and the identified profile of the content creator, wherein the data is generated based on movement of the external electronic device in three-dimensional (3D) space relative to the recognized object;
render an action list and a menu for generating an augmented reality (AR) content based on the identified context;
analyze the rendered action list and the menu; and
build the context based on the analyzed rendered action list, menu and the identified context;
generate the AR content based on the determined context and the data; and
display the generated AR content via the display.

### Claim 10 (depends on 9)

The electronic device of claim 9, wherein the processor is further configured to:
lock the recognized object to establish a session.

### Claim 11 (depends on 10)

The electronic device of claim 10, wherein to lock the recognized object to establish the session, the processor is further configured to:
lock the recognized object upon receiving the command related to the object from the external electronic device.

### Claim 12 (depends on 9)

The electronic device of claim 9, wherein:
the global context includes data belonging to the content creator stored on a server, and
the local context is selected from one or more of, date and time of recognizing the object, geolocation of the object and physical conditions of the object.

### Claim 13 (depends on 9)

The electronic device of claim 9, wherein the processor is further configured to:
generate a subsequent context-aware action list and a subsequent context-aware menu, based on the built context and the data; and
render the subsequent context-aware action-list and the subsequent context-aware menu for generating the AR content.

### Claim 14 (depends on 9)

The electronic device of claim 9, wherein the processor is further configured to:
authenticate a content recipient;
authenticate at least one condition set by a content creator for delivery of a content generated by the content creator;
notify, by the electronic device to the content recipient, generation of the AR content by the content creator upon successful authentication of the at least one condition;
retrieve the generated AR content; and
render, for the content recipient, the generated AR content.

### Claim 15 (depends on 14)

The electronic device of claim 14, wherein the at least one condition includes at least one of location of the object, physical conditions of the object, or confirming a presence of other real world objects in a vicinity of the object.

### Claim 16 (depends on 9)

The electronic device of claim 9, wherein:
the external electronic device includes a stylus or a pen.