# US 2014/0152698 A1 · Method for operating augmented reality contents and device and system for supporting the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0152698 A1
- Document ID: US20140152698A1
- Status: Published application
- Kind: A1
- Application number: 14/095,100
- Filed: 2013-12-03
- Published: 2014-06-05
- Earliest claimed: 2012-12-03
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Eunsun KIM; Hwayoung SONG; Yujin LEE; Hayoung JEONG
- Web page: https://patentgrep.com/patent/US20140152698A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06T 11/40: Filling planar surfaces by adding surface attributes, e.g. adding colours or textures
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Disclosed are methods of operating augmented reality (AR) contents, and a device and a system supporting the same. In one method, a real world image is captured using a camera in a portable device. A virtual space corresponding to the real world image is built, by partitioning image elements included in the virtual space into plural background objects and displaying the background objects. Augmented reality contents are generated by mapping one or more user contents onto the background objects. In other embodiments, location information of the portable device is used to obtain AR contents for enhancing captured images.

## Claims

### Claim 1

A method for operating augmented reality contents, the method comprising:
capturing a real world image using the camera;
building a virtual space corresponding to the real world image, by partitioning image elements included in the virtual space into plural background objects, and displaying the background objects; and
generating augmented reality contents by mapping one or more user contents onto the background objects.

### Claim 2 (depends on 1)

The method of claim 1, wherein displaying the background objects comprises one of:
identifying a number of predefined user contents to be included within the augmented reality contents, and partitioning the image elements into the background objects corresponding to the number of user contents; and
partitioning the image elements into background objects according to a number of user contents defined by default.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
receiving a request for increasing or decreasing a number of background elements; and
increasing or decreasing the number of background objects in the virtual space according to the request, re-partitioning the image elements into background objects.

### Claim 4 (depends on 1)

The method of claim 1, wherein displaying the background objects comprises:
identifying a number of predefined user contents to be included within the augmented reality contents and partitioning the image elements into background objects corresponding to the number of user contents; and
displaying allocation areas on which the user contents are to be placed within predetermined areas of the background objects.

### Claim 5 (depends on 4)

The method of claim 4, wherein displaying the background objects comprises setting sizes, positions, or inclinations of the allocation areas according to respective states of sizes, positions, and inclinations of the background elements.

### Claim 6 (depends on 1)

The method of claim 1, further comprising detecting selection of a placement scheme for placing the user contents on the virtual space in a predetermined form, wherein detecting selection comprises collecting at least one of a placement element for each of a plurality of depths in the virtual space and a placement element for each of a plurality of positions in the virtual space and at least one of a movement trace and movement speed of the device according to a motion of the device.

### Claim 7 (depends on 6)

The method of claim 6, wherein generating the augmented reality contents comprises:
partitioning the image elements into background objects in the virtual space based on the collected placement element and the movement trace and speed; and
mapping the user contents onto the background objects.

### Claim 8 (depends on 1)

The method of claim 1, wherein generating the augmented reality contents comprises:
displaying a list of the user contents; and
mapping one or more user contents included in the list onto at least one of the background objects designated by an input event.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
collecting current position information; and
inserting the current position information into the augmented reality contents and then storing the augmented reality contents.

### Claim 10 (depends on 9)

The method of claim 9, further comprising transmitting the augmented reality contents to an augmented reality providing server apparatus.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
distinguishing background objects and user contents in particular augmented reality contents and displaying the distinguished background objects and user contents on a display unit when an editing mode is selected; and
editing mapping information of the user contents and the background objects according to reception of an input event.

### Claim 12 (depends on 11)

The method of claim 11, further comprising performing resizing in displaying the augmented reality contents such that a whole screen of the augmented reality contents is displayed on the display unit.

### Claim 13 (depends on 1)

The method of claim 1, wherein the user contents include at least one of still image contents, video contents, and audio contents.

### Claim 14

A method for operating an augmented reality image, the method comprising:
capturing a surrounding image and current position information in association with a mode for using an augmented reality service;
detecting augmented reality contents corresponding to the current position information;
identifying an area corresponding to the captured surrounding image from the augmented reality contents; and
outputting user contents mapped onto the area corresponding to the captured surrounding image together with the captured surrounding image.

### Claim 15 (depends on 14)

The method of claim 14, further comprising at least one of:
generating augmented reality contents by mapping particular user contents onto one or more partitioned background objects based on a virtual space corresponding to a predetermined geographic area provided based on at least one image including the captured image and image elements of the virtual space, and collecting and storing position information of a electronic device; and
receiving the augmented reality contents from an augmented reality providing server apparatus.

### Claim 16 (depends on 14)

The method of claim 14, wherein the user contents include at least one of still image contents, video contents, and audio contents.

### Claim 17 (depends on 16)

The method of claim 16, wherein outputting the user contents comprises:
when the user contents are the still image contents, overlaying the still image contents with the captured surrounding image and displaying the overlaid still image contents;
when the user contents are the video contents, displaying the video contents together with the captured surrounding image, wherein a video is reproduced and displayed; and
when the user contents are the audio contents, displaying information indicating that the user contents are the audio contents together with the captured surrounding image, wherein an audio signal is also output.

### Claim 18 (depends on 17)

The method of claim 17, wherein outputting the user contents further comprises:
when the user contents are the video contents or the audio contents, receiving a separate input event or a motion sensor signal for reproducing the video or audio contents; and
reproducing the video contents or outputting an audio signal of the audio contents according to the input event or the motion sensor signal.

### Claim 19

A method of operating an augmented reality image, the method comprising:
capturing a surrounding image and current position information of an electronic device in association with a mode for using an augmented reality service;
transmitting the current position information and the captured surrounding image to an augmented reality providing server apparatus;
receiving user contents mapped onto the captured surrounding image in augmented reality contents corresponding to the current position information from the augmented reality providing server apparatus; and
outputting the received user contents together with the captured surrounding image, wherein the augmented reality contents are generated by mapping particular user contents onto one or more partitioned background objects based on a virtual space corresponding to the captured surrounding image.

### Claim 20 (depends on 19)

The method of claim 19, wherein outputting the received user contents comprises:
when the user contents are still image contents, overlaying the still image contents with the captured surrounding image and displaying the overlaid still image contents;
when the user contents are video contents, displaying the video contents together with the captured surrounding image, wherein a video is reproduced and displayed; and
when the user contents are audio contents, displaying information indicating that the user contents are the audio contents together with the captured surrounding image, wherein an audio signal is also output.

### Claim 21

A device supporting an operation of an augmented reality contents, the device comprising:
a camera unit configured to capture a real world image;
a display unit; and
a controller configured to build a virtual space corresponding to the captured image, by partitioning image elements included in the virtual space into plural background objects, and control to the display unit to display the background objects, and to generate augmented reality contents by mapping one or more user contents onto the background objects.

### Claim 22 (depends on 21)

The device of claim 21, further comprising:
a position information collection module is configured to collect image acquisition position information; and
a storage unit for storing augmented reality contents including the virtual space, information on the background objects onto which the user contents are mapped, and the position information.

### Claim 23 (depends on 22)

The device of claim 22, wherein the controller is configured to control for to distinguish user contents and background objects of augmented reality contents stored in the storage unit, and to display the distinguished user contents and background objects on the display unit when an editing function is selected.

### Claim 24 (depends on 22)

The device of claim 22, further comprising a communication unit is configured to form at least one of a communication channel to provide the augmented reality contents to an augmented reality providing server apparatus and a communication channel to receive augmented reality contents stored in the augmented reality providing server apparatus.

### Claim 25 (depends on 21)

The device of claim 21, wherein the display unit is configured to display one or more allocation areas on which the user contents to be mapped into the background objects are placed.

### Claim 26

A system for operating augmented reality contents, the system comprising:
a device for generating and providing augmented reality contents including one or more background objects provided based on a virtual space built based on the captured image and image elements included in the virtual space, one or more user contents mapped onto the background objects, and image acquisition position information; and
an augmented reality providing server apparatus for receiving the augmented reality contents from the device and storing the augmented reality contents.

### Claim 27 (depends on 26)

The system of claim 26, wherein the device searches for other augmented reality contents stored in the augmented reality providing server apparatus based on position information and receives the other augmented reality contents.