# US 2018/0322674 A1 · Real-time AR Content Management and Intelligent Data Analysis System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322674 A1
- Document ID: US20180322674A1
- Status: Published application
- Kind: A1
- Application number: 15/973,509
- Filed: 2018-05-07
- Published: 2018-11-08
- Earliest claimed: 2017-05-06
- Applicants: Integem, Inc.
- Inventors: Eliza Y. Du
- Web page: https://patentgrep.com/patent/US20180322674A1

## CPC

- G06V 10/764: using classification, e.g. of video objects
- G06F 16/434: {using image data, e.g. images, photos, pictures taken by a user}
- G06F 18/24: Classification techniques
- G06F 21/10: Protecting distributed programs or content, e.g. vending or licensing of copyrighted material (protection in video systems or pay television H04N7/16) {; Digital rights management [DRM]}
- G06F 21/6218: {to a system of files or objects, e.g. local or distributed file system or database}
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Described is a real-time content management and data analytics system for AR-based platforms. The data management system described herein manages AR content and user interactions with the AR content. Additionally, a new multiple-stage information augmentation design based on real-time data analysis and live AR interaction is described. In this design, AR content design is very flexible and may be organized into one or more stages containing pre-defined content, on-line searched content, user generated content, other user generated content, real-time user interactively generated content, or some combination. The flexibility content structure brings allows for a high customizable AR experience for each user to maximize user relevance and stickiness. An example application of the system described herein is for interactive advertisements.

## Claims

### Claim 1

A method for managing content comprising:
receiving augmented reality (AR) source content, the AR source content having one or more stages;
storing at least one stage of AR source content in a content database;
analyzing at least one content item in the content database to extract a content selection criteria; and
providing at least one content item from the content database, the provided content items selected using the content selection criteria, wherein the provided content items are incorporated into the stages of AR source content.

### Claim 2 (depends on 1)

The method of claim 1, wherein the content items stored in the content database comprise AR source content, AR user content, or AR combined content, the AR combined content comprising AR source content and AR user content.

### Claim 3 (depends on 1)

The method of claim 1, wherein a stage in the AR source content comprises;
graphics content including 2D images, 2D animation images, 2D video images, 2D/3D images, or 2D/3D animations;
audio content including music, voice, or sound effects; and
interactive control information including AR process controls programmed to occur in response to physical actions of one or more users, wherein the AR process controls are pre-defined, real-time learned, or some combination.

### Claim 4 (depends on 1)

The method of claim 1, wherein different AR user content is generated for different user interactions with the same AR source content.

### Claim 5 (depends on 1)

The method of claim 1, wherein the stages of the AR source content are independently modifiable so the stages can be updated without effecting the operation of any of other stages of the AR source content.

### Claim 6 (depends on 1)

The method of claim 1, wherein the content items incorporated into the stages of AR source content comprises AR combined content comprising one or more stages of user generated AR source content created from unique user interactions with the AR source content.

### Claim 7 (depends on 6)

The method of claim 6, further comprising: receiving user generated AR source content and updating one or more stages of AR source content with the new user generated AR source content.

### Claim 8 (depends on 7)

The method of claim 7, wherein the user generated AR source content is generated and incorporated into one or more stages of AR source content to provide live user interaction with an AR environment.

### Claim 9 (depends on 1)

The method of claim 1, wherein the content selection criteria comprises user preferences, user information, user identification, user behavior, user bio-information, user AR interaction, user facial expressions, user physical movements, user gaze direction, user gaze duration, user connected device information, user connected device interaction, user dress, user social media data, user recent purchase history, user social-economic status analytics, pedestrian counting, aggregated data analytics, people counting, location information analytics, or other analytics information.

### Claim 10 (depends on 1)

The method of claim 1, further comprising restricting access to at least one of the AR source content or the AR user content according to at least one authentication criteria.

### Claim 11 (depends on 1)

The method of claim 1, wherein the authentication criteria comprises: admin access information, user access information, user sharing preferences, user privacy preferences, organization access information, organization sharing preferences, organization privacy preferences, online access information, or offline access information.

### Claim 12

A content management system comprising;
a database portion for storing at least one of AR source content or AR user content, the AR source content having one or more stages;
an application portion comprising:
a data processing pipeline for ingesting raw data, the raw data comprising sensor data and metadata related to at least one of AR source content or AR user content;
data analytics logic for generating classification data from processing raw data, the classification data including one or more classifiers associated with a particular item of AR source content or AR user content;
content management logic for classifying one or more stages of AR source content using the classification data; and
an access control portion for restricting access to AR source content stored in the content database according to the classifiers a user has permission to access the access layer prevents unauthenticated devices from accessing the stored AR source content;

### Claim 13 (depends on 12)

The method of claim 12, wherein stages of AR source content, can be updated independently from other stages by incorporating at least one of AR source content, AR user content, the AR combined content contains user generated AR content created from live user interactions with AR source content.

### Claim 14 (depends on 12)

The method of claim 12, wherein the content management logic provides at least one of classification data, one or more stages of AR source content, or AR user content to a user.

### Claim 15 (depends on 13)

The method of claim 13, wherein the stages of AR source content or AR user content are incorporated into AR source content displayed on a user device to provide live interactions with an AR environment.

### Claim 16

An apparatus for content management comprising:
memory for storing a content management application and data access control module, the content management application programmed to manage AR content;
processors for controlling analysis and transmission of raw data, the raw data including measurement data from one or more sensors and metadata including in AR content, analysis controlled by the processors comprises extracting classification data from at least one of the measurement data or the metadata, the classification data including one or more classifiers associated with an item of AR content; and
a database that interfaces with the content management application and data access control module, the database comprising:
a content library portion for storing a plurality of content items;
a classification portion for storing the classification data, the classification data associated with particular content items,
the database organized to restrict access to the content items in the content library according to instructions provided by the data access control module, the instructions including one or more classifiers associated with particular content items a user has permission to access.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the content library includes one or more AR content items, the AR content items having one or more stages.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the database is organized to provide one or more stages of the AR content items based on classification data, the classification data including one or more classifiers associated with the stages of AR content.

### Claim 19 (depends on 18)

The apparatus of claim 18, wherein the classifiers associated with the stages of AR content include user preferences, user information, user identification, user behavior, user bio-information, user AR interaction, user facial expressions, user physical movements, user gaze direction, user gaze duration, user connected device information, user connected device interaction, user dress, user social media data, user recent purchase history, user social-economic status analytics, pedestrian counting, aggregated data analytics, people counting, location information analytics, or other classification information.

### Claim 20 (depends on 17)

The apparatus of claim 17, wherein the AR content comprises at least one of AR source content, AR user content, AR combined content, or user generated AR content.