# US 10,311,094 B2 · Synthetic audiovisual description scheme, method and system for MPEG-7

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,094 B2
- Document ID: US10311094B2
- Status: Granted
- Kind: B2
- Application number: 14/531,542
- Filed: 2014-11-03
- Published: 2015-02-26
- Granted: 2019-06-04
- Earliest claimed: 1999-02-01
- Assignees: AT&T Intellectual Property II, L.P.
- Applicants: AT&T Intellectual Property II, L.P.
- Inventors: Qian Huang; Joern Ostermann; Atul Puri; Raj Kumar Rajendran
- Web page: https://patentgrep.com/patent/US10311094B2

## CPC

- G06F 16/48: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F 16/40: of multimedia data, e.g. slideshows comprising image and additional audio data (retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70)
- G06F 16/71: Indexing; Data structures therefor; Storage structures
- G06F 16/951: Indexing; Web crawling techniques
- H04N 1/00209: {Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and system for description of synthetic audiovisual content makes it easier for humans, software components or devices to identify, manage, categorize, search, browse and retrieve such content. For instance, a user may wish to search for specific synthetic audiovisual objects in digital libraries, Internet web sites or broadcast media; such a search is enabled by the invention. Key characteristics of synthetic audiovisual content itself such as the underlying 2d or 3d models and parameters for animation of these models are used to describe it. To represent features of synthetic audiovisual content, depending on the description scheme to be used, a number of descriptors are selected and assigned values. The description scheme instantiated with descriptor values is used to generate the description, which is then stored for actual use during query/search.

## Claims

### Claim 1

A method comprising:
receiving audiovisual data having a natural feature and a synthetic feature;
extracting and representing the synthetic feature of the audiovisual data having a defined animation parameter as a descriptor, wherein the descriptor is selected according to a synthetic audiovisual description scheme that specifies a structure and semantics of relationships between components of the audiovisual data which include descriptors;
assigning, via a processor, a value to the descriptor based on the synthetic feature; and
generating a description based on the value of the descriptor, wherein the description comprises spatial and temporal relationships of audiovisual objects in a scene.

### Claim 2 (depends on 1)

The method of claim 1, wherein the components comprise other constituent description schemes.

### Claim 3 (depends on 2)

The method of claim 2, wherein the other constituent description schemes comprise an animation event description scheme corresponding to dynamic characteristics of the audiovisual data.

### Claim 4 (depends on 2)

The method of claim 2, wherein the constituent description schemes comprise an animation object description scheme corresponding to static characteristics of the audiovisual data in the scene.

### Claim 5 (depends on 2)

The method of claim 2, wherein the constituent description schemes comprise an animation event and a relational description scheme.

### Claim 6 (depends on 1)

The method of claim 1, wherein the synthetic feature is a model feature used to generate the audiovisual data.

### Claim 7 (depends on 6)

The method of claim 6, wherein the model feature comprises one of a two-dimensional model and a three-dimensional model.

### Claim 8 (depends on 1)

The method of claim 1, wherein each audiovisual object in the audiovisual objects is associated with synthetic features.

### Claim 9 (depends on 8)

The method of claim 8, wherein the synthetic features of an audiovisual object are grouped together as being one of visual, audio, and a relationship of media.

### Claim 10 (depends on 1)

The method of claim 1, wherein each audiovisual object in the audiovisual objects is associated with an animation stream.

### Claim 11 (depends on 1)

The method of claim 1, wherein the value is an instantiation of the descriptor corresponding to the audiovisual data.

### Claim 12 (depends on 1)

The method of claim 1, wherein the representing, the assigning and the generating are performed using extensible markup language.

### Claim 13 (depends on 1)

The method of claim 1, wherein the synthetic audiovisual description scheme comprises a document type definition that provides a list of example descriptors used during the representing.

### Claim 14 (depends on 13)

The method of claim 13, wherein the document type definition specifies a set of rules for a structure of the audiovisual data.

### Claim 15 (depends on 1)

The method of claim 1, wherein the description comprises the value and is associated with the synthetic audiovisual description scheme.

### Claim 16 (depends on 15)

The method of claim 15, wherein the synthetic audiovisual description scheme is incorporated in the description.

### Claim 17 (depends on 16)

The method of claim 16, wherein the synthetic audiovisual description scheme is stored at a remote location and the description is linked to the synthetic audiovisual description scheme.

### Claim 18 (depends on 17)

The method of claim 17, wherein a device at the remote location is accessible over a network.

### Claim 19

A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, result in operations comprising:
receiving audiovisual data having a natural feature and a synthetic feature;
extracting and representing the synthetic feature of the audiovisual data having a defined animation parameter as a descriptor, wherein the descriptor is selected according to a synthetic audiovisual description scheme that specifies a structure and semantics of relationships between components of the audiovisual data which include descriptors;
assigning a value to the descriptor based on the synthetic feature; and
generating a description based on the value of the descriptor, wherein the description comprises spatial and temporal relationships of audiovisual objects in a scene.

### Claim 20

A computer-readable storage device having instructions stored which, when executed by a computing device, result in operations comprising:
receiving audiovisual data having a natural feature and a synthetic feature;
extracting and representing the synthetic feature of the audiovisual data having a defined animation parameter as a descriptor, wherein the descriptor is selected according to a synthetic audiovisual description scheme that specifies a structure and semantics of relationships between components of the audiovisual data which include descriptors;
assigning a value to the descriptor based on the synthetic feature; and
generating a description based on the value of the descriptor, wherein the description comprises spatial and temporal relationships of audiovisual objects in a scene.