# US 2013/0254794 A1 · Simulation method and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0254794 A1
- Document ID: US20130254794A1
- Status: Published application
- Kind: A1
- Application number: 13/901,700
- Filed: 2013-05-24
- Published: 2013-09-26
- Earliest claimed: 2008-06-12
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Sara H. Basson; Dimitri Kanevsky; Edward E. Kelley; Bhuvana Ramabhadran
- Web page: https://patentgrep.com/patent/US20130254794A1

## CPC

- G10L 25/63: for estimating an emotional state
- G10L 15/25: using position of the lips, movement of the lips or face analysis
- G10L 21/00: Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence)
- G10L 21/06: Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids (G10L15/26 takes precedence)
- G11B 27/028: with computer assistance
- G11B 27/034: on discs (G11B27/036, G11B27/038 take precedence)
- G11B 27/036: Insert-editing
- G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)}
- G16H 50/50: for simulation or modelling of medical disorders
- H04N 21/44213: {Monitoring of end-user related data (arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33)}
- G10L 2021/0135: {Voice conversion or morphing}
- G10L 2021/065: {Aids for the handicapped in understanding}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A simulation method and system. A computing system receives a first audio and/or video data stream. The first audio and/or video data stream includes data associated with a first person. The computing system monitors the first audio and/or video data stream. The computing system identifies emotional attributes comprised by the first audio and/or video data stream. The computing system generates a second audio and/or video data stream associated with the first audio and/or video data stream. The second audio and/or video data stream includes the data without the emotional attributes. The computing system stores the second audio and/or video data stream.

## Claims

### Claim 1

A simulation method, comprising:
receiving, by a computing system, a first audio/video data stream, wherein said first audio/video data stream comprises first speech data associated with a first person;
identifying, by said computing system, first emotional attributes comprised by said first audio/video data stream;
generating, by said computing system in response to said identifying said first emotional attributes, an audio data stream associated with said first audio/video data stream, wherein said audio data stream comprises said first speech data, and wherein said audio data stream does not comprise said first emotional attributes;
associating, by said computing system, a first audible portion of said audio data stream with a first attribute of said first emotional attributes;
generating, by said computing system, an audible label for said first audible portion of said audio data stream, wherein said audible label indicates said first attribute;
applying, by said computing system, said audible label to said first audible portion of said audio data stream;
generating, by said computing system in response to said applying, a second audio data stream associated with said audio data stream, wherein said second audio data stream comprises said audio data stream and said first audible portion of said audio data stream comprising said audible label;
generating, by said computing system, a video data stream associated with said audio/video data stream, wherein said video data stream comprises video data associated with said first person, and wherein said video data does not comprise said first emotional attributes;
associating, by said computing system, a first visual object of said video data with said first attribute of said first emotional attributes;
generating, by said computing system, a viewable label for said first visual object, wherein said viewable label indicates said first attribute of said first emotional attributes;
applying, by said computing system, said viewable label to said first visual object; and
generating, by said computing system in response to said applying, a second audio/video data stream associated with said second video data stream and said second audio data stream, wherein said second audio/video data stream comprises said audio data stream, said first audible portion of said audio data stream comprising said audible label, said second video data, and said first visual object comprising said viewable label.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
broadcasting, by said computing system, said second audio/video data stream.

### Claim 3 (depends on 1)

The method of claim 1, wherein said first emotional attributes comprise audible and visual vocal expressions of said first person.

### Claim 4 (depends on 1)

The method of claim 1, wherein said viewable label comprises a label selected from the group consisting of a polygon label comprising a perimeter for surrounding said first visual object and a label comprising different color applied to said first visual object.

### Claim 5 (depends on 1)

The method of claim 1, wherein said first emotional attributes comprise facial expressions of said first person.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable program code in the mobile device, said program code being executed by the computer processor to implement said receiving, said identifying, said generating said audio data stream, said associating said first audible portion, said generating said audible label, said applying said audible label to said first audible portion of said audio data stream, said generating, said second audio data stream, said generating said video data stream, said associating said first visual object, said generating said viewable label, said applying said viewable label to said first visual object, and said generating said second audio/video data stream.

### Claim 7

A computer program product, comprising a computer storage device storing a computer readable program code, said computer readable program code configured to perform a method upon being executed by a processor of a computing system, said method comprising:
receiving, by said computing system, a first audio/video data stream, wherein said first audio/video data stream comprises first speech data associated with a first person;
identifying, by said computing system, first emotional attributes comprised by said first audio/video data stream;
generating, by said computing system in response to said identifying said first emotional attributes, an audio data stream associated with said first audio/video data stream, wherein said audio data stream comprises said first speech data, and wherein said audio data stream does not comprise said first emotional attributes;
associating, by said computing system, a first audible portion of said audio data stream with a first attribute of said first emotional attributes;
generating, by said computing system, an audible label for said first audible portion of said audio data stream, wherein said audible label indicates said first attribute;
applying, by said computing system, said audible label to said first audible portion of said audio data stream;
generating, by said computing system in response to said applying, a second audio data stream associated with said audio data stream, wherein said second audio data stream comprises said audio data stream and said first audible portion of said audio data stream comprising said audible label;
generating, by said computing system, a video data stream associated with said audio/video data stream, wherein said video data stream comprises video data associated with said first person, and wherein said video data does not comprise said first emotional attributes;
associating, by said computing system, a first visual object of said video data with said first attribute of said first emotional attributes;
generating, by said computing system, a viewable label for said first visual object, wherein said viewable label indicates said first attribute of said first emotional attributes;
applying, by said computing system, said viewable label to said first visual object; and
generating, by said computing system in response to said applying, a second audio/video data stream associated with said second video data stream and said second audio data stream, wherein said second audio/video data stream comprises said audio data stream, said first audible portion of said audio data stream comprising said audible label, said second video data, and said first visual object comprising said viewable label.

### Claim 8 (depends on 7)

The computer program product of claim 7, wherein said method further comprises:
broadcasting, by said computing system, said second audio/video data stream.

### Claim 9 (depends on 7)

The computer program product of claim 7, wherein said first emotional attributes comprise audible and visual vocal expressions of said first person.

### Claim 10 (depends on 7)

The computer program product of claim 7, wherein said viewable label comprises a label selected from the group consisting of a polygon label comprising a perimeter for surrounding said first visual object and a label comprising different color applied to said first visual object.

### Claim 11 (depends on 7)

The computer program product of claim 7, wherein said first emotional attributes comprise facial expressions of said first person.

### Claim 12

A computing system comprising a processor coupled to a computer-readable memory unit, said memory unit comprising a computer readable code configured to be executed by the processor to perform a method comprising:
receiving, by said computing system, a first audio/video data stream, wherein said first audio/video data stream comprises first speech data associated with a first person;
identifying, by said computing system, first emotional attributes comprised by said first audio/video data stream;
generating, by said computing system in response to said identifying said first emotional attributes, an audio data stream associated with said first audio/video data stream, wherein said audio data stream comprises said first speech data, and wherein said audio data stream does not comprise said first emotional attributes;
associating, by said computing system, a first audible portion of said audio data stream with a first attribute of said first emotional attributes;
generating, by said computing system, an audible label for said first audible portion of said audio data stream, wherein said audible label indicates said first attribute;
applying, by said computing system, said audible label to said first audible portion of said audio data stream;
generating, by said computing system in response to said applying, a second audio data stream associated with said audio data stream, wherein said second audio data stream comprises said audio data stream and said first audible portion of said audio data stream comprising said audible label;
generating, by said computing system, a video data stream associated with said audio/video data stream, wherein said video data stream comprises video data associated with said first person, and wherein said video data does not comprise said first emotional attributes;
associating, by said computing system, a first visual object of said video data with said first attribute of said first emotional attributes;
generating, by said computing system, a viewable label for said first visual object, wherein said viewable label indicates said first attribute of said first emotional attributes;
applying, by said computing system, said viewable label to said first visual object; and
generating, by said computing system in response to said applying, a second audio/video data stream associated with said second video data stream and said second audio data stream, wherein said second audio/video data stream comprises said audio data stream, said first audible portion of said audio data stream comprising said audible label, said second video data, and said first visual object comprising said viewable label.

### Claim 13 (depends on 12)

The computing system of claim 12, wherein said method further comprises:
broadcasting, by said computing system, said second audio/video data stream.

### Claim 14 (depends on 12)

The computing system of claim 12, wherein said first emotional attributes comprise audible and visual vocal expressions of said first person.

### Claim 15 (depends on 12)

The computing system of claim 12, wherein said viewable label comprises a label selected from the group consisting of a polygon label comprising a perimeter for surrounding said first visual object and a label comprising different color applied to said first visual object.

### Claim 16 (depends on 12)

The computing system of claim 12, wherein said first emotional attributes comprise facial expressions of said first person.
