# US 2023/0057722 A1 · Enhanced Emotive Engagement with Volumetric Content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0057722 A1 - Document ID: US20230057722A1 - Status: Published application - Kind: A1 - Application number: 17/408,542 - Filed: 2021-08-23 - Published: 2023-02-23 - Earliest claimed: 2021-08-23 - Assignees: AT&T Intellectual Property I, L.P. - Applicants: AT&T Intellectual Property I, L.P. - Inventors: Eric Zavesky; David Gibbon; Wen-Ling Hsu; Jianxiong Dong; Richard Palazzo - Web page: https://patentgrep.com/patent/US20230057722A1 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - H04N 13/366: using viewer tracking - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes} - H04N 13/167: Synchronising or controlling image signals - H04N 13/172: image signals comprising non-image signal components, e.g. headers or format information - H04N 13/398: Synchronisation thereof; Control thereof - H04N 23/60: Control of cameras or camera modules - G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection} - G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A volumetric content enhancement system (“the system”) can annotate at least a portion of a plurality of voxels from a volumetric video with contextual data. The system can determine at least one actionable position within the volumetric video. The system can create an annotated volumetric video that includes the volumetric video, an annotation with the contextual data, and the at least one actionable position. The system can provide the annotated volumetric video to a volumetric content playback system. The system can obtain viewer feedback associated with the viewer and can determine an emotional state of the viewer based, at least in part, upon the viewer feedback. The system can receive viewer position information that identifies a specific actionable position of the viewer. The system can generate manipulation instructions to instruct the volumetric content playback system to manipulate the annotated volumetric content to achieve a desired emotional state of the viewer. ## Claims ### Claim 1 A method comprising: creating, by a volumetric content enhancement system comprising a processor, an annotated volumetric video comprising a volumetric video, an annotation, and at least one actionable position; providing, by the volumetric content enhancement system, the annotated volumetric video to a volumetric content playback system through which a viewer can view the annotated volumetric video; obtaining, by the volumetric content enhancement system, viewer feedback associated with a viewer; determining, by the volumetric content enhancement system, an emotional state of the viewer based, at least in part, upon the viewer feedback; receiving, by the volumetric content enhancement system, viewer position information from the volumetric content playback system, wherein the viewer position information identifies a specific actionable position of the at least one actionable position comprised in the annotated volumetric video; generating, by the volumetric content enhancement system, manipulation instructions to instruct the volumetric content playback system to manipulate the annotated volumetric content to achieve a desired emotional state of the viewer, wherein the manipulation instructions are based, at least in part, upon the viewer feedback, the emotional state, and the viewer position information; and providing, by the volumetric content enhancement system, the manipulation instructions to the volumetric content playback system. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving, by the volumetric content enhancement system, the volumetric from a volumetric content capture system; partitioning, by the volumetric content enhancement system, the volumetric video into a plurality of video frames; analyzing, by the volumetric content enhancement system, the plurality of video frames to identify a plurality of voxels; annotating, by the volumetric content enhancement system, at least a portion of the plurality of voxels with the annotation, wherein the annotation comprises contextual data about the portion of the plurality of voxels; and determining, by the volumetric content enhancement system, the at least one actionable position within the volumetric video. ### Claim 3 (depends on 1) The method of claim 1, further comprising initializing, by the volumetric content enhancement system, the emotional state of the viewer of the annotated volumetric video. ### Claim 4 (depends on 1) The method of claim 1, wherein the viewer feedback comprises explicit feedback provided by the viewer, passive feedback provided by at least one sensor monitoring the viewer, or the explicit feedback and the passive feedback. ### Claim 5 (depends on 1) The method of claim 1, wherein the portion of the plurality of voxels comprises a region, and wherein the region identifies a character or an object depicted in the volumetric video. ### Claim 6 (depends on 1) The method of claim 1, further comprising generating, by the volumetric content enhancement system, viewer guidance instructions to instruct the viewer to change from the specific actionable position to a different actionable position. ### Claim 7 (depends on 1) The method of claim 1, further comprising: obtaining, by the volumetric content enhancement system, new viewer feedback associated with the viewer; and reassessing, by the volumetric content enhancement system, the emotional state of the viewer based, at least in part, the new viewer feedback. ### Claim 8 A volumetric content enhancement system comprising: a processor; and a memory comprising instructions that, when executed by the processor, cause the processor to perform operations comprising creating an annotated volumetric video comprising a volumetric video, an annotation, and at least one actionable position, providing the annotated volumetric video to a volumetric content playback system through which a viewer can view the annotated volumetric video, obtaining viewer feedback associated with a viewer, determining an emotional state of the viewer based, at least in part, upon the viewer feedback, receiving viewer position information from the volumetric content playback system, wherein the viewer position information identifies a specific actionable position of the at least one actionable position comprised in the annotated volumetric video, generating manipulation instructions to instruct the volumetric content playback system to manipulate the annotated volumetric video to achieve a desired emotional state of the viewer, wherein the manipulation instructions are based, at least in part, upon the viewer feedback, the emotional state, and the viewer position information, and providing the manipulation instructions to the volumetric content playback system. ### Claim 9 (depends on 8) The volumetric content enhancement system of claim 8, wherein the operations further comprise: receiving the volumetric from a volumetric content capture system; partitioning the volumetric video into a plurality of video frames; analyzing the plurality of video frames to identify a plurality of voxels; annotating at least a portion of the plurality of voxels with the annotation, wherein the annotation comprises contextual data about the portion of the plurality of voxels; and determining the at least one actionable position within the volumetric video. ### Claim 10 (depends on 8) The volumetric content enhancement system of claim 8, wherein the operations further comprise initializing the emotional state of the viewer of the annotated volumetric video. ### Claim 11 (depends on 8) The volumetric content enhancement system of claim 8, wherein the portion of the plurality of voxels comprises a region, and wherein the region identifies a character or an object depicted in the volumetric video. ### Claim 12 (depends on 8) The volumetric content enhancement system of claim 8, wherein the viewer feedback comprises explicit feedback provided by the viewer, passive feedback provided by at least one sensor monitoring the viewer, or the explicit feedback and the passive feedback. ### Claim 13 (depends on 8) The volumetric content enhancement system of claim 8, wherein the operations further comprise generating viewer guidance instructions to instruct the viewer to change from the specific actionable position to a different actionable position. ### Claim 14 (depends on 8) The volumetric content enhancement system of claim 8, wherein the operations further comprise: obtaining new viewer feedback associated with the viewer; and reassessing the emotional state of the viewer based, at least in part, the new viewer feedback. ### Claim 15 A computer-readable storage medium comprising computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising: creating an annotated volumetric video comprising a volumetric video, an annotation, and at least one actionable position; providing the annotated volumetric video to a volumetric content playback system through which a viewer can view the annotated volumetric video; obtaining viewer feedback associated with a viewer; determining an emotional state of the viewer based, at least in part, upon the viewer feedback; receiving viewer position information from the volumetric content playback system, wherein the viewer position information identifies a specific actionable position of the at least one actionable position comprised in the annotated volumetric video; generating manipulation instructions to instruct the volumetric content playback system to manipulate the annotated volumetric video to achieve a desired emotional state of the viewer, wherein the manipulation instructions are based, at least in part, upon the viewer feedback, the emotional state, and the viewer position information; and providing the manipulation instructions to the volumetric content playback system. ### Claim 16 (depends on 15) The computer-readable storage medium of claim 15, wherein the operations further comprise: receiving the volumetric from a volumetric content capture system; partitioning the volumetric video into a plurality of video frames; analyzing the plurality of video frames to identify a plurality of voxels; annotating at least a portion of the plurality of voxels with the annotation, wherein the annotation comprises contextual data about the portion of the plurality of voxels; and determining the at least one actionable position within the volumetric video. ### Claim 17 (depends on 15) The computer-readable storage medium of claim 15, wherein the operations further comprise initializing the emotional state of the viewer of the annotated volumetric video. ### Claim 18 (depends on 15) The computer-readable storage medium of claim 15, wherein the viewer feedback comprises explicit feedback provided by the viewer, passive feedback provided by at least one sensor monitoring the viewer, or the explicit feedback and the passive feedback. ### Claim 19 (depends on 15) The computer-readable storage medium of claim 15, wherein the operations further comprise generating viewer guidance instructions to instruct the viewer to change from the specific actionable position to a different actionable position. ### Claim 20 (depends on 15) The computer-readable storage medium of claim 15, wherein the operations further comprise: obtaining new viewer feedback associated with the viewer; and reassessing the emotional state of the viewer based, at least in part, the new viewer feedback.