# US 2021/0342109 A1 · Content reproduction device, picture data output device, content creation device, content reproduction method, picture data output method, and content creation method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0342109 A1 - Document ID: US20210342109A1 - Status: Published application - Kind: A1 - Application number: 17/285,803 - Filed: 2018-10-26 - Published: 2021-11-04 - Earliest claimed: 2018-10-26 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Sony Interactive Entertainment Inc. - Inventors: Junichi Muramoto - Web page: https://patentgrep.com/patent/US20210342109A1 ## CPC - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation} - H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking - H04N 21/431: Generation of visual interfaces {for content selection or interaction}; Content or additional data rendering - H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} - H04N 21/854: Content authoring - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - G09G 2354/00: Aspects of interface with display user - G09G 2370/20: Details of the management of multiple sources of image data - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract Original pictures photographed from a plurality of photograph viewpoints are prepared, display pictures for regions corresponding to visual-lines are generated while switching to an original picture photographed from another viewpoint is being performed according to movement of a subject 81 or the like. The original pictures are associated with tag data indicating the positions of images of particular subjects. When the viewpoint is switched, the visual line is controlled such that the position indicated by a tag for a same subject is included within the visual field before and after the switching. Accordingly, a post-switching region is displayed. ## Claims ### Claim 1 A content reproduction device comprising: a data acquisition section that acquires data on an original whole-sky picture for use in display, and tag data indicating a position of a particular subject in the original picture; a display-viewpoint acquisition section that acquires a display-time viewpoint and a display-time visual line with respect to the original picture; a display-picture generation section that generates, from the original picture, a display picture in a visual field based on the display-time viewpoint and the display-time visual line; and a data output section that outputs data on the display picture to a display device, wherein, in a state where a predetermined condition is satisfied, the display-picture generation section performs processing for including the position in the original picture indicated by the tag into the visual field, by referring to the tag data. ### Claim 2 (depends on 1) The content reproduction device according to claim 1, wherein the data acquisition section acquires a plurality of original pictures obtained by full-azimuth photographing from different viewpoints, and positional information regarding viewpoints when the photographing is performed, and, when switching the original picture for use in the display according to a change of the display-time viewpoint, the display-picture generation section changes the display by controlling the post-switching visual line such that the position indicated by the tag for a same subject is included within the visual field before and after the switching. ### Claim 3 (canceled) ### Claim 4 (depends on 1) The content reproduction device according to claim 1, wherein the display-picture generation section changes the display by controlling the visual line such that the position indicated by the tag is included within the visual field when the viewer operates predetermined input means. ### Claim 5 (depends on 1) The content reproduction device according to claim 1, wherein the data acquisition section acquires a plurality of original pictures obtained by full-azimuth photographing from different viewpoints, and positional information regarding viewpoints when the photographing is performed, and, when switching the original picture for use in display according to a change of the display-time viewpoint, the display-picture generation section displays a geometrical figure to indicate, in the switched original picture, a relative direction of the position indicated by the tag for a same subject that is included within the visual field before the switching. ### Claim 6 (depends on 1) The content reproduction device according to claim 1, wherein the data acquisition section acquires the original picture data which is a whole-sky video including a plurality of frames, and the tag data associated with the respective frames. ### Claim 7 A picture data output device that outputs data on an original whole-sky picture for use in display, the picture data output device comprising: an original picture acquisition section that acquires, as data on the original picture, a video including a plurality of frames obtained by full-azimuth photographing from different viewpoints; a tag data generation section that generates tag data for display control, the tag data indicating a position of a particular subject in each of the frames; and a data output section that outputs data on the original picture and the tag data in association with each other. ### Claim at position 8 -9. (canceled) ### Claim 10 A content creation device comprising: a data acquisition section that acquires data on a plurality of original pictures obtained by full-azimuth photographing from different viewpoints, and data on tags indicating positions of a particular subject in the respective original pictures; a content generation section that generates video data using the original pictures by setting data on an original picture for use in display, a timing of switching the original picture, and a visual field to be displayed, with respect to a time axis of the display; and a data output section that outputs the video data, wherein the content generation section controls a post-switching visual field at the timing of switching the original picture such that the position indicated by the tag for a same subject is included within the visual field before and after switching. ### Claim 11 A content reproduction method by a content reproduction device, comprising: acquiring data on an original whole-sky picture for use in display, and tag data indicating a position of a particular subject in the original picture; acquiring a display-time viewpoint and a display-time visual line with respect to the original picture; generating, from the original picture, a display picture in a visual field based on the display-time viewpoint and the display-time visual line; and outputting data on the display picture to a display device, wherein the generating the display picture involves performing, in a state where a predetermined condition is satisfied, processing for including the position in the original picture indicated by the tag into the visual field, by referring to the tag data. ### Claim 12 A picture data output method by a picture data output device that outputs data on an original whole-sky picture for use in display, the picture data output method comprising: acquiring, as data on the original picture, a video including a plurality of frames obtained by full-azimuth photographing from different viewpoints; generating tag data for display control, tag data indicating a position of a particular subject in each of the frames; and outputting data on the original picture and the tag data in association with each other. ### Claim 13 A content creation method by a content creation device, comprising: acquiring data on a plurality of original pictures obtained by full-azimuth photographing from different viewpoints, and data on tags indicating positions of a particular subject in the respective original pictures; generating video data using the original pictures by setting data on an original picture for use in display, a timing of switching the original picture, and a visual field to be displayed, with respect to a time axis of the display; and outputting the video data, wherein the generating the video data involves controlling a post-switching visual field at the timing of switching the original picture such that the position indicated by the tag for a same subject is included within the visual field before and after the switching. ### Claim 14 A non-transitory, computer readable storage medium containing a computer program, which when executed by a computer, causes the computer to carry out actions, comprising: acquiring data on an original whole-sky picture for use in display and tag data indicating a position of a particular subject in the original picture; acquiring a display-time viewpoint and a display-time visual line with respect to the original picture; generating, from the original picture, a display picture in a visual field based on the display-time viewpoint and the display-time visual line; and outputting data on the display picture to a display device, wherein generating the display picture involves performing, in a state where a predetermined condition is satisfied, processing for including the position in the original picture indicated by the tag into the visual field, by referring to the tag data. ### Claim 15 A non-transitory, computer readable storage medium containing a computer program, which when executed by a computer, causes the computer to carry out actions, comprising: a function of acquiring, as data on the original picture, a video including a plurality of frames obtained by full-azimuth photographing from different viewpoints; a function of generating tag data for display control, the tag data indicating a position of a particular subject in each of the frames; and a function of outputting data on the original picture and the tag data in association with each other. ### Claim 16 A non-transitory, computer readable storage medium containing a computer program, which when executed by a computer, causes the computer to carry out actions, comprising: a function of acquiring data on a plurality of original pictures obtained by full-azimuth photographing from different viewpoints and data on tags indicating positions of a particular subject in the respective original pictures; a function of generating video data using the original pictures by setting data on an original picture for use in display, a timing of switching the original picture, and a visual filed to be displayed, with respect to a time axis of the display; and a function of outputting the video data, wherein the function of generating video data involves controlling a post-switching visual field at the timing of switching the original picture such that the position indicated by the tag for a same subject is included within the visual field before and after the switching. ### Claim 17 (depends on 6) The content reproduction device according to claim 6, wherein the display-viewpoint acquisition section switches the original picture for use in display to an original picture photographed from a different viewpoint, according to movement of the subject corresponding to the tag, and the display-picture generation section controls the visual line such that a position of the subject indicated by the tag at a time point of the switching is included in the visual field of the switched original picture.