# US 10,482,641 B2 · Virtual reality display > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,641 B2 - Document ID: US10482641B2 - Status: Granted - Kind: B2 - Application number: 15/591,540 - Filed: 2017-05-10 - Published: 2017-11-23 - Granted: 2019-11-19 - Earliest claimed: 2016-05-20 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Francesco Cricri; Jukka Pentti Paivio Saarinen - Web page: https://patentgrep.com/patent/US10482641B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06T 11/60: Creating or editing images; Combining images with text - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06F 3/012: {Head tracking input arrangements} - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06T 2210/22: Cropping - G06V 20/44: {Event detection} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method comprises providing video data representing at least part of virtual space to a user for viewing, identifying a current viewed sector of the virtual space based on user position, determining a sub-portion of said viewing sector, identifying an event occurring in a non-viewed sector of the virtual space, and displaying content indicative of the event in the sub-portion of said current viewing sector. The displaying step may comprise displaying a graphical notification of the event in the sub-portion, or in alternative embodiments, displaying video data showing the event in the sub-portion. ## Claims ### Claim 1 A method comprising: providing video data representing at least part of a virtual space to a user for viewing; identifying a current viewed sector of the virtual space based on user position; determining a sub-portion of said current viewed sector; identifying an event occurring in video content of at least one non-viewed sector of the virtual space, wherein the event is a detected change in the video content in sequential video frames of the at least one non-viewed sector of the virtual space; dividing the at least one non-viewed sector of the virtual space in which the change in the video content is detected into spatial crop areas; and displaying in the sub-portion, content indicative of the event, wherein the displaying content indicative of the event comprises displaying one of the spatial crop areas selected based on a size relative to other spatial crop areas-showing the event, and receiving video data corresponding to different spatial crop areas of the at least one non-viewed sector and wherein if the identified event occurs within multiple spatial crop areas, the displaying content indicative of the event comprises displaying video data from the smallest crop area. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving manual selection through a user interface of the sub-portion of said current viewed sector. ### Claim 3 (depends on 1) The method of claim 1, wherein the determining the sub-portion is performed automatically using image or video attributes within the viewed sector. ### Claim 4 (depends on 1) The method of claim 1, wherein the determining the sub-portion uses predefined rules to identify at least one of substantially static or uniform graphical areas within the viewed sector. ### Claim 5 (depends on 3) The method of claim 3, wherein multiple possible sub-portions are identified and ranked using the predefined rules, and wherein the method further comprises selecting a sub-portion based on rank. ### Claim 6 (depends on 3) The method of claim 3, further comprising detecting a focal area within the current viewed sector and wherein the sub-portion is determined to be outside of the focal area. ### Claim 7 (depends on 6) The method of claim 6, wherein detecting the focal area is based on the detected gaze direction of the user. ### Claim 8 (depends on 6) The method of claim 6, wherein a trained model is used to determine the sub-portion. ### Claim 9 (depends on 1) The method of claim 1, wherein identifying the event comprises receiving manual selection through a user interface. ### Claim 10 (depends on 1) The method of claim 1, wherein identifying the event is performed automatically using predefined rules. ### Claim 11 (depends on 1) The method of claim 1, wherein identifying the event is performed automatically using a trained model. ### Claim 12 (depends on 11) The method of claim 11, wherein the trained model uses a convolutional neural network and a recurrent neural network. ### Claim 13 (depends on 10) The method of claim 10, wherein the at least one of the at least one non-viewed sector is divided into multiple sub-sectors and each is analysed individually to identify an event. ### Claim 14 (depends on 1) The method of claim 1, wherein the video data showing the identified event is substantially kept the same size when displayed. ### Claim 15 (depends on 1) The method of claim 1, wherein the video data showing the identified event is transformed from a three-dimensional or polar representation to a two-dimensional representation. ### Claim 16 A non-transitory computer-readable medium having stored thereon computer program code, which, when executed by a computer, causes the computer to perform a method, comprising: providing video data representing at least part of a virtual space to a user for viewing; identifying a current viewed sector of the virtual space based on user position; determining a sub-portion of said current viewed sector; identifying an event occurring in video content of at least one non-viewed sector of the virtual space, wherein the event is a detected change in the video content in sequential video frames of the at least one non-viewed sector of the virtual space; dividing the at least one non-viewed sector of the virtual space in which the change in the video content is detected into spatial crop areas; displaying in the sub-portion, content indicative of the event in the sub-portion, wherein the displaying content indicative of the event comprises displaying one of the spatial crop areas selected based on a size relative to other spatial crop areas showing the event; and receiving video data corresponding to different spatial crop areas of the at least one non-viewed sector and wherein if the identified event occurs within multiple spatial crop areas, the displaying content indicative of the event comprises displaying video data from the smallest crop area. ### Claim 17 An apparatus having at least one processor and at least one memory having computer program code stored thereon which when executed controls the apparatus: to provide video data representing at least part of a virtual space to a user for viewing; to identify a current viewed sector of the virtual space based on user position; to determine a sub-portion of said current viewed sector; to identify an event occurring in video content of at least one non-viewed sector of the virtual space, wherein the event is a detected change in the video content in sequential video frames of the at least one non-viewed sector of the virtual space; to divide the at least one non-viewed sector of the virtual space in which the change in the video content is detected into spatial crop areas; to display in the sub-portion, content indicative of the event, wherein displaying content indicative of the event comprises displaying one of the spatial crop areas selected based on a size relative to other spatial crop areas showing the event, and to receive video data corresponding to different spatial crop areas of the at least one non-viewed sector and wherein if the identified event occurs within multiple spatial crop areas, the displaying content indicative of the event comprises displaying video data from the smallest crop area. ### Claim 18 (depends on 17) The apparatus of claim 17, wherein the computer program code controls the apparatus to detect a focal area within the current viewed sector and wherein the sub-portion is determined to be outside of the focal area. ### Claim 19 (depends on 1) The method of claim 1, wherein a smallest spatial crop area of the spatial crop areas is selected for display.