# US 12,118,676 B2 · Sensory stimulus management in head mounted display > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,676 B2 - Document ID: US12118676B2 - Status: Granted - Kind: B2 - Application number: 17/348,686 - Filed: 2021-06-15 - Published: 2021-10-07 - Granted: 2024-10-15 - Earliest claimed: 2014-08-08 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Sony Interactive Entertainment Inc. - Inventors: Xiaodong Mao - Web page: https://patentgrep.com/patent/US12118676B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - A63F 13/211: using inertial sensors, e.g. accelerometers or gyroscopes - A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) - A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client - A63F 13/49: Saving the game status; Pausing or ending the game - A63F 13/53: involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game - A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use - A63F 13/822: Strategy games; Role-playing games (A63F13/825, A63F13/828 take precedence) - A63F 13/825: Fostering virtual characters - G02B 27/0172: {characterised by optical features} - G06F 3/048: Interaction techniques based on graphical user interfaces [GUI] - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods, systems, and computer programs are presented for managing the sensory stimulus generated by a head mounted display (HMD). One method includes identifying a game state of a game being executed for a current session by a computer for display on a head mounted display (HMD) for a user. Then, determining a game intensity value for said game state based on user intensity feedback received from one or more prior gaming sessions by users. The user intensity feedback describes levels of sensory stimulus obtained from the users during user play of the game via an HMD of the respective users, and the determined game intensity value is calculated by analysis of the levels of sensory stimulus received from the users. Processing the game intensity value to cause a higher rendering priority to a viewpoint of the user's gaze than regions outside of the viewpoint. The viewpoint is a portion of the display of the HMD. The regions outside of the viewport are caused to include a reduction in interactive content presented during execution of the current session of the game. ## Claims ### Claim 1 A method comprising: identifying a plurality of game states of a game when rendered for display on a head mounted display (HMD); calculating game intensity values for said plurality of game states based on tracked user behavior obtained from one or more users during prior gaming sessions of the game, the tracked user behavior provides information regarding sensory stimulus experienced during said plurality of game states; and adjusting, during a current session, said game intensity values for content presented to a user, the adjusting reduces graphical elements rendered for a scene being presented on a display of the HMD; wherein the identifying and calculating is processed by a cloud computing system for said tracked user behavior during said prior gaming sessions, and wherein said cloud computing system further processing the adjusting of the game intensity values for content during the current game session for the use; wherein said HMD receives streaming video for said game and renders to the display of the HMD. ### Claim 2 (depends on 1) The method of claim 1, wherein the game intensity values are calculated from feedback received from the one or more users during said prior game session of the game, the game intensity values being associated to areas in the game where higher levels of intensity is anticipated. ### Claim 3 (depends on 2) The method of claim 2, wherein said feedback is captured from physical manifestations of the one or more users during play of the game during said prior game sessions. ### Claim 4 (depends on 3) The method of claim 3, wherein the physical manifestations are associated with eye movement or head movements of the one or more users. ### Claim 5 (depends on 4) The method of claim 4, wherein the eye movement is tracked using a camera, and wherein some eye movement includes erratic or rapid movement indicative of a potential of motion sickness. ### Claim 6 (depends on 1) The method of claim 1, wherein said adjusting for the user of the current user is processed based on profile of the user that defines a level of intensity that is acceptable to the user and a current game state that is associated with one or more of said game intensity values. ### Claim 7 (depends on 1) The method as recited in claim 1, wherein the tracked user behavior is associated to one or more of a current game level, or a user play history, or a user age, or a rate of user input for a game state, or a combination of two or more thereof. ### Claim 8 (depends on 1) The method as recited in claim 1, wherein calculating the game intensity value further includes: calculating a weighted average of parameters related to intensity contributing inputs obtained from users by weighted averaging two or more of physical responses of users while playing the game, or current game level, or user play history, or user age, or rate of user input for the game state, or game developer intensity settings for the game state, or type of game activity configured to take place. ### Claim 9 (depends on 1) The method as recited in claim 1, further comprising, providing, for rendering by the HMD, one or more of a game intensity bar, or a numerical value, or a sound, or a verbal output, or a color selected from a plurality of colors associated with respective game intensity values or a combination of two or more thereof. ### Claim 10 (depends on 1) The method as recited in claim 1, wherein the scene presented on a display of the HMD is one of a virtual reality scene or an augmented reality scene. ### Claim 11 (depends on 1) The method of claim 1, wherein the identifying and calculating is processed by a cloud computing system for said tracked user behavior during said prior gaming sessions, and wherein said cloud computing system further processing the adjusting of the game intensity values for content during the current game session for the use; wherein said HMD receives streaming video for said game and renders to the display of the HMD. ### Claim 12 A method comprising: identifying a plurality of game states of a game when rendered for display on a head mounted display (HMD); calculating, by a computer, game intensity values for said plurality of game states based on tracked user behavior obtained from one or more users during prior gaming sessions of the game, the tracked user behavior provides information regarding sensory stimulus experienced during said plurality of game states; and adjusting, during a current session, said game intensity values for content presented to a user, the adjusting reduces graphical elements rendered for a scene being presented on a display of the HMD; wherein the game intensity values are calculated from feedback received from the one or more users during said prior game session of the game, the game intensity values being associated to areas in the game where higher levels of intensity is anticipated; wherein said feedback is received inputs received from said one or more users, the inputs include a rating of intensity perceived by said one or more users; wherein the inputs represent tags that associate the rating of intensity to specific areas in a virtual space of the game presented via the HMD. ### Claim 13 (depends on 12) The method of claim 12, wherein clusters of said tags identify each of the specific areas in the virtual space. ### Claim 14 A method comprising: identifying a plurality of game states of a game when rendered for display on a head mounted display (HMD); calculating, by a computer, game intensity values for said plurality of game states based on tracked user behavior obtained from one or more users during prior gaming sessions of the game; and adjusting, during a current session, said game intensity values for content presented to a user, the adjusting reduces graphical elements rendered for a scene being presented on a display of the HMD, wherein the tracked user behavior includes tracked eye movement that is indicative a possibility of vertigo. ### Claim 15 Computer readable media having program instructions for presenting content to a head mounted display (HMD), comprising: program instructions for identifying a plurality of game states of a game when rendered for display on a head mounted display (HMD); program instructions for calculating, by a computer, game intensity values for said plurality of game states based on tracked user behavior obtained from one or more users during prior gaming sessions of the game, the tracked user behavior provides information regarding sensory stimulus experienced during said plurality of game states; and program instructions for adjusting, during a current session, said game intensity values for content presented to a user, the adjusting reduces graphical elements rendered for a scene being presented on the display of the HMD, wherein the tracked user behavior includes tracked eye movement that is processed to determine an indication of vertigo. ### Claim 16 (depends on 15) The computer readable media of claim 15, wherein the game intensity values are calculated from feedback received from the one or more users during said prior game session of the game, the game intensity values being associated to areas in the game where higher levels of intensity is anticipated. ### Claim 17 (depends on 15) The computer readable media of claim 15, wherein said adjusting for the user of the current user is processed based on profile of the user that defines a level of intensity that is acceptable to the user and a current game state that is associated with one or more of said game intensity values. ### Claim 18 Computer readable media having program instructions for presenting content to a head mounted display (HMD), comprising: program instructions for identifying a plurality of game states of a game when rendered for display on a head mounted display (HMD); program instructions for calculating, by a computer, game intensity values for said plurality of game states based on tracked user behavior obtained from one or more users during prior gaming sessions of the game; and program instructions for adjusting, during a current session, said game intensity values for content presented to a user, the adjusting reduces graphical elements rendered for a scene being presented on the display of the HMD; wherein the tracked user behavior is associated to one or more of a current game level, or a user play history, or a user age, or a rate of user input for a game state, or a combination of two or more thereof; and wherein the identifying and calculating is processed by a cloud computing system for said tracked user behavior during said prior gaming sessions, and wherein said cloud computing system further processing the adjusting of the game intensity values for content during the current game session for the use; wherein said HMD receives streaming video for said game and renders to the display of the HMD.