# US 11,026,612 B2 · Information processing system, information processing device, storage medium storing information processing program, and information processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,026,612 B2 - Document ID: US11026612B2 - Status: Granted - Kind: B2 - Application number: 16/570,951 - Filed: 2019-09-13 - Published: 2020-01-02 - Granted: 2021-06-08 - Earliest claimed: 2014-08-07 - Assignees: Nintendo Co., Ltd. - Applicants: Nintendo Co., Ltd. - Inventors: Yurie Hattori; Masahiro Kondo - Web page: https://patentgrep.com/patent/US11026612B2 ## CPC - A61B 5/16: Devices for psychotechnics (using teaching or educational appliances G09B1/00 - G09B7/00); Testing reaction times {; Devices for evaluating the psychological state} - A47C 31/00: Details or accessories for chairs, beds, or the like, not provided for in other groups of this subclass, e.g. upholstery fasteners, mattress protectors, stretching devices for mattress nets - A61B 5/0022: {Monitoring a patient using a global network, e.g. telephone networks, internet} - A61B 5/0205: Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition - A61B 5/024: Measuring pulse rate or heart rate - A61B 5/11: Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb {(A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721)} - A61B 5/1118: {Determining activity level} - A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety} - A61B 5/4806: {Sleep evaluation (A61B5/4821 takes precedence; devices for inducing sleep A61M21/02)} - A61B 5/4809: {Sleep detection, i.e. determining whether a subject is asleep or not} - A61B 5/4812: {Detecting sleep stages or cycles} - A61B 5/486: {Biofeedback (using electroencephalography [EEG] A61B5/375)} - A61B 5/7271: {Specific aspects of physiological measurement analysis (specific diagnostics methods using bioelectric or biomagnetic signals A61B5/316)} - A61B 5/7405: {using sound} - A61B 5/742: {using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339)} - A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity - G06F 16/2228: {Indexing structures} - G06Q 20/10: specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems - G06Q 30/02: Marketing; Price estimation or determination; Fundraising - G06Q 30/0641: {utilising user interfaces specially adapted for shopping} - G06Q 50/22: Social work or social welfare, e.g. community support activities or counselling services - G16H 10/20: for electronic clinical trials or questionnaires - G16H 10/60: for patient-specific data, e.g. for electronic patient records - G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising - G16H 40/63: for local operation - G16H 40/67: for remote operation - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G16H 50/30: for calculating health indices; for individual health risk assessment - G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients - H04N 9/3179: {Video signal processing therefor} - H04N 9/3182: {Colour adjustment, e.g. white balance, shading or gamut (white balance per se H04N9/73; control of amplitude of colour signals H04N9/68; colour control circuits for displays in general G09G5/02, G09G3/2003, G09G3/3607)} - H04N 9/3194: {including sensor feedback} - A61B 5/08: Measuring devices for evaluating the respiratory organs - A61B 2560/0242: adapted to measure environmental factors, e.g. temperature, pollution - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - A47C: CHAIRS; SOFAS; BEDS - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An example information processing system executes a game application. The information processing system obtains user information for calculating information relating to sleep of a user, and calculates health information relating to sleep and/or fatigue of the user based on the obtained user information. The information processing system controls a game progress of the game application based on the health information. ## Claims ### Claim 1 An information processing system, comprising: a sensor; and an electronic device comprising: a memory for storing instructions; and one or more processors in communication with the memory, wherein the instructions, when executed, configure the one or more processors to: acquire awake information regarding a user including at least either of activity information indicating an activity performed by the user during awakening and environment information indicating an environment around the user during awakening; detect the user's getting in bed or homecoming; and on a condition that the user's getting in bed or homecoming is detected, display a game image indicating a state in which a game advances based on the acquired awake information, and in the game image, the awake information is reflected on at least one of a game space or a player object appearing in the game space. ### Claim 2 (depends on 1) The information processing system according to claim 1, wherein the game image includes an image of the player object reproducing the activity of the user indicated by the activity information. ### Claim 3 (depends on 1) The information processing system according to claim 1, wherein the game image includes an image of a game space representing the environment around the user indicated by the environment information. ### Claim 4 (depends on 1) The information processing system according to claim 1, wherein the game image includes a numerical value indicating an evaluation result of health of the user calculated based on the awake information. ### Claim 5 (depends on 1) The information processing system according to claim 1, wherein the instructions further configure the one or more processors to: in a period before the user's getting in bed or homecoming is detected, display the game image indicating the state in which the game advances based on the awake information; and the game image displayed on the condition that the user's getting in bed or homecoming is detected includes at least a part of the game image displayed in the period. ### Claim 6 (depends on 1) The information processing system according to claim 1, wherein the instructions further configure the one or more processors to: calculate an evaluation result of health of the user based on the awake information; and the game image includes advice information calculated based on the awake information and indicating advice for the user to improve the evaluation result. ### Claim 7 (depends on 1) The information processing system according to claim 1, wherein the game image includes information indicating a privilege that the user is to be awarded in accordance with a game result based on the awake information. ### Claim 8 (depends on 1) The information processing system according to claim 1, wherein the instructions further configure the one or more processors to: acquire information regarding sleep of the user; detect awakening of the user based on the information regarding the sleep; and on a condition that the awakening of the user is detected, display a first numerical value indicating health of the user and calculated based on the information regarding the sleep; and the game image displayed on the condition that the user's getting in bed or homecoming is detected includes a second numerical value calculated based on the awake information and of the same type as the first numerical value displayed on the condition that the awakening of the user is detected. ### Claim 9 (depends on 8) The according to claim 8, wherein the second numerical value is calculated by correcting the first numerical value based on the awake information. ### Claim 10 (depends on 1) The information processing system according to claim 1, wherein the instructions further configure the one or more processors to: acquire information regarding sleep of the user; detect awakening of the user based on the information regarding the sleep; and on a condition that the awakening of the user is detected, display a game image representing that the player object starts a predetermined action; and the game image displayed on the condition that the user's getting in bed or homecoming is detected includes an image representing that the player object ends the predetermined action. ### Claim 11 (depends on 10) The information processing system according to claim 10, wherein the instructions further configure the one or more processors to: in a period from when the awakening of the user is detected to when the user's getting in bed or homecoming is detected, display a game image indicating the player object in the middle of performing the predetermined action. ### Claim 12 (depends on 1) The information processing system according to claim 1, wherein: the awake information further includes emotion information indicating emotion of the user; and the game image includes an image indicating the player object on which the emotion of the user indicated by the emotion information is reflected. ### Claim 13 An information processing device comprising: a memory for storing instructions; and one or more processors in communication with the memory, wherein the instructions, when executed, configure the one or more processors to: acquire awake information regarding a user including at least either of activity information indicating an activity performed by the user during awakening and environment information indicating an environment around the user during awakening; detect the user's getting in bed or homecoming; and on a condition that the user's getting in bed or homecoming is detected, display a game image indicating a state in which a game advances based on the acquired awake information, and in the game image, the awake information is reflected on at least one of a game space or a player object appearing in the game space. ### Claim 14 A non-transitory computer-readable storage medium storing an information processing program to be executed on a computer of an information processing device, the information processing program, when executed by the computer, causing the computer to: acquire awake information regarding a user including at least either of activity information indicating an activity performed by the user during awakening and environment information indicating an environment around the user during awakening; detect the user's getting in bed or homecoming; and on a condition that the user's getting in bed or homecoming is detected, display a game image indicating a state in which a game advances based on the acquired awake information, and in the game image, the awake information is reflected on at least one of a game space or a player object appearing in the game space. ### Claim 15 An information processing method for executing on an information processing system that executes a game application, the method comprising: acquiring awake information regarding a user including at least either of activity information indicating an activity performed by the user during awakening and environment information indicating an environment around the user during awakening; detecting the user's getting in bed or homecoming; and on a condition that the user's getting in bed or homecoming is detected, displaying a game image indicating a state in which a game advances based on the acquired awake information, and in the game image, the awake information is reflected on at least one of a game space or a player object appearing in the game space.