# US 10,504,617 B2 · Information processing system, information processing device, storage medium storing information processing program, and information processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,504,617 B2 - Document ID: US10504617B2 - Status: Granted - Kind: B2 - Application number: 15/421,708 - Filed: 2017-02-01 - Published: 2017-05-18 - Granted: 2019-12-10 - Earliest claimed: 2014-08-07 - Assignees: Nintendo Co., Ltd. - Applicants: Nintendo Co., Ltd. - Inventors: Yurie Hattori; Masahiro Kondo - Web page: https://patentgrep.com/patent/US10504617B2 ## CPC - A61B 5/16: Devices for psychotechnics (using teaching or educational appliances G09B1/00 - G09B7/00); Testing reaction times {; Devices for evaluating the psychological state} - A47C 31/00: Details or accessories for chairs, beds, or the like, not provided for in other groups of this subclass, e.g. upholstery fasteners, mattress protectors, stretching devices for mattress nets - A61B 5/0022: {Monitoring a patient using a global network, e.g. telephone networks, internet} - A61B 5/0205: Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition - A61B 5/024: Measuring pulse rate or heart rate - A61B 5/11: Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb {(A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721)} - A61B 5/1118: {Determining activity level} - A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety} - A61B 5/4806: {Sleep evaluation (A61B5/4821 takes precedence; devices for inducing sleep A61M21/02)} - A61B 5/4809: {Sleep detection, i.e. determining whether a subject is asleep or not} - A61B 5/4812: {Detecting sleep stages or cycles} - A61B 5/486: {Biofeedback (using electroencephalography [EEG] A61B5/375)} - A61B 5/7271: {Specific aspects of physiological measurement analysis (specific diagnostics methods using bioelectric or biomagnetic signals A61B5/316)} - A61B 5/7405: {using sound} - A61B 5/742: {using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339)} - A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity - G06F 16/2228: {Indexing structures} - G06Q 20/10: specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems - G06Q 30/02: Marketing; Price estimation or determination; Fundraising - G06Q 30/0641: {utilising user interfaces specially adapted for shopping} - G06Q 50/22: Social work or social welfare, e.g. community support activities or counselling services - G16H 10/20: for electronic clinical trials or questionnaires - G16H 10/60: for patient-specific data, e.g. for electronic patient records - G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising - G16H 40/63: for local operation - G16H 40/67: for remote operation - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G16H 50/30: for calculating health indices; for individual health risk assessment - G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients - H04N 9/3179: {Video signal processing therefor} - H04N 9/3182: {Colour adjustment, e.g. white balance, shading or gamut (white balance per se H04N9/73; control of amplitude of colour signals H04N9/68; colour control circuits for displays in general G09G5/02, G09G3/2003, G09G3/3607)} - H04N 9/3194: {including sensor feedback} - A61B 5/08: Measuring devices for evaluating the respiratory organs - A61B 2560/0242: adapted to measure environmental factors, e.g. temperature, pollution - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - A47C: CHAIRS; SOFAS; BEDS - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An example information processing system executes a game application. The information processing system obtains user information for calculating information relating to sleep of a user, and calculates health information relating to sleep and/or fatigue of the user based on the obtained user information. The information processing system controls a game progress of the game application based on the health information. ## Claims ### Claim 1 An information processing system, comprising: a sensor; and an electronic device comprising: a memory for storing instructions; and one or more processors in communication with the memory, wherein the instructions, when executed, configure the one or more processors to: obtain, based on a user awakening detection for a user, an evaluation result based on sleep information sensed by the sensor during a sleep period of the user; obtain, based on the evaluation result, a numerical value; obtain an adjusted numerical value based on awake information acquired during an awake period of the user; and control a video game based at least in part on the adjusted numerical value. ### Claim 2 (depends on 1) The information processing system according to claim 1, wherein the numerical value corresponds to an energy level of a game object of the video game. ### Claim 3 (depends on 1) The information processing system according to claim 1, wherein the instructions, when executed, configure the one or more processors to automatically control the video game based on the adjusted numerical value satisfying a predetermined condition. ### Claim 4 (depends on 1) The information processing system according to claim 1, wherein: the instructions, when executed, configure the one or more processors to: accept, based on the user awakening detection or a user out-of-bed detection, an instruction input for starting execution of another video game; and control the other video game based on operation inputs. ### Claim 5 (depends on 4) The information processing system according to claim 4, wherein the instructions, when executed, configure the one or more processors to accept the the instruction input for starting execution of the other video game within a predetermined period of time from a point in time from the user awakening detection or the user out-of-bed detection. ### Claim 6 (depends on 4) The information processing system according to claim 4, wherein the instructions, when executed, configure the one or more processors to control an aspect of the video game based on a result of the other video game. ### Claim 7 (depends on 4) The information processing system according to claim 4, wherein the numerical value is based on fatigue-related information of the user based on an input in the other video game and a sleep-related state of the user. ### Claim 8 (depends on 1) The information processing system according to claim 1, wherein the instructions, when executed, configure the one or more processors to: obtain sleep information during a plurality of sleep periods of the user; obtain, for each sleep period, a respective numerical value based on the sleep information acquired in the sleep period; and control the video game based on a comparison result between a respective first numerical value for a latest sleep period and a respective second numerical value for a prior sleep period prior to the latest sleep period. ### Claim 9 (depends on 8) The information processing system according to claim 8, wherein the instructions, when executed, configure the one or more processors to control the video game based on a difference between the first and second numerical values. ### Claim 10 (depends on 8) The information processing system according to claim 8, wherein the instructions, when executed, configure the one or more processors to determine whether or not the first numerical value represents an improvement from the second numerical value by a predetermined reference level or more, and control the video game based on the determination result. ### Claim 11 (depends on 1) The information processing system according to claim 1, wherein: the instructions, when executed, configure the one or more processors to control the video game based on a comparison result between the adjusted numerical value for the user and numerical values for one or more other users. ### Claim 12 (depends on 1) The information processing system according to claim 1, wherein the instructions, when executed, configure the one or more processors to: evaluate sleep and/or fatigue of the user; generate advice information and/or recommendation information for improving the evaluation; and control display of the advice information and/or recommendation information. ### Claim 13 (depends on 1) The information processing system according to claim 1, wherein the instructions, when executed, configure the one or more processors to identify one or more of: (1) an amount of time from the user getting in bed to sleep onset; (2) an amount of time from the user awakening to getting out of bed; (3) a ratio of an amount of time of deep sleep with respect to a predetermined period of time since user sleep onset; (4) a ratio of an amount of time of light sleep with respect to a predetermined period of time until user awakening; (5) a REM sleep cycle; (6) a ratio between REM sleep time and non-REM sleep time; (7) a sleep time; (8) a difference between ideal sleep onset timing and actual sleep onset timing; (9) a difference between ideal awakening timing and actual awakening timing; (10) a number of mid-sleep awakenings; (11) a timing of mid-sleep awakenings; (12) a wake time after sleep onset; (13) a number of times of tossing and turning; (14) a level of snoring; and (15) an environment during sleep. ### Claim 14 (depends on 1) The information processing system according to claim 1, further comprising: a server configured for communication with the electronic, wherein the server accumulates the numerical value. ### Claim 15 (depends on 1) The information processing system according to claim 1, wherein the instructions, when executed, configure the one or more processors to provide a wake-up alarm. ### Claim 16 (depends on 1) The information processing system according to claim 1, wherein the sensor is configured to sense at least one of pulse, breathing and body movements of the user. ### Claim 17 (depends on 1) The information processing system according to claim 1, wherein the sensor comprises a contactless biological information sensor. ### Claim 18 (depends on 17) The information processing system according to claim 17, wherein the sensor is configured to emit radio waves or acoustic waves, receive reflected waves, and output biological information based on the received reflected waves. ### Claim 19 (depends on 18) The information processing system according to claim 18, wherein the sensor comprises a Doppler sensor. ### Claim 20 A non-transitory computer-readable storage medium storing an information processing program to be executed on a computer of an information processing device that executes a video game program for a video game, the information processing program, when executed by the computer, causing the computer to: obtain, based on detecting an awakening of a user, an evaluation result based on sleep information sensed by a sensor during a sleep period of the user; obtain, based on the evaluation result, a numerical value; obtain an adjusted numerical value based on awake information acquired during an awake period of the user; and control a video game based at least in part on the adjusted numerical value. ### Claim 21 An information processing method for executing on an information processing system that executes a game application, the method comprising: obtaining, based on detecting an awakening of a user, an evaluation result based on sleep information sensed by a sensor during a sleep period of the user; obtaining, based on the evaluation result, a numerical value; obtaining an adjusted numerical value based on awake information acquired during an awake period of the user; and controlling a video game based at least in part on the adjusted numerical value. ### Claim 22 (depends on 1) The information processing system according to claim 1, wherein the video game is automatically executed based on a user in-bed detection and a character of the video game is controlled based at least in part on the adjusted numerical value at a time of the user in-bed detection. ### Claim 23 An electronic device, comprising: a memory for storing instructions; and one or more processors in communication with the memory, wherein the instructions, when executed, configure the one or more processors to: obtain, based on detecting an awakening of a user, an evaluation result based on sleep information sensed by a sensor during a sleep period of the user; obtain, based on the evaluation result, a numerical value; obtain an adjusted numerical value based on awake information acquired during an awake period of the user; and control a video game based at least in part on the adjusted numerical value. ### Claim 24 (depends on 23) The electronic device according to claim 23, wherein the adjusted numerical value corresponds a characteristic of an object of the video game. ### Claim 25 (depends on 23) The electronic device according to claim 23, wherein the adjusted numerical value is obtained from a server. ### Claim 26 (depends on 23) The electronic device according to claim 23, wherein the adjusted numerical value is obtained by the electronic device calculating the adjusted numerical value. ### Claim 27 (depends on 23) The electronic device according to claim 23, wherein the video game is automatically executed based on a user in-bed detection. ### Claim 28 (depends on 23) The electronic device according to claim 23, wherein the awake information comprises activity information of the user. ### Claim 29 (depends on 23) The electronic device according to claim 23, wherein the awake information comprises environment information about a surrounding environment of the user. ### Claim 30 (depends on 23) The electronic device according to claim 23, wherein the awake information comprises emotion information of the user. ### Claim 31 (depends on 23) A system comprising: an electronic device according to claim 23; and a server for calculating the adjusted numerical value, wherein the electronic device obtains the adjusted numerical value from the server.