# US 2012/0075463 A1 · User interface system and method using thermal imaging > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2012/0075463 A1 - Document ID: US20120075463A1 - Status: Published application - Kind: A1 - Application number: 12/969,501 - Filed: 2010-12-15 - Published: 2012-03-29 - Earliest claimed: 2010-09-23 - Assignees: Sony Computer Entertainment Inc. - Inventors: RUXIN CHEN; Steven Osman - Web page: https://patentgrep.com/patent/US20120075463A1 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G01S 11/12: using electromagnetic waves other than radio waves - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints - H04N 23/23: from thermal infrared radiation - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A thermal imaging interface for control of a computer program may obtain one or more thermal infrared images of one or more objects with one or more thermographic cameras. The images may be analyzed to identify one or more characteristics of the objects. Such characteristics may be used as a control input in the computer program. ## Claims ### Claim 1 A thermal imaging interface method for control of a computer program, comprising: a) obtaining one or more thermal infrared images of one or more objects with one or more thermographic cameras; b) analyzing the one or more thermal infrared images; c) identifying one or more characteristics of the one or more objects from the one or more thermal infrared images; and d) using the one or more characteristics as a control input in the computer program. ### Claim 2 (depends on 1) The method of claim 1, wherein the one or more objects include one or more users. ### Claim 3 (depends on 2) The method of claim 2, wherein the one or more characteristics in c) include one or more vital signs for the one or more of the users determined by measuring a temperature or temperature change of the one or more users. ### Claim 4 (depends on 3) The method of claim 3, wherein the one or more vital signs is used to control a pace of actions occurring during execution of the program. ### Claim 5 (depends on 4) The method of claim 4, wherein the pace of actions includes a tempo of music. ### Claim 6 (depends on 3) The method of claim 3, wherein the one or more vital signs is used to control one or more graphics associated with the computer program. ### Claim 7 (depends on 3) The method of claim 3, wherein the one or more vital signs is used to measure a performance of the one or more users in conjunction with the computer program. ### Claim 8 (depends on 3) The method of claim 3, wherein the one or more vital signs is used to distinguish living objects from inanimate objects. ### Claim 9 (depends on 3) The method of claim 3, wherein the one or more vital signs is used to control a response speed of one or more user interfaces associated with the computer program. ### Claim 10 (depends on 3) The method of claim 3 wherein the one or more vital signs is mapped to trigger an action of a virtual object. ### Claim 11 (depends on 3) The method of claim 3 wherein the one or more vital signs is sampled and used by the computer program to generate a sound. ### Claim 12 (depends on 3) The method of claim 3, wherein the one or more vital signs is a heart beat rate, respiration rate, body temperature, or blood pressure. ### Claim 13 (depends on 12) The method of claim 12, further comprising analyzing the one or more vital signs and determining a health condition of a user from results of analysis of the vital signs. ### Claim 14 (depends on 1) The method of claim 1, wherein the one or more characteristics in c) include a respiration pattern for one or more users determined by measuring the airflow and temperature change around the nose and mouth of the one or more users. ### Claim 15 (depends on 14) The method of claim 14, wherein the respiration pattern for the one or more users is used to identify a source of sounds originating amongst the one or more users. ### Claim 16 (depends on 14) The method of claim 14, wherein the respiration pattern for the one or more users is used to determine whether the one or more of the users is an authorized user. ### Claim 17 (depends on 1) The method of claim 1, wherein the one or more characteristics in c) include a depth of the one or more objects in relation to the one or more thermographic cameras determined by infrared reflection coupled with interpolation using the one or more thermal infrared images. ### Claim 18 (depends on 17) The method of claim 17, wherein d) includes using the one or more characteristics for each of the one or more users in combination with one or more normal images of the one or more users to track the movement of one or more body parts associated with the one or more users. ### Claim 19 (depends on 1) The method of claim 1, wherein the one or more characteristics in c) include a segmentation cue that distinguishes the one or more objects from their background. ### Claim 20 An apparatus for control of a computer program, comprising: one or more thermographic cameras; one or more processors coupled to the one or more thermographic cameras; and instructions executable by the one or more processors configured to: a) obtain one or more thermal infrared images of one or more objects with the one or more thermographic cameras; b) analyze the one or more thermal infrared images; c) identify one or more characteristics of the one or more objects from the one or more thermal infrared images; d) use the one or more characteristics as a control input for a computer program. ### Claim 21 (depends on 20) The apparatus of claim 20, wherein the one or more objects include one or more users. ### Claim 22 (depends on 21) The apparatus of claim 21, wherein the one or more characteristics in c) include a one or more vital signs for the one or more users determined by measuring a temperature or temperature change of the one or more users. ### Claim 23 (depends on 20) The apparatus of claim 20, wherein the one or more characteristics in c) include a respiration rate for each of the one or more users determined by measuring the airflow and temperature change around the nose and mouth of the one or more users. ### Claim 24 (depends on 20) The apparatus of claim 20, wherein the one or more characteristics in c) include a depth of the one or more objects in relation to the one or more thermographic cameras determined by infrared reflection coupled with interpolation using the one or more thermal infrared images. ### Claim 25 (depends on 20) The apparatus of claim 20, wherein the one or more thermographic cameras include a thermographic camera mounted to a video display that is coupled to the processor. ### Claim 26 (depends on 20) The apparatus of claim 20, wherein the one or more processors include one or more dedicated processors, wherein each of the one or more thermographic cameras is coupled to a corresponding dedicated processor. ### Claim 27 (depends on 26) The apparatus of claim 26, wherein b), c) and d) are implemented by the one or more dedicated processors. ### Claim 28 A computer program product, comprising: a non-transitory computer-readable storage medium having computer readable program code embodied in said medium for controlling a computer program using a thermal image interface, said computer program product having: a) computer readable program code means for obtaining one or more thermal infrared images of one or more objects with one or more thermographic cameras; b) computer readable program code means for analyzing the one or more thermal infrared images; c) computer readable program code means for identifying one or more characteristics of the one or more objects from the one or more thermal infrared images; and d) computer readable program code means for using the one or more characteristics as a control input in the computer program.