# US 10,375,384 B2 · Interactive imaging systems and methods for motion control by users > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,375,384 B2 - Document ID: US10375384B2 - Status: Granted - Kind: B2 - Application number: 16/047,090 - Filed: 2018-07-27 - Published: 2018-11-29 - Granted: 2019-08-06 - Earliest claimed: 2006-12-19 - Assignees: Cuesta Technology Holdings, LLC - Applicants: Cuesta Technology Holdings, LLC - Inventors: Matthew Flagg; Greg Roberts - Web page: https://patentgrep.com/patent/US10375384B2 ## CPC - H04N 17/002: {for television cameras} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - H04N 23/11: for generating image signals from visible and infrared light wavelengths - G06T 2207/10012: Stereo images - G06T 2207/10048: Infrared image - G06T 2207/10152: Varying illumination - G06T 2207/30196: Human being; Person - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract In various embodiments, the present invention provides a system and associated methods of calibration and use for an interactive imaging environment based on the optimization of parameters used in various segmentation algorithm techniques. These methods address the challenge of automatically calibrating an interactive imaging system, so that it is capable of aligning human body motion, or the like, to a visual display. As such the present invention provides a system and method of automatically and rapidly aligning the motion of an object to a visual display. ## Claims ### Claim 1 An interactive imaging system, comprising: an infrared image sensor configured to detect infrared illumination energy in a field of view; and a control unit communicatively coupled to the infrared image sensor, wherein the control unit is configured to: detect a user in the field of view; create audio/visual signals in response to the detected user; and modify the audio/visual signals in response to the presence and/or motion of the user, thereby providing user interaction in a virtual environment. ### Claim 2 (depends on 1) The interactive imaging system of claim 1, wherein the control unit is further configured to calibrate the infrared image sensor using a segmentation algorithm with parameters and an objective function to optimize the parameters of the segmentation algorithm. ### Claim 3 (depends on 2) (The interactive imaging system of claim 2, wherein the objective function comprises at least one of a gradient descent method and a Levenberg-Marquardt method. ### Claim 4 (depends on 2) The interactive imaging system of claim 2, wherein the segmentation algorithm is at least one of background subtraction and stereo disparity-based foreground detection. ### Claim 5 (depends on 1) The interactive imaging system of claim 1, further comprising one or more illumination energy devices configured to flood the field of view with illumination energy for detection by the infrared image sensor. ### Claim 6 (depends on 5) The interactive imaging system of claim 5, wherein the one or more illumination energy devices and the infrared image sensor are integrally formed and utilize a common illumination energy source. ### Claim 7 (depends on 1) The interactive imaging system of claim 1, wherein the infrared light comprises at least one of i) structured (patterned) infrared light and ii) structured (patterned) and strobed infrared light. ### Claim 8 (depends on 1) The interactive imaging system of claim 1, wherein the infrared image sensor is operable for detecting illumination energy from about 700 nm to about 1,100 nm. ### Claim 9 (depends on 1) The interactive imaging system of claim 1, wherein the infrared image sensor is operable for detecting illumination energy from about 700 nm to about 10,000 nm. ### Claim 10 (depends on 1) The interactive imaging system of claim 1, further comprising an optical filter coupled to the infrared image sensor operable for filtering out visible light. ### Claim 11 (depends on 1) The interactive imaging system of claim 1, wherein the infrared image sensor is utilized by the control unit for calibration. ### Claim 12 (depends on 1) The interactive imaging system of claim 1, wherein the control unit is further configured to detect a body of the user in the field of view and segment the body from a background. ### Claim 13 An interactive imaging method, comprising: providing an infrared image sensor configured to detect infrared illumination energy in a field of view; and providing a control unit communicatively coupled to the infrared image sensor, wherein the control unit is configured to: detect a user in the field of view; create audio/visual signals in response to the detected user; and modify the audio/visual signals in response to the presence and/or motion of the user, thereby providing user interaction in a virtual environment. ### Claim 14 (depends on 13) The interactive imaging method of claim 13, wherein the control unit is further configured to calibrate the infrared image sensor using a segmentation algorithm with parameters and an objective function to optimize the parameters of the segmentation algorithm. ### Claim 15 (depends on 14) The interactive imaging method of claim 14, wherein the objective function comprises at least one of a gradient descent method and a Levenberg-Marquardt method. ### Claim 16 (depends on 14) The interactive imaging method of claim 14, wherein the segmentation algorithm is at least one of background subtraction and stereo disparity-based foreground detection. ### Claim 17 (depends on 13) The interactive imaging method of claim 13, further comprising one or more illumination energy devices configured to flood the field of view with illumination energy for detection by the infrared image sensor. ### Claim 18 (depends on 17) The interactive imaging method of claim 17, wherein the one or more illumination energy devices and the infrared image sensor are integrally formed and utilize a common illumination energy source. ### Claim 19 (depends on 17) The interactive imaging method of claim 17, wherein the infrared light comprises at least one of i) structured (patterned) infrared light and ii) structured (patterned) and strobed infrared light. ### Claim 20 (depends on 13) The interactive imaging method of claim 13, wherein the infrared image sensor is operable for detecting illumination energy from about 700 nm to about 1,100 nm.