# US 12,217,378 B2 · System and method for video-captured initialization of peripheral devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,378 B2 - Document ID: US12217378B2 - Status: Granted - Kind: B2 - Application number: 18/072,886 - Filed: 2022-12-01 - Published: 2023-11-30 - Granted: 2025-02-04 - Earliest claimed: 2018-11-20 - Applicants: Whirlwind Vr, Inc. - Inventors: Matthew Ryan Gilg; Timothy Chia-Chieh Sun - Web page: https://patentgrep.com/patent/US12217378B2 ## CPC - A63F 13/26: having at least one additional display device, e.g. on the game controller or outside a game booth - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) - A63F 13/214: for locating contacts on a surface, e.g. floor mats or touch pads - A63F 13/235: using a wireless connection, e.g. infrared or piconet - A63F 13/24: Constructional details thereof, e.g. game controllers with detachable joystick handles - A63F 13/28: responding to control signals received from the game device for affecting ambient conditions, e.g. for vibrating players' seats, activating scent dispensers or affecting temperature or light - A63F 13/285: Generating tactile feedback signals via the game input device, e.g. force feedback - A63F 13/52: involving aspects of the displayed game scene - A63F 13/54: involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall - A63F 13/98: Accessories, i.e. detachable arrangements optional for the use of the video game device, e.g. grip supports of game controllers - G06F 1/20: Cooling means - G06F 1/206: {comprising thermal management} - G06F 1/3231: Monitoring the presence, absence or movement of users - G06F 1/3287: by switching off individual functional units in the computer system - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials - G06V 10/56: relating to colour - G06V 10/764: using classification, e.g. of video objects - G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60) - G08B 6/00: Tactile signalling systems, e.g. tactile personal calling systems - G08B 7/06: using electric transmission {, e.g. involving audible and visible signalling through the use of sound and light sources} - H05B 47/165: following a pre-assigned programmed sequence; Logic control [LC] - A63F 13/424: involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition - A63F 13/53: involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game - G06F 13/102: {where the program performs an interfacing function, e.g. device driver (G06F13/105 takes precedence; contention policies within device drivers G06F9/4881; scheduling within device drivers G06F9/52)} - G06T 2219/2004: Aligning objects, relative positioning of parts - G06V 2201/07: Target detection - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - H05B: ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract A method for an image-captured initialization of an array of devices, said method comprising the steps of: receiving a video stream featuring a plurality of end devices in a gaming station, wherein said end device renders at least one of a lighting, haptic, or air-blowing effect from a specific location within the gaming station; detecting the plurality of end devices by vision processing at least a frame of the gaming station stream; and positioning a virtual representation of the detected end devices on a digital canvas representing a user's physical space based on a tonal frequency-dependent light-emission from a frequency-defined end device. ## Claims ### Claim 1 A method for an image-captured initialization of an array of devices, said method implemented by a processor comprising the steps of: receiving a video stream featuring a plurality of end devices in a gaming station, wherein said end device renders at least one of a lighting, haptic, or air-blowing effect from a specific location within the gaming station; detecting, by the processor, the plurality of end devices by vision processing at least a frame of the gaming station stream; and positioning a virtual representation of the detected end devices on a digital canvas representing a user's physical space based on a tonal frequency-dependent light-emission from a frequency-defined end device. ### Claim 2 (depends on 1) The method of claim 1, wherein the video stream received is from any video-capture device of at least a single perspective of the user's game station. ### Claim 3 (depends on 1) The method of claim 1, wherein the plurality of end devices are configured in an array rendering an array effect during a user experience. ### Claim 4 (depends on 3) The method of claim 3, wherein the end device is at least one of a device with a vibro-tactile or light emitting function. ### Claim 5 (depends on 4) The method of claim 4, wherein the end devices with light-emitting function are at least one of a light-emitting gaming device, including a display/monitor, mouse, game controller, head-set, computing tower, game console, any portion and/or component of a computing tower, light-strip, individual bulb or diode in a lighting device. ### Claim 6 (depends on 1) The method of claim 1, wherein positioning the end device is validated by playing a predetermined tonal frequency associated with a respective end device for light confirmation of the end device. ### Claim 7 (depends on 1) The method of claim 1, wherein positioning the end device is validated by playing a predetermined plurality of tonal frequencies associated with a respective end device for light confirmation of the end device. ### Claim 8 (depends on 1) The method of claim 1, wherein the vision processing is achieved by at least one of a color-thresholding, blob detection, or object detection technique. ### Claim 9 (depends on 1) The method of claim 1, further comprising a visual preview of the array effects based on variable virtual positioning of devices on the digital canvas. ### Claim 10 A method for an image-captured initialization of an array of devices, said method implemented by a processor comprising the steps of: receiving a video stream featuring a plurality of end devices in a gaming station, wherein said end device renders at least one of a lighting, haptic, or air-blowing effect from a specific location within the gaming station; and positioning a virtual representation of the detected end devices on a digital canvas representing a user's physical space based on a tonal frequency-dependent light-emission from a frequency-defined end device; and modulating an effect on the end device corresponding to the digital canvas and scene triggers to render an effect from the array of devices. ### Claim 11 (depends on 10) The method of claim 10, wherein positioning the end device is validated by playing a predetermined tonal frequency associated with a respective end device for light confirmation. ### Claim 12 (depends on 10) The method of claim 10, wherein positioning the end device is validated by playing a predetermined plurality of tonal frequencies associated with a respective end device for light confirmation. ### Claim 13 (depends on 10) The method of claim 10, wherein the vision processing is achieved by at least one of a color-thresholding, blob detection, or object detection technique. ### Claim 14 (depends on 10) The method of claim 10, further comprising a visual preview of the array effects based on variable virtual positioning of devices on the digital canvas. ### Claim 15 A system for an image-captured initialization of an array of devices, said system comprising: a processor; a memory element with instructions coupled to said processor, when implemented, configures said processor to: receive a video stream featuring a plurality of end devices in a gaming station, wherein said end device renders at least one of a lighting, haptic, or air-blowing effect from a specific location within the gaming station; and position a virtual representation of the detected end devices on a digital canvas representing a user's physical space based on a tonal frequency-dependent light-emission from a frequency-defined end device. ### Claim 16 (depends on 15) The system of claim 15, wherein the vision processing is achieved by at least one of a color-thresholding, blob detection, or object detection technique. ### Claim 17 (depends on 15) The system of claim 15, further comprising a visual preview of the array effects based on variable virtual positioning of devices on the digital canvas. ### Claim 18 (depends on 15) The system of claim 15, wherein positioning the end device is validated by playing a predetermined tonal frequency associated with a respective end device for light confirmation. ### Claim 19 (depends on 15) The system of claim 15, wherein the virtual representation of the detected end device is a digital twin of the end device, wherein the twin may be manipulated to cause an effect on the end device, or vice versa. ### Claim 20 (depends on 15) The system of claim 15, wherein the arrayed devices are configured across a public scape, indoors and/or outdoors, for rippling a scaped effect based on the users grabbed screen.