# US 12,118,151 B2 · Haptic feedback device and method for providing haptic sensation based on video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,151 B2 - Document ID: US12118151B2 - Status: Granted - Kind: B2 - Application number: 18/481,013 - Filed: 2023-10-04 - Published: 2024-01-25 - Granted: 2024-10-15 - Earliest claimed: 2017-09-20 - Applicants: Niki Mani; Alex Hamid Mani - Inventors: Niki Mani; Alex Hamid Mani - Web page: https://patentgrep.com/patent/US12118151B2 ## CPC - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - A63F 13/285: Generating tactile feedback signals via the game input device, e.g. force feedback - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G09B 21/00: Teaching, or communicating with, the blind, deaf or mute (audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B) - G09B 21/003: {using tactile presentation of the information, e.g. Braille displays} - G09B 21/007: {using both tactile and audible presentation of the information} - H04N 21/42204: {User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor (constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205)} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 5/144: {Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A device for providing haptic sensation includes a haptic feedback interface. A plurality of videos each recording a real time or near-real time event in a three-dimensional real-world area from a different viewing angle is received and a user input via a mode selector in form of a gesture or a physical touch input is received to indicate a selection of an image-capture device is received. A plurality of different motion of a plurality of objects in an upcoming scene of a captured video is detected based on a look-ahead buffer of the video. A plurality of different haptic cues that includes the one or more movable haptic cues is generated based on the determined haptic feedback. Further, movement of the one or more movable haptic cues is controlled on the haptic feedback interface such that an inter-movement among the plurality of objects is discernible by tactioception. ## Claims ### Claim 1 A haptic feedback device for providing haptic sensation, comprising: a haptic feedback interface that comprises a plurality of haptic elements; a processing circuitry configured to: receive a plurality of videos from a plurality of image-capture devices, wherein the plurality of image-capture devices has different field-of-views and captures the plurality of videos to record a real time or near-real time event in a three-dimensional (3D) real-world area from different viewing angles; receive a first user input via a mode selector in form of at least one of a gesture or a physical touch input on the mode selector, to indicate a selection of a first image-capture device from the plurality of image-capture devices, wherein the selected first image-capture device captures a first video of the plurality of videos, the first video of the real time or near-real time event from a specific viewing angle that a user prefers to experience; receive a look-ahead buffer of an upcoming scene of a real time or near-real time of the first video; detect a plurality of different motion associated with a plurality of objects in the upcoming scene based on the look-ahead buffer of the upcoming scene; and a haptic feedback generator configured to: determine a haptic feedback for the upcoming scene of the first video based on the look-ahead buffer of the first video and the detected plurality of different motion associated with the plurality of objects; generate a plurality of different haptic cues that includes one or more movable haptic cues, based on the determined haptic feedback; and control movement of the one or more movable haptic cues generated on the haptic feedback interface such that an inter-movement among the plurality of objects is discernible by tactioception. ### Claim 2 (depends on 1) The haptic feedback device according to claim 1, wherein the processing circuitry is further configured to detect a plurality of different objects in the upcoming scene of the first video, based on the look-ahead buffer of the first video. ### Claim 3 (depends on 1) The haptic feedback device according to claim 1, wherein the real time or near-real time event corresponds to an ongoing sports event. ### Claim 4 (depends on 1) The haptic feedback device according to claim 1, wherein the real time or near-real time event corresponds to an outside scene of a vehicle and each image-capture device of the plurality of image-capture devices captures the outside scene as visible from different display windows of the vehicle. ### Claim 5 (depends on 4) The haptic feedback device according to claim 4, wherein the different display windows of the vehicle include a left-side front display window, a right-side front display window, a left-side rear display window, a right-side rear display window, a front windshield, or a rear windshield. ### Claim 6 (depends on 1) The haptic feedback device according to claim 1, wherein the mode selector is configured to recognize a gesture of a user as the first user input for selecting the first image-capture device. ### Claim 7 (depends on 1) The haptic feedback device according to claim 1, wherein the haptic feedback generator is further configured to selectively reproduce on the haptic feedback interface, a current scene of the first video captured by the selected first image-capture device. ### Claim 8 (depends on 1) The haptic feedback device according to claim 1, wherein the processing circuitry is further configured to receive, after the first user input, a second user input that indicates a selection of a second image-capture device from the plurality of image-capture devices, wherein the selected second image-capture device captures a second video of the plurality of videos. ### Claim 9 (depends on 8) The haptic feedback device according to claim 8, wherein the haptic feedback generator is further configured to switch from a current scene of the first video to selectively reproduce on the haptic feedback interface a current scene of the second video captured by the selected second image-capture device based on the second user input. ### Claim 10 (depends on 9) The haptic feedback device according to claim 9, wherein the one or more movable haptic cues are generated on the haptic feedback interface using the plurality of haptic elements in synchronization with the current scene of the first video. ### Claim 11 (depends on 10) The haptic feedback device according to claim 10, wherein the haptic feedback generator is further configured to control a relative positioning of the plurality of different haptic cues on the haptic feedback interface to selectively reproduce the current scene of the first video on the haptic feedback interface. ### Claim 12 (depends on 11) The haptic feedback device according to claim 11, wherein the haptic feedback generator is further configured to control deformation of a haptic cue of the generated plurality of different haptic cues on the haptic feedback interface such that an intra-movement of an object of the plurality of objects is discernible by the tactioception. ### Claim 13 (depends on 1) The haptic feedback device according to claim 1, further comprising a network interface to receive the look-ahead buffer of the upcoming scene of the first video in real time or near-real time from the first image-capture device communicatively coupled to the haptic feedback device. ### Claim 14 (depends on 1) The haptic feedback device according to claim 1, wherein the processing circuitry is further configured to identify an object-type of each of the plurality of objects in the upcoming scene based on the look-ahead buffer of the first video. ### Claim 15 A method for providing haptic sensation, comprising: in a haptic feedback device that comprises at least a haptic feedback interface that includes a plurality of haptic elements: receiving a plurality of videos from a plurality of image-capture devices, wherein the plurality of image-capture devices has different field-of-views and captures the plurality of videos to record a real time or near-real time event in a three-dimensional (3D) real-world area from different viewing angles; receiving a first user input via a mode selector in form of at least one of a gesture or a physical touch input on the mode selector, to indicate a selection of a first image-capture device from the plurality of image-capture devices, wherein the selected first image-capture device captures a first video of the plurality of videos, the first video of the real time or near-real time event from a specific viewing angle that a user prefers to experience; detecting a plurality of different motion associated with a plurality of objects in an upcoming scene of the first video based on a look-ahead buffer of the first video; determining a haptic feedback for the upcoming scene of the first video based on the look-ahead buffer of the first video and the detected plurality of different motion associated with the plurality of objects; generating a plurality of different haptic cues that includes one or more movable haptic cues based on the determined haptic feedback; and controlling movement of the one or more movable haptic cues generated on the haptic feedback interface such that an inter-movement among the plurality of objects is discernible by tactioception. ### Claim 16 (depends on 15) The method according to claim 15, further comprising detecting a plurality of different objects in the upcoming scene of the first video, based on the look-ahead buffer of the first video. ### Claim 17 (depends on 15) The method according to claim 15, wherein the real time or near-real time event corresponds to an ongoing sports event. ### Claim 18 (depends on 15) The method according to claim 15, wherein the real time or near-real time event corresponds to an outside scene of a vehicle and each image-capture device of the plurality of image-capture devices captures the outside scene as visible from different display windows of the vehicle. ### Claim 19 (depends on 15) The method according to claim 15, further comprising receiving after the first user input, a second user input that indicates a selection of a second image-capture device from the plurality of image-capture devices, wherein the selected second image-capture device captures a second video of the plurality of videos. ### Claim 20 (depends on 19) The method according to claim 19, further comprising switching from a current scene of the first video to selectively reproduce on the haptic feedback interface a current scene of the second video captured by the selected second image-capture device based on the second user input.