# US 2017/0257664 A1 · Method, electronic device, and system for immersive experience > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0257664 A1 - Document ID: US20170257664A1 - Status: Published application - Kind: A1 - Application number: 14/910,779 - Filed: 2016-02-02 - Published: 2017-09-07 - Earliest claimed: 2016-02-02 - Assignees: Telefonaktiebolaget LM Ericsson (publ) - Applicants: Telefonaktiebolaget LM Ericsson (publ) - Inventors: Kwan To TAM; Leijun YANG; Xiaoli TONG - Web page: https://patentgrep.com/patent/US20170257664A1 ## CPC - H04N 21/436: Interfacing a local distribution network, e.g. communicating with another STB {or one or more peripheral devices} inside the home - H04L 12/2834: {Switching of information between an external network and a home network (access arrangements H04L12/2856)} - H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream - H04N 21/44: Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs - H04N 21/6332: directed to client - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract The present disclosure proposes a system, a method, and electronic devices for providing a user with immersive experience comprising special effects (SFX) during playback of a media stream. The system may comprise a content delivery server, an IoT gateway, and at least one IoT device. The content delivery server transmits the media stream comprising an SFX instruction. The IoT gateway monitors the media stream from the content delivery server; determine the SFX instruction from the media stream; determine, according to the SFX instruction, at least one IoT device which is able to provide at least one special effect corresponding to the SFX instruction; and transmit to the at least one IoT device an IoT device specific command corresponding to the SFX instruction. The at least one IoT device receives the IoT device specific command from the IoT gateway; and provide the special effect according to the command. ## Claims ### Claim 1 A system for providing Special Effect (SFX) during a media stream, the system comprising: a content delivery server adapted to transmit the media stream comprising an SFX instruction; an Internet of Things (IoT) gateway adapted to monitor the media stream from the content delivery server, determine the SFX instruction from the media stream, determine, according to the SFX instruction, at least one IoT device that is able to provide at least one special effect corresponding to the SFX instruction, and transmit to the at least one IoT device an IoT device specific command corresponding to the SFX instruction; and the at least one IoT device adapted to receive the IoT device specific command from the IoT gateway, and provide the special effect according to the IoT device specific command. ### Claim 2 A method, performed by an Internet of Things (IoT) gateway, for controlling at least one IoT device, the method comprising: determining a Special Effect (SFX) instruction from a media stream; determining, according to the SFX instruction, at least one IoT device that is able to provide at least one special effect corresponding to the SFX instruction; and transmitting to the at least one IoT device an IoT device specific command corresponding to the SFX instruction. ### Claim 3 (depends on 2) The method according to claim 2, wherein the SFX instruction is coded into the media stream as a track, and wherein the determination of the SFX instruction comprises: extracting the coded SFX instruction track from the media stream; and decoding the SFX instruction from the coded SFX instruction track. ### Claim 4 (depends on 2) The method according to claim 2, wherein the SFX instruction is determined by an algorithm for determining control parameters from media content in the media stream. ### Claim 5 (depends on 2) The method according to claim 2, wherein the SFX instruction comprises at least one of: action, action parameter, time for action, duration of action, capability, and location. ### Claim 6 (depends on 2) The method according to claim 2, wherein the at least one IoT device is pre-registered in a registration list at the IoT gateway, and wherein the determination of at least one IoT device that is able to provide at least one special effect comprises: searching, according to the SFX instruction, the registration list for the at least one IoT device that is able to provide at least one special effect corresponding to the SFX instruction. ### Claim 7 (depends on 2) The method according to claim 2, wherein the at least one IoT device comprises at least one of: a home appliance, a computer, and a haptic feedback device. ### Claim 8 (depends on 7) The method according to claim 7, wherein the home appliance comprises at least one of: a television, an audio system, an air conditioner, a heater, and a light. ### Claim 9 (depends on 7) The method according to claim 7, wherein the haptic feedback device comprises a haptic/actuated garment, an electronic bracelet, a furniture with haptic feedback capabilities. ### Claim 10 (depends on 9) The method according to claim 9, wherein the haptic/actuated garment comprises at least one of haptic/actuated gloves and shoes, and the furniture with haptic feedback capabilities comprises at least one of a chair and a sofa. ### Claim 11 (depends on 2) The method according to claim 2, wherein the transmission of the IoT device specific command comprises: determining, based on the SFX instruction, the IoT device specific command from a translation table which is specific to the at least one IoT device; and transmitting the IoT device specific command to the at least one IoT device. ### Claim 12 (depends on 5) The method according to claim 5, wherein the IoT device specific command is transmitted to the at least one IoT device by the IoT gateway in a way such that the action will be performed at the time for action. ### Claim 13 (depends on 12) The method according to claim 12, wherein the IoT device specific command is transmitted to the at least one IoT device by the IoT gateway before the time for action and the at least one IoT device will perform the action at the time for action. ### Claim 14 (depends on 12) The method according to claim 12, wherein the IoT device specific command is transmitted to the at least one IoT device by the IoT gateway at the time for action and the at least one IoT device will perform the action immediately. ### Claim 15 (depends on 2) The method according to claim 2, further comprising: receiving from the at least one IoT device a command execution result in response to the IoT device specific command. ### Claim 16 (depends on 2) The method according to claim 2, further comprising: receiving information about user intervention from the at least one IoT device. ### Claim 17 (depends on 15) The method according to claim 15, further comprising: transmitting feedback comprising the command execution result to a content delivery server. ### Claim 18 (canceled) ### Claim 19 An electronic device for controlling at least one Internet of Things (IoT) device, the electronic device comprising: a processor; and a memory containing instructions which, when executed by the processor, causes the processor to: determine a Special Effect (SFX) instruction from a media stream, determine, according to the SFX instruction, at least one IoT device that is able to provide at least one special effect corresponding to the SFX instruction, and transmit to the at least one IoT device an IoT device specific command corresponding to the SFX instruction. ### Claim 20 A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a processor, causes the processor (506) to perform operations by an Internet of Things (IoT), the operations comprising: determining a Special Effect (SFX) instruction form a media stream; determining, according to the SFX instruction, at least one IoT device that is able to provide at least one special effect corresponding to the SFX instruction; and transmitting to the at least one IoT device an IoT device specific command corresponding to the SFX instruction. ### Claim 21 (canceled) ### Claim 22 (canceled) ### Claim 23 A method, performed by an Internet of Things (IoT) device, for providing a Special Effect (SFX), the method comprising: receiving an IoT device specific command corresponding to an SFX instruction that is determined from a media stream; and executing the IoT device specific command to provide the SFX. ### Claim 24 (depends on 23) The method according to claim 23, wherein the IoT device is pre-registered in a registration list at an IoT gateway. ### Claim 25 (depends on 23) The method according to claim 23, further comprising: transmitting to an IoT gateway a command execution result in response to the IoT device specific command. ### Claim 26 (depends on 23) The method according to claim 23, further comprising: transmitting information about user intervention to an IoT gateway. ### Claim 27 (depends on 23) The method according to claim 23, wherein the IoT device maintains its SFX-related information comprising one or more of: device ID, device type, device location, resource availability, resource permission, and resource capability. ### Claim 28 (depends on 27) The method according to claim 27, further comprising: reporting the SFX-related information to an IoT gateway. ### Claim 29 (canceled) ### Claim 30 An electronic device for providing a Special Effect (SFX), the electronic device comprising: a processor; and a memory containing instructions which, when executed by the processor, causes the processor to: receive an Internet of Things (IoT) device specific command corresponding to an SFX instruction which is determined from a media stream; and execute the IoT device specific command to provide the SFX. ### Claim 31 A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a processor, causes the processor to perform operations an Internet of Things (IoT) device for providing a Special Effect (SFX), the operations comprising: receiving an IoT device specific command corresponding to an SFX instruction that is determined from a media stream; and executing the IoT device specific command to provide the SFX. ### Claim 32 (canceled) ### Claim 33 (canceled) ### Claim 34 (depends on 16) The method according to claim 16, further comprising: transmitting feedback comprising the information about user intervention to a content delivery server.