# US 12,223,578 B2 · System and method for controlling effects on live video stream and for controlling accessories via live video stream interface > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,578 B2 - Document ID: US12223578B2 - Status: Granted - Kind: B2 - Application number: 18/541,028 - Filed: 2023-12-15 - Published: 2024-10-10 - Granted: 2025-02-11 - Earliest claimed: 2023-04-06 - Assignees: Hytto Pte. Ltd. - Applicants: Hytto Pte. Ltd. - Inventors: Dan Liu; Jilin Qiu - Web page: https://patentgrep.com/patent/US12223578B2 ## CPC - G06T 13/80: Two-dimensional [2D] animation, e.g. using sprites - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34) - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06T 13/20: Three-dimensional [3D] animation - G06T 13/40: of characters, e.g. humans, animals or virtual beings - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - H04N 21/2187: Live feed - H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04N 21/4781: {Games} - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method for controlling effects on a live video stream includes providing a video image from a content creator device and displaying the video image on a first viewer device. A selection of a sticker graphic is received on the first viewer device. A position of a user input operation on the displayed video image is determined. The determined position of the user input operation on the displayed video image is converted to a position of the user input operation on the video image. The selected sticker graphic is applied to the video image at the position of the user input operation on the video image. The applied sticker graphic is displayed along with the video image on at least the first viewer device. ## Claims ### Claim 1 A method for controlling accessories within a live video stream, comprising: registering one or more accessories to a content creator device, wherein the content creator device is configured to initiate a live feed video broadcast to at least one viewer device; locating the one or more accessories within the live feed video broadcast; receiving a control gesture from a viewer of the at least one viewer device; determining a location range of the control gesture on the at least one viewer device; determining whether the location range of the control gesture falls within a location range of the one or more accessories within the live feed video broadcast; and when it is determined that the location of the control gesture falls within the location range of the one or more accessories within the live feed video broadcast, controlling an operation of the one or more accessories in accordance with the control gesture using information obtained as part of the registration of the one or more accessories. ### Claim 2 (depends on 1) The method of claim 1, further comprising: converting the determined location from the at least one viewer device to a location on the live feed video broadcast, wherein determining whether the location range of the control gesture falls within the location range of the one or more accessories includes determining whether the location of the control gesture on the live feed video broadcast falls within the location range of the one or more accessories within the live feed video broadcast. ### Claim 3 (depends on 1) The method of claim 1, wherein the control gesture is a touch gesture or a user input by: keyboard, mouse, trackpad, trackball, or stylus. ### Claim 4 (depends on 1) The method of claim 1, wherein upon controlling an operation of the one or more accessories in accordance with the control gesture, the operation of the one or more accessories is filmed within the live video stream and streamed back to the at least one viewer device. ### Claim 5 (depends on 1) The method of claim 1, wherein the registering of the one or more accessories to the content creator device includes supplying an image or QR code associated with the one or more accessories and locating the one or more accessories within the live feed video broadcast includes using computer vision processing to identify the supplied image or QR code within the live feed video broadcast. ### Claim 6 (depends on 1) The method of claim 1, wherein an ability to provide the control gesture is gated by a performance of a tipping operation of at least a predetermined value. ### Claim 7 (depends on 1) The method of claim 1, wherein the live feed video broadcast is a content creator live feed video broadcast and wherein the controlling of the operation of the one or more accessories is observable within the content creator live feed video broadcast. ### Claim 8 (depends on 1) The method of claim 1, wherein a coordinate system of the live feed video broadcast is different from a coordinate system of the at least one viewer device. ### Claim 9 A system for controlling accessories within a live video stream, comprising: one or more accessories registering to a content creator device; and at least one viewer device initiating a live feed video broadcast, wherein the one or more accessories are located within the live feed video broadcast, wherein the at least one viewer device receives a control gesture from a viewer of the at least one viewer device, wherein a location range of the control gesture on the at least one viewer device is determined, wherein the determined location from the at least one viewer device is converted device to a location on the live feed video broadcast, wherein it is determined whether the location of the control gesture on the live feed video broadcast falls within a location range of the one or more accessories within the live feed video broadcast, and wherein when it is determined that the location of the control gesture on the live feed video broadcast falls within the location range of the one or more accessories within the live feed video broadcast, an operation of the one or more accessories is controlled in accordance with the control gesture using information obtained as part of the registration of the one or more accessories. ### Claim 10 (depends on 9) The system of claim 9, wherein the control gesture is a touch gesture or a user input by: keyboard, mouse, trackpad, trackball, or stylus. ### Claim 11 (depends on 9) The system of claim 9, wherein upon controlling an operation of the one or more accessories in accordance with the touch control gesture, the operation of the one or more accessories is filmed within the live video stream and streamed back to the at least one viewer device. ### Claim 12 (depends on 9) The system of claim 9, wherein the registering of the one or more accessories to the content creator device includes supplying an image or QR code associated with the one or more accessories and locating the one or more accessories within the live feed video broadcast includes using computer vision processing to identify the supplied image or QR code within the live feed video broadcast. ### Claim 13 (depends on 9) The system of claim 9, wherein an ability to provide the control gesture is gated by a performance of a tipping operation of at least a predetermined value. ### Claim 14 (depends on 9) The system of claim 9, wherein the live feed video broadcast is a content creator live feed video broadcast and wherein the controlling of the operation of the one or more accessories is observable within the content creator live feed video broadcast. ### Claim 15 (depends on 9) The system of claim 9, wherein a coordinate system of the live feed video broadcast is different from a coordinate system of the at least one viewer device. ### Claim 16 A non-transitory computer-readable storage medium, comprising: machine-readable instructions, wherein the machine-readable instructions when executed by a processor of a controller, enable the controller to: register one or more accessories to a content creator device; initiate a live feed video broadcast to at least one viewer device; locate the one or more accessories within the live feed video broadcast; receive a control gesture from a viewer of the at least one viewer device; determine a location range of the control gesture on the at least one viewer device; convert the determined location from the at least one viewer device to a location on the live feed video broadcast; determine whether the location of the control gesture on the live feed video broadcast falls within a location range of the one or more accessories within the live feed video broadcast; and when it is determined that the location of the control gesture on the live feed video broadcast falls within the location range of the one or more accessories within the live feed video broadcast, control an operation of the one or more accessories in accordance with the touch control gesture using information obtained as part of the registration of the one or more accessories. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the control gesture is a touch gesture or a user input by: keyboard, mouse, trackpad, trackball, or stylus. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein upon controlling an operation of the one or more accessories in accordance with the control gesture, the operation of the one or more accessories is filmed within the live video stream and streamed back to the at least one viewer device. ### Claim 19 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the registering of the one or more accessories to the content creator device includes supplying an image or QR code associated with the one or more accessories and locating the one or more accessories within the live feed video broadcast includes using computer vision processing to identify the supplied image or QR code within the live feed video broadcast. ### Claim 20 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein an ability to provide the control gesture is gated by a performance of a tipping operation of at least a predetermined value. ### Claim 21 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the live feed video broadcast is a content creator live feed video broadcast and wherein the controlling of the operation of the one or more accessories is observable within the content creator live feed video broadcast. ### Claim 22 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein a coordinate system of the live feed video broadcast is different from a coordinate system of the at least one viewer device.