# US 12,190,914 B2 · System and method for extracting objects from videos in real-time to create virtual situations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,190,914 B2 - Document ID: US12190914B2 - Status: Granted - Kind: B2 - Application number: 18/092,929 - Filed: 2023-01-04 - Published: 2023-07-06 - Granted: 2025-01-07 - Earliest claimed: 2022-01-05 - Assignees: Silverlabs Technologies Inc - Applicants: Lakshminath Reddy Dondeti; Vidya Narayanan - Inventors: Lakshminath Reddy Dondeti; Vidya Narayanan - Web page: https://patentgrep.com/patent/US12190914B2 ## CPC - G11B 27/036: Insert-editing - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} - H04N 5/91: Television signal processing therefor - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Exemplary embodiments of present disclosure are directed towards a system and method for extracting objects from videos in real-time to create virtual situations, comprising a computing device comprises video creating and editing module configured to enable a user to record videos and select frames automatically from the user recorded videos thereby transferring the automatically selected frames from the computing device to a server. The server comprises video processing module configured receive the automatically selected frames thereby detecting and extracting objects from the automatically selected frames and transfer extracted objects to computing device and display the extracted objects to the user. The video creating and editing module configured to place the extracted objects on a new frame automatically and allow the user to reposition extracted objects on new frame and enable the user to customize the background and foreground elements in the new frame to create virtual situations. ## Claims ### Claim 1 A method for extracting objects from videos in real-time to create virtual situations, comprising: enabling a user to record one or more videos by a video creating and editing module on a computing device; selecting one or more frames automatically from the one or more user recorded videos by the video creating and editing module; transferring the one or more automatically selected frames from the computing device to a server by the video creating and editing module over a network; receiving the one or more automatically selected frames by a video processing module enabled in the server; detecting and extracting one or more objects from the one or more automatically selected frames by the video processing module; transferring the one or more extracted objects to the computing device by the video processing module over the network; receiving the one or more extracted objects by the video creating and editing module and displaying the one or more extracted objects to the user on the computing device; placing the one or more extracted objects on a new frame automatically by the video creating and editing module; allowing the user to reposition the one or more extracted objects on the new frame by the video creating and editing module; and enabling the user to customize background and foreground elements in the new frame to create virtual situations by the video creating and editing module. ### Claim 2 (depends on 1) The method of claim 1, comprising a step of enabling the user to access a camera icon to record the one or more videos on the computing device by a video recording module. ### Claim 3 (depends on 1) The method of claim 1, comprising a step of displaying a series of frames to the user from the one or more user recorded videos by a frame selection module. ### Claim 4 (depends on 1) The method of claim 1, comprising a step of allowing the user to select one or more frames from the series of frames displayed on the computing device by the frame selection module. ### Claim 5 (depends on 1) The method of claim 1, comprising a step of transferring the one or more user selected frames from the computing device to the server by the frame selection module. ### Claim 6 (depends on 5) The method of claim 5, comprising a step of receiving the one or more user selected frames from the frame selection module by a frames receiving module. ### Claim 7 (depends on 6) The method of claim 6, comprising a step of detecting and extracting the objects from the user selected frames by the objects extraction and classification module. ### Claim 8 (depends on 7) The method of claim 7, comprising a step of classifying the extracted objects into the appropriate categories by the objects extraction and classification module. ### Claim 9 (depends on 7) The method of claim 7, comprising a step of guiding the user selected one or more frames by inflection points in a background audio track used in the one or more videos. ### Claim 10 (depends on 7) The method of claim 7, comprising a step of guiding the user selected one or more frames by visual changes in the one or more videos (for example, a pause after a dance move). ### Claim 11 (depends on 7) The method of claim 7, comprising a step of transferring the extracted objects to the video creating and editing module by the objects extraction and classification module. ### Claim 12 (depends on 11) The method of claim 11, comprising a step of receiving and displaying the one or more extracted objects to the user by an objects displaying module. ### Claim 13 (depends on 1) The method of claim 1, comprising a step of enabling the user to select a new frame from the one or more user recorded videos by a new frame selection module. ### Claim 14 (depends on 12) The method of claim 12, comprising a step of allowing the user to select the one more extracted objects to transfer into the new frame to create the virtual situation by the objects selection module. ### Claim 15 (depends on 1) The method of claim 1, comprising a step of enabling the user to customize the background and foreground elements of the new frame by a background and foreground customization module. ### Claim 16 (depends on 15) The method of claim 15, comprising a step of enabling the user to upload their own assets while customizing the background and foreground elements. ### Claim 17 (depends on 1) The method of claim 1, comprising a step of enabling the existing user directly to access the video creating and editing module to create the virtual situations. ### Claim 18 (depends on 1) The method of claim 1, comprising a step of enabling the new user to watch a tutorial for creating virtual situations. ### Claim 19 A system for extracting objects from videos in real-time to create virtual situations, comprising: a computing device comprising a processor for executing instructions from a video creating and editing module stored in memory within the computing device, whereby the a video creating and editing module configured to enable a user to record one or more videos; the video creating and editing module configured to select one or more frames automatically from the one or more user recorded videos, whereby the video creating and editing module configured to transfer the one or more automatically selected frames from the computing device to a server over a network; the server comprising a video processing module configured to receive the one or more automatically selected frames, whereby the video processing module configured to detect and extract one or more objects from the one or more automatically selected frames and transfer the one or more extracted objects to the computing device over the network; and the video creating and editing module configured to receive one or more extracted objects and display the one or more extracted objects to the user on the computing device, whereby the video creating and editing module configured to place the one or more extracted objects on a new frame automatically, the video creating and editing module configured to allow the user to reposition the one or more extracted objects on the new frame, the video creating and editing module configured to enable the user to customize background and foreground elements in the new frame to create virtual situations. ### Claim 20 (depends on 19) The system of claim 19, wherein the video creating and editing module configured to display the series of frames to the user from the one or more user recorded videos. ### Claim 21 (depends on 19) The system of claim 19, wherein the video creating and editing module configured to allow the user to select one or more frames from the series of frames displayed on the computing device. ### Claim 22 (depends on 19) The system of claim 19, wherein the video creating and editing module configured to enable the user to select the new frame from the one or more user recorded videos. ### Claim 23 (depends on 19) The system of claim 19, wherein the video creating and editing module configured to allow the user to select and place the one or more extracted objects into the new frame. ### Claim 24 (depends on 19) The system of claim 19, wherein the video creating and editing module configured to delete the one or more user selected frames after extracting the one or more objects and replacing one or more user selected frames with the new frame. ### Claim 25 (depends on 19) The system of claim 19, wherein the video creating and editing module configured to select the one or more frames automatically from the one or more user recorded videos to create virtual situations on the computing device without the cloud server. ### Claim 26 (depends on 19) The system of claim 19, wherein the extracted objects comprises a human body, animals, and other objects in real-time from one or more frames of the video. ### Claim 27 (depends on 19) The system of claim 19, wherein the new frame comprises a movie poster, a movie scene, a movie song ending, movie trailers, video memes, brand interactions, and virtual landscapes. ### Claim 28 A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, said program code including instructions to: enable a user to record one or more videos by a video creating and editing module on a computing device; select one or more frames automatically from the one or more user recorded videos by the video creating and editing module; transfer the one or more automatically selected frames from the computing device to a server by the video creating and editing module over a network; receive the one or more automatically selected frames by a video processing module enabled in the server; detect and extract one or more objects from the one or more automatically selected frames by the video processing module; transfer the one or more extracted objects to the computing device by the video processing module over the network; receive the one or more extracted objects by the video creating and editing module and displaying the one or more extracted objects to the user on the computing device; place the one or more extracted objects on a new frame automatically by the video creating and editing module; allow the user to reposition the one or more extracted objects on the new frame by the video creating and editing module; and enable the user to customize background and foreground elements in the new frame to create virtual situations by the video creating and editing module.