# US 12,217,142 B2 · Using machine learning to detect which part of the screen includes embedded frames of an uploaded video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,142 B2 - Document ID: US12217142B2 - Status: Granted - Kind: B2 - Application number: 18/520,532 - Filed: 2023-11-27 - Published: 2024-03-28 - Granted: 2025-02-04 - Earliest claimed: 2017-01-13 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Filip Pavetic; King Hong Thomas Leung; Dmitrii Tochilkin - Web page: https://patentgrep.com/patent/US12217142B2 ## CPC - G06N 20/00: Machine learning - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/241: relating to the classification model, e.g. parametric or non-parametric approaches - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/764: using classification, e.g. of video objects - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A system and methods are disclosed for using a trained machine learning model to identify constituent images within composite images. A method may include providing data identifying a first image as input to a machine learning model trained using training data identifying a plurality of composite images that each include one or more constituent images, and determining, using one or more outputs of the trained machine learning model, that the first image is a composite image that includes a first constituent image, wherein at least a portion of the first constituent image is in a spatial area of the first image, and wherein the first constituent image corresponds to a frame of a video embedded into the first image. ## Claims ### Claim 1 A method comprising: providing data identifying a first image as input to a machine learning model trained using training data identifying a plurality of composite images that each include one or more constituent images; and determining, using one or more outputs of the trained machine learning model, that the first image is a composite image that includes a first constituent image, wherein at least a portion of the first constituent image is in a spatial area of the first image, and wherein the first constituent image corresponds to a frame of a video embedded into the first image. ### Claim 2 (depends on 1) The method of claim 1, wherein: the one or more outputs of the trained machine learning model comprise a level of confidence value indicating a likelihood of the first image being the composite image including the first constituent image; and the method further comprises: determining that the level of confidence value satisfies a threshold condition; and extracting the first constituent image from the spatial area of the first image. ### Claim 3 (depends on 1) The method of claim 1, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein the plurality of spatial areas are uniform in size and shape. ### Claim 4 (depends on 1) The method of claim 1, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein at least two of the plurality of spatial areas have different sizes. ### Claim 5 (depends on 1) The method of claim 1, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein at least two of the plurality of spatial areas overlap. ### Claim 6 (depends on 1) The method of claim 1, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein the plurality of spatial areas are non-overlapping. ### Claim 7 (depends on 1) The method of claim 1, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein at least two of the plurality of spatial areas have different shapes. ### Claim 8 (depends on 1) The method of claim 1, wherein the training data comprises an input-output mapping comprising an input and an output, the input based on pixel data of a composite image, the composite image comprising a first portion containing pixel data of a second image and a second portion containing pixel data of a third image, and the output identifying a position of the first portion within the composite image. ### Claim 9 (depends on 8) The method of claim 8 wherein the position of the first portion within the composite image comprises coordinates of an upper left corner of the second image and coordinates of a lower right corner of the second image. ### Claim 10 A system comprising: a memory; and a processing device, coupled to the memory, to perform operations comprising: providing data identifying a first image as input to a machine learning model trained using training data identifying a plurality of composite images that each include one or more constituent images; and determining, using one or more outputs of the trained machine learning model, that the first image is a composite image that includes a first constituent image, wherein at least a portion of the first constituent image is in a spatial area of the first image, and wherein the first constituent image corresponds to a frame of a video embedded into the first image. ### Claim 11 (depends on 10) The system of claim 10, wherein: the one or more outputs of the trained machine learning model comprise a level of confidence value indicating a likelihood of the first image being the composite image including the first constituent image; and the operations further comprise: determining that the level of confidence value satisfies a threshold condition; and extracting the first constituent image from the spatial area of the first image. ### Claim 12 (depends on 10) The system of claim 10, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein the plurality of spatial areas are uniform in size and shape. ### Claim 13 (depends on 10) The system of claim 10, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein at least two of the plurality of spatial areas have different sizes. ### Claim 14 (depends on 10) The system of claim 10, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein at least two of the plurality of spatial areas overlap. ### Claim 15 (depends on 10) The system of claim 10, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein the plurality of spatial areas are non-overlapping. ### Claim 16 (depends on 10) The system of claim 10, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein at least two of the plurality of spatial areas have different shapes. ### Claim 17 (depends on 10) The system of claim 10, wherein the training data comprises an input-output mapping comprising an input and an output, the input based on pixel data of a composite image, the composite image comprising a first portion containing pixel data of a second image and a second portion containing pixel data of a third image, and the output identifying a position of the first portion within the composite image. ### Claim 18 A non-transitory computer readable medium comprising instructions, which when executed by a processing device, cause the processing device to perform operations comprising: providing data identifying a first image as input to a machine learning model trained using training data identifying a plurality of composite images that each include one or more constituent images; and determining, using one or more outputs of the trained machine learning model, that the first image is a composite image that includes a first constituent image, wherein at least a portion of the first constituent image is in a spatial area of the first image, and wherein the first constituent image corresponds to a frame of a video embedded into the first image. ### Claim 19 (depends on 18) The non-transitory computer readable medium of claim 18, wherein: the one or more outputs of the trained machine learning model comprise a level of confidence value indicating a likelihood of the first image being the composite image including the first constituent image; and the operations further comprise: determining that the level of confidence value satisfies a threshold condition; and extracting the first constituent image from the spatial area of the first image. ### Claim 20 (depends on 18) The non-transitory computer readable medium of claim 18, wherein the spatial area is one of a plurality of spatial areas of the first image, and wherein one or more of the following conditions are satisfied: the plurality of spatial areas are uniform in size and shape; at least two of the plurality of spatial areas have different sizes; at least two of the plurality of spatial areas overlap; the plurality of spatial areas are non-overlapping; or at least two of the plurality of spatial areas have different shapes.