# US 2023/0067541 A1 · Patch based video coding for machines > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0067541 A1 - Document ID: US20230067541A1 - Status: Published application - Kind: A1 - Application number: 17/795,178 - Filed: 2021-04-15 - Published: 2023-03-02 - Earliest claimed: 2020-04-16 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Jill Boyce; Palanivel Guruva Reddiar; Praveen Prasad - Web page: https://patentgrep.com/patent/US20230067541A1 ## CPC - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - H04N 19/33: in the spatial domain - G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 40/161: {Detection; Localisation; Normalisation} - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/167: Position within a video image, e.g. region of interest [ROI] - H04N 19/17: the unit being an image region, e.g. an object - G06N 20/00: Machine learning - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Devices and techniques related to implementing patch based video coding for machines are discussed. Such patch based video coding includes detecting regions of interest in a frame of video, extracting the detected regions of interest to one or more atlases absent the frame at a resolution not less than the resolution of the regions of interest, and encoding the one or more atlases to a bitstream. ## Claims ### Claim at position 1 1-25. (canceled) ### Claim 26 A system comprising: a memory to store at least a portion of input video; and processor circuitry coupled to the memory, the processor circuitry to: detect a plurality of regions of interest for a machine learning operation in a full frame of the input video; form one or more atlases comprising the regions of interest at a first resolution; generate metadata corresponding to the one or more atlases and indicative of a size and location of each of the regions of interest in the full frame of video; and encode the one or more atlases and the metadata into one or more bitstreams, the one or more bitstreams absent a representation of the full frame of video at the first resolution or a resolution higher than the first resolution. ### Claim 27 (depends on 26) The system of claim 26, the processor circuitry to: downscale the full frame of video to a second resolution less than the first resolution; and include the downscaled frame of video at the second resolution in the one or more atlases for encode into the one or more bitstreams. ### Claim 28 (depends on 26) The system of claim 26, wherein the processor circuitry to encode a first region of interest of the plurality of regions of interest comprises the processor circuitry to perform scalable video encode based on the first region of interest and a corresponding region of the full frame of video at a resolution lower than the first resolution. ### Claim 29 (depends on 26) The system of claim 26, wherein the metadata comprises, for a first region of interest of the plurality of regions of interest, a top left position of the first region in the full frame and a scaling factor. ### Claim 30 (depends on 26) The system of claim 26, wherein a first region of interest of the plurality of regions of interest is in a first atlas, the processor circuitry to: detect a second region of interest in a subsequent frame of the video; and resize the first atlas and adding the second region of interest to the resized first atlas, wherein the processor circuitry to encode the one or more atlases and the metadata comprises the processor circuitry to encode the resized first atlas. ### Claim 31 (depends on 26) The system of claim 26, wherein the processor circuitry to detect a first region of interest of the plurality of regions of interest comprises the processor circuitry to: perform lookahead analysis to detect corresponding subsequent first regions of interest in a plurality of temporally subsequent frames relative to the full frame; and size the first region of interest to include the first region of interest and all subsequent first regions of interest. ### Claim 32 (depends on 26) The system of claim 26, wherein the processor circuitry to detect a first region of interest of the plurality of regions of interest comprises the processor circuitry to: determine a detected region around an object in the first region of interest; and expand the detected region to the first region of interest to provide a buffer around the region. ### Claim 33 (depends on 26) The system of claim 26, wherein a first region of interest of the plurality of regions of interest comprises a representation of a face, the processor circuitry to separate the first region of interest into a first atlas and encrypt a first bitstream corresponding to the first atlas. ### Claim 34 At least one machine readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to code video for machine learning by: detecting a plurality of regions of interest for a machine learning operation in a full frame of video; forming one or more atlases comprising the regions of interest at a first resolution; generating metadata corresponding to the one or more atlases and indicative of a size and location of each of the regions of interest in the full frame of video; and encoding the one or more atlases and the metadata into one or more bitstreams, the one or more bitstreams absent a representation of the full frame of video at the first resolution or a resolution higher than the first resolution. ### Claim 35 (depends on 34) The machine readable medium of claim 34, further comprising instructions that, in response to being executed on the computing device, cause the computing device to code video for machine learning by: downscaling the full frame of video to a second resolution less than the first resolution; and including the downscaled frame of video at the second resolution in the one or more atlases for encode into the one or more bitstreams, wherein encoding a first region of interest of the plurality of regions of interest comprises scalable video encoding based on the first region of interest and a corresponding region of the full frame of video at a resolution lower than the first resolution. ### Claim 36 (depends on 34) The machine readable medium of claim 34, wherein a first region of interest of the plurality of regions of interest is in a first atlas, the machine readable medium further comprising instructions that, in response to being executed on the computing device, cause the computing device to code video for machine learning by: detecting a second region of interest in a subsequent frame of the video; and resizing the first atlas and adding the second region of interest to the resized first atlas, wherein encoding the one or more atlases and the metadata comprises encoding the resized first atlas. ### Claim 37 (depends on 34) The machine readable medium of claim 34, wherein detecting a first region of interest of the plurality of regions of interest comprises: performing lookahead analysis to detect corresponding subsequent first regions of interest in a plurality of temporally subsequent frames relative to the full frame; and sizing the first region of interest to include the first region of interest and all subsequent first regions of interest. ### Claim 38 A system comprising: a memory to store at least a portion of one or more bitstreams comprising one or more atlases representative of a plurality of regions of interest at a first resolution and metadata to locate the of regions of interest in a full frame of video, the one or more bitstreams absent a representation of the full frame of video at the first resolution or a resolution higher than the first resolution; and processor circuitry coupled to the memory, the processor circuitry to: decode the one or more bitstreams to generate the plurality of regions of interest; and apply machine learning to one or more of the regions of interest based in part on corresponding one or more locations of the one or more of the regions of interest in the full frame to generate a machine learning output. ### Claim 39 (depends on 38) The system of claim 38, the processor circuitry to: decode the one or more bitstreams to generate the full frame at a second resolution less than the first resolution, wherein the processor circuitry to decode a first region of interest of the plurality of regions of interest comprise the processor circuitry to perform scalable video decoding based on the first region of interest and a corresponding downscaled region of the full frame of video. ### Claim 40 (depends on 38) The system of claim 38, wherein the metadata comprises, for a first region of interest of the plurality of regions of interest, a top left position of the first region in the full frame and a scaling factor. ### Claim 41 (depends on 38) The system of claim 38, wherein a first region of interest of the plurality of regions of interest comprises a representation of a face, the processor circuitry to decrypt at least a portion of the one or more bitstreams comprising a representation of the first region of interest. ### Claim 42 (depends on 38) The system of any of claim 38, wherein the regions of interest comprise face detection regions and the processor circuitry to apply the machine learning comprising the processor circuitry to apply face recognition to the one or more of the regions of interest. ### Claim 43 At least one machine readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to perform machine learning by: receiving one or more bitstreams comprising one or more atlases representative of a plurality of regions of interest at a first resolution and metadata to locate the of regions of interest in a full frame of video, the one or more bitstreams absent a representation of the full frame of video at the first resolution or a resolution higher than the first resolution; decoding the one or more bitstreams to generate the plurality of regions of interest; and applying machine learning to one or more of the regions of interest based in part on corresponding one or more locations of the one or more of the regions of interest in the full frame to generate a machine learning output. ### Claim 44 (depends on 43) The machine readable medium of claim 43, further comprising instructions that, in response to being executed on the computing device, cause the computing device to perform machine learning by: decoding the one or more bitstreams to generate the full frame at a second resolution less than the first resolution, wherein decoding a first region of interest of the plurality of regions of interest comprise scalable video decoding based on the first region of interest and a corresponding downscaled region of the full frame of video. ### Claim 45 (depends on 43) The machine readable medium of claim 43, wherein a first region of interest of the plurality of regions of interest comprises a representation of a face, the machine readable medium further comprising instructions that, in response to being executed on the computing device, cause the computing device to perform machine learning by decrypting at least a portion of the one or more bitstreams comprising a representation of the first region of interest.