# US 10,321,138 B2 · Adaptive video processing of an interactive environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,321,138 B2 - Document ID: US10321138B2 - Status: Granted - Kind: B2 - Application number: 14/457,007 - Filed: 2014-08-11 - Published: 2014-11-27 - Granted: 2019-06-11 - Earliest claimed: 2008-09-26 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Walid Ali - Web page: https://patentgrep.com/patent/US10321138B2 ## CPC - H04N 19/164: Feedback from the receiver or from the transmission channel - H04L 1/0015: {characterised by the adaptation strategy} - H04L 12/1827: {Network arrangements for conference optimisation or adaptation} - H04N 7/15: Conference systems - H04N 19/115: Selection of the code volume for a coding unit prior to coding - H04N 19/166: concerning the amount of transmission errors, e.g. bit error rate [BER] - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/513: Processing of motion vectors - H04N 19/60: using transform coding - H04N 21/2402: {Monitoring of the downstream path of the transmission network, e.g. bandwidth available (traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876)} - H04N 21/42203: {sound input device, e.g. microphone} - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 21/440263: {by altering the spatial resolution, e.g. for displaying on a connected PDA} - H04N 21/44209: {Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network (arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896)} - H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} - H04L 1/0009: {by adapting the channel coding (H04L1/1812 takes precedence)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Embodiments are configured to provide interactive communication functionality including adaptive video processing functionality that can be used to process aspects of a video signal, but the embodiments are not so limited. In an embodiment, components of a video conferencing system can operate to provide a video signal based in part on the use of adaptive processing features which include scaling and/or other pixel processing features. In one embodiment, components of an interactive video system can operate to adaptively manage and control video payload parameters to adapt to various communication conditions associated with a real-time or near-real time interactive video environment. ## Claims ### Claim 1 A method comprising: determining a packet loss parameter associated with a video packet transmission based on packet reception information, wherein the packet reception information comprises a number of packets received by a video receiving device during an amount of time; adaptively adjusting a communication parameter based at least in part on the packet loss parameter to form an adjusted communication parameter based at least on a feedback loop between a pre-processor and an encoder, wherein the communication parameter comprises: a scaling factor for scaling an unencoded video signal, a video quantization factor for encoding the unencoded video signal, and a level of payload protection for adjusting a protection level of packet loss; pre-processing, by the pre-processor, the unencoded video signal with at least the scaling factor to form a pre-processed video signal; encoding, by the encoder, the pre-processed video signal with at least the video quantization factor to form an encoded signal; adjusting a protection level of packet loss on the encoded signal with at least the level of payload protection; and transmitting the encoded signal and the adjusted communication parameter. ### Claim 2 (depends on 1) The method of claim 1, further comprising adapting a size of the video packet transmission to network conditions. ### Claim 3 (depends on 1) The method of claim 1, further comprising modifying the protection level by assigning a number of protecting bytes to the video packet transmission. ### Claim 4 (depends on 1) The method of claim 1, further comprising appending one or more protection packets to the encoded signal before transmitting the encoded signal over a communication channel. ### Claim 5 (depends on 1) The method of claim 1, wherein encoding the pre-processed video signal further comprises using one or more of range compression operations, motion estimation operations, transformation operations, quantization operations, de-blocking operations, reference operations, prediction operations, variable-length coding operations, buffer management operations, estimated distortion heuristics, quality impact techniques, and inter/intra prediction optimizations. ### Claim 6 (depends on 1) The method of claim 1, wherein the pre-processing of the unencoded video signal further comprises: scaling a group of pixels of a frame of pixel data associated with the unencoded video signal. ### Claim 7 (depends on 1) The method of claim 1 further comprising scaling horizontal and vertical dimensions of the video signal. ### Claim 8 (depends on 1) The method of claim 1, wherein the communication parameter comprises a video processing bandwidth. ### Claim 9 (depends on 8) The method of claim 8, wherein adaptively adjusting the communication parameter further comprises: reducing the video processing bandwidth and increasing the scaling factor when the packet loss parameter is greater than a predetermined value. ### Claim 10 (depends on 1) The method of claim 1, wherein adaptively adjusting the communication parameter further comprises: decreasing the scaling factor when the packet loss parameter is less than a predetermined value. ### Claim 11 (depends on 1) The method of claim 1, wherein the communication parameter comprises a spatial resolution, and adaptively adjusting the communication parameter further comprises: reducing the spatial resolution when the packet loss parameter is greater than a predetermined value. ### Claim 12 A method comprising: determining a packet loss parameter associated with a video packet transmission based on packet reception information, wherein the packet reception information comprises a number of packets received by a video receiving device during an amount of time; adaptively adjusting a communication parameter based at least on the packet loss parameter to form an adjusted communication parameter based at least on a feedback loop between a pre-processor and an encoder, wherein the communication parameter comprises: a scaling factor for scaling an unencoded video signal, a video quantization factor for encoding the unencoded video signal, and a level of payload protection for adjusting a protection level of packet loss; pre-processing, by the pre-processor, the unencoded video signal with at least the scaling factor to form a pre-processed video signal, wherein the pre-processing further comprises pre-processing a frame of pixel data by discarding one or more of even and odd lines of the frame of pixel data to obtain a field of pixel data; encoding, by the encoder, the pre-processed video signal with at least the video quantization factor to form an encoded signal; adjusting a protection level of packet loss on the encoded signal with at least the level of payload protection; and transmitting the encoded signal and the adjusted communication parameter. ### Claim 13 (depends on 12) The method of claim 12, wherein pre-processing the video signal further comprises pre-processing consecutive frames of pixel data to provide a plurality of fields of pixel data. ### Claim 14 An adaptive video processing system comprising: at least one processor; memory storing computer executable instructions that when executed by the at least one processor cause the processor to: determine a packet loss ratio associated with a video packet based on packet reception information, wherein the packet reception information comprises a number of packets received by a video receiving device during an amount of time; adaptively adjust a communication parameter based at least on the packet loss ratio to form an adjusted communication parameter based at least on a feedback loop between a pre-processor and an encoder, wherein the communication parameter comprises: a scaling factor for scaling an unencoded video signal, a video quantization factor for encoding the unencoded video signal, and a level of payload protection for adjusting a protection level of packet loss; pre-process, by the pre-processor, the unencoded video signal with at least in part the scaling factor to form a pre-processed video signal; encode, by the encoder, the pre-processed video signal with at least in part the video quantization factor to form an encoded signal; adjust a protection level of packet loss on the encoded signal with at least in part the level of payload protection; and transmit the encoded signal and the adjusted communication parameter to the network. ### Claim 15 (depends on 14) The adaptive video processing system of claim 14, wherein the communication parameter comprises a video processing bandwidth, and the system further comprises computer executable instructions stored in the memory that when executed by the processor cause the processor to: reduce the video processing bandwidth when the packet loss ratio exceeds a threshold. ### Claim 16 (depends on 15) The adaptive video processing system of claim 15, wherein the communication parameter comprises a spatial resolution, and the system further comprises computer executable instructions stored in the memory that when executed by the processor cause the processor to: reduce the spatial resolution when the packet loss ratio exceeds a threshold. ### Claim 17 (depends on 14) The adaptive video processing system of claim 14, wherein the communication parameter comprises a spatial resolution, and the system further comprises computer executable instructions stored in the memory that when executed by the processor cause the processor to: increase the spatial resolution when the packet loss ratio is less than a threshold. ### Claim 18 (depends on 14) The adaptive video processing system of claim 14, wherein the communication parameter comprises a video processing bandwidth, and the system further comprises computer executable instructions stored in the memory that when executed by the processor cause the processor to: increase the video processing bandwidth when the packet loss ratio is less than a threshold. ### Claim 19 (depends on 14) The adaptive video processing system of claim 14 further comprising computer executable instructions stored in the memory that when executed by the processor cause the processor to: receive, from the network, the transmitted encoded signal and the adjusted communication parameter; decode the received encoded signal; post-process the decoded signal to form a post-processed video signal; adjust the packet loss ratio based on the received encoded signal; transmit the adjusted packet loss ratio to the network; and display the post-processed video signal.