# US 10,768,886 B2 · System, method and computer program product for generating remote views in a virtual mobile device platform using efficient color space conversion and frame encoding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,768,886 B2 - Document ID: US10768886B2 - Status: Granted - Kind: B2 - Application number: 16/451,767 - Filed: 2019-06-25 - Published: 2019-10-17 - Granted: 2020-09-08 - Earliest claimed: 2016-07-28 - Assignees: Intelligent Waves LLC - Applicants: Intelligent Waves LLC - Inventors: Robert David Lee; Hemal Yogesh Gandhi; Akshay P. Joshi - Web page: https://patentgrep.com/patent/US10768886B2 ## CPC - G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay} - G06F 9/452: {Remote windowing, e.g. X-Window System, desktop virtualisation (protocols for virtual reality H04L67/131)} - G06F 9/45558: {Hypervisor-specific management and integration aspects} - G06F 21/53: by executing in a restricted environment, e.g. sandbox or secure virtual machine - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - H04L 67/565: Conversion or adaptation of application format or content (adding application control or application functional data H04L67/561) - H04N 19/103: Selection of coding mode or of prediction mode - H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence) - H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/186: the unit being a colour or a chrominance component - H04N 19/27: involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC] - H04N 19/423: characterised by memory arrangements (H04N19/433 takes precedence) - H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109) - H04N 19/517: by encoding - H04N 19/547: Motion estimation performed in a transform domain - H04N 19/61: in combination with predictive coding - H04N 19/625: using discrete cosine transform [DCT] - G06F 2221/032: Protect output to user by software means - G06F 2221/2149: Restricted operating environment - G09G 5/12: Synchronisation between the display unit and other units, e.g. other display units, video-disc players - G09G 2340/02: Handling of images in compressed format, e.g. JPEG, MPEG - G09G 2340/06: Colour space transformation - G09G 2350/00: Solving problems of bandwidth in display systems - G09G 2358/00: Arrangements for display data security - G09G 2360/121: using a cache memory - G09G 2360/18: Use of a frame buffer in a display terminal, inclusive of the display panel - G09G 2370/022: Centralised management of display operation, e.g. in a server instead of locally - G09G 2370/027: Arrangements and methods specific for the display of internet documents - H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference - H04N 19/172: the region being a picture, frame or field - H04W 88/02: Terminal devices - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract Embodiments disclosed herein provide systems, methods and computer readable media for generating remote views in a virtual mobile device platform. A virtual mobile device platform may be coupled to a physical mobile device over a network and generate frames of data for generating views on the physical device. These frames can be generated using an efficient display encoding pipeline on the virtual mobile device platform. Such efficiencies may include, for example, the synchronization of various processes or operations, the governing of various processing rates, the elimination of duplicative or redundant processing, the application of different encoding schemes, the efficient detection of duplicative or redundant data or the combination of certain operations. ## Claims ### Claim 1 A system for efficient frame processing, the system comprising: a computer system that comprises one or more processors programmed with computer program instructions that, when executed, cause the computer system to: obtain a frame of display data; identify, for each macroblock of the frame of display data, whether the macroblock is a zero motion vector (ZMV) macroblock; generate a converted frame by performing color space conversion on the frame of display data based on the identification of the macroblock; and generate an encoded frame by encoding the converted frame, wherein performing the color space conversion on the frame of display data based on the identification of the macroblock includes performing the color space conversion on at least one macroblock of the frame not identified as the ZMV macroblock and not performing the color space conversion on at least one other macroblock of the frame identified as the ZMV macroblock. ### Claim 2 (depends on 1) The system of claim 1, wherein generating the converted frame includes replacing corresponding one or more macroblocks of a previously converted frame with converted macroblocks of the frame not identified as ZMV macroblocks. ### Claim 3 (depends on 2) The system of claim 2, wherein the previously converted frame is stored in a buffer. ### Claim 4 (depends on 1) The system of claim 1, wherein encoding of the converted frame includes only encoding macroblocks of the converted frame not identified as ZMV macroblocks. ### Claim 5 (depends on 1) The system of claim 1, wherein identifying, for each macroblock of the frame of display data, whether the macroblock is the ZMV macroblock includes comparing the frame of display data to a previous frame of display data. ### Claim 6 (depends on 1) The system of claim 1, wherein the computer system is caused to send the encoded frame to a physical device. ### Claim 7 (depends on 1) The system of claim 1, wherein performing the color space conversion on the frame of display data includes performing the color space conversion from an RGB color space to a YUV color space. ### Claim 8 (depends on 1) The system of claim 1, wherein the computer system is caused to: generate a data structure including static and non-static macroblock information for the frame of display data based on the identification of the macroblock; and perform the color space conversion on the frame of display data based on the data structure. ### Claim 9 (depends on 8) The system of claim 8, wherein the static macroblock information indicates a first subset of macroblocks of the frame of display data that is unchanged compared to corresponding one or more macroblocks of a previous frame of display data and the non-static macroblock information indicates a second subset of macroblocks of the frame of display data that has changed compared to corresponding one or more macroblocks of the previous frame of display data. ### Claim 10 A method for efficient frame processing, the method being implemented by one or more processors configured to execute one or more computer program instructions, the method comprising: obtaining a frame of display data; identifying, for each macroblock of the frame of display data, whether the macroblock is unchanged; generating a converted frame by performing color space conversion on the frame of display data based on the identification of the macroblock; and generating an encoded frame by encoding the converted frame, wherein performing the color space conversion on the frame of display data based on the identification of the macroblock includes performing the color space conversion on at least one macroblock of the frame not identified as unchanged and not performing the color space conversion on at least one other macroblock of the frame identified as unchanged. ### Claim 11 (depends on 10) The method of claim 10, wherein generating the converted frame includes replacing corresponding one or more macroblocks of a previously converted frame with converted macroblocks of the frame not identified as unchanged. ### Claim 12 (depends on 11) The method of claim 11, wherein the previously converted frame is stored in a buffer. ### Claim 13 (depends on 10) The method of claim 10, wherein encoding of the converted frame includes only encoding macroblocks of the converted frame not identified as unchanged. ### Claim 14 (depends on 10) The method of claim 10, wherein identifying, for each macroblock of the frame of display data, whether the macroblock is unchanged includes comparing the frame of display data to a previous frame of display data. ### Claim 15 (depends on 10) The method of claim 10, further comprising: sending the encoded frame to a physical device. ### Claim 16 (depends on 10) The method of claim 10, wherein performing the color space conversion on the frame of display data includes performing the color space conversion from an RGB color space to a YUV color space. ### Claim 17 (depends on 10) The method of claim 10, further comprising: generating a data structure including static and non-static macroblock information for the frame of display data based on the identification of the macroblock; and performing the color space conversion on the frame of display data based on the data structure. ### Claim 18 One or more non-transitory, computer-readable media storing instructions that, when executed by one or more processors, effectuate operations comprising: obtaining a frame of display data; identifying, for each macroblock of the frame of display data, whether the macroblock is unchanged; generating a converted frame by performing color space conversion on the frame of display data based on the identification of the macroblock; and generating an encoded frame by encoding the converted frame, wherein performing the color space conversion on the frame of display data based on the identification of the macroblock includes performing the color space conversion on at least one macroblock of the frame not identified as unchanged and not performing the color space conversion on at least one other macroblock of the frame identified as unchanged.