# US 2001/0026591 A1 · Multimedia stream compression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2001/0026591 A1
- Document ID: US20010026591A1
- Status: Published application
- Kind: A1
- Application number: 09/770,767
- Filed: 2001-01-25
- Published: 2001-10-04
- Earliest claimed: 1998-07-27
- Assignees: Microsoft Corporation
- Inventors: Avishai Keren; Meir Feder; Ofir Paz
- Web page: https://patentgrep.com/patent/US20010026591A1

## CPC

- H04N 7/165: {Centralised control of user terminal (subsequent to an upstream request signal H04N7/17345); Registering at central (by two-way working H04N7/17309)}
- H04N 19/124: Quantisation
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/134: characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N 19/587: involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
- H04N 19/61: in combination with predictive coding
- H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/23412: {for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects}
- H04N 21/234336: {by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text}
- H04N 21/234354: {by altering signal-to-noise ratio parameters, e.g. requantization}
- H04N 21/234363: {by altering the spatial resolution, e.g. for clients with a lower screen resolution}
- H04N 21/234381: {by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping}
- H04N 21/23655: {Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization}
- H04N 21/2402: {Monitoring of the downstream path of the transmission network, e.g. bandwidth available (traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876)}
- H04N 21/2407: {Monitoring of transmitted content, e.g. distribution time, number of downloads (arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14)}
- H04N 21/241: Operating system [OS] processes, e.g. server setup
- H04N 21/2543: Billing {, e.g. for subscription services (payment schemes, architectures or protocols G06Q20/00; e-commerce G06Q30/00; arrangements for billing for the use of broadcast information or broadcast-related information H04H60/21)}
- H04N 21/262: Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists {(scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62)}
- H04N 21/2668: Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)}
- H04N 21/274: Storing end-user {multimedia} data in response to end-user request {, e.g. network recorder}
- H04N 21/478: Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N 21/488: Data services, e.g. news ticker {(systems specially adapted for using meteorological information in broadcast systems H04H60/71)}
- H04N 21/6168: {involving cable transmission, e.g. using a cable modem}
- H04N 21/6187: {involving transmission via a telephone network, e.g. POTS}
- H04N 21/6405: Multicasting {(data broadcast and multicast in packet switching networks H04L12/18)}
- H04N 21/6408: Unicasting
- H04N 21/6547: comprising parameters, e.g. for client setup
- H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- H04N 21/8173: {End-user applications, e.g. Web browser, game}
- H04N 21/8453: {by locking or enabling a set of features, e.g. optional functionalities in an executable program}
- H04N 21/8543: using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] {(information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80)}
- H04N 21/8586: {by using a URL (processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02)}
- A63F 2300/409: Data transfer via television network
- A63F 2300/538: for performing operations on behalf of the game client, e.g. rendering
- A63F 2300/5506: using advertisements
- A63F 2300/577: for watching a game played by other players
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

A computer may be remotely accessed. At a first location, display commands are generated. The display commands are covered into a compressed video data stream. Each display element (5) is checked if it is encoded (52). If object is encoded, it is transcribed into MPEG (54). The image is adjusted for display (56) and compression (58). Additional steps of motion determination (60), change detection (62), compression depth and frame determination (66, 68) are executed. Then the data is transmitted to a second location. The display commands are decompressed and displayed as an image at the second location.

## Claims

### Claim 1

A method of generating a compressed video stream, comprising:
receiving a plurality of display commands; and
directly converting said commands into a compressed video stream, without first generating a display raster.

### Claim 2

A method of generating a compressed video stream, comprising:
receiving a plurality of display commands; and
directly converting said commands, into said compressed video stream, wherein said converting comprises motion estimation and wherein said motion estimation is performed directly on said commands without first generating a display raster.

### Claim 3

A method of generating a compressed video stream, comprising:
receiving a plurality of display commands; and
directly converting said commands into said compressed video stream, wherein said converting comprises change detection and wherein said change detection is performed directly on said commands without first generating a display raster.

### Claim 4

A method of motion estimation for image stream compression, comprising:
providing a plurality of display commands;
identifying display manipulation commands among said commands which manipulation commands manipulate image portions in said image stream, said portions being generated by other display commands among said plurality of display commands; and
generating at least one motion vector, responsive to said identified display manipulation commands.

### Claim 5 (depends on 4)

A method according to claim 4, wherein said display manipulation commands include a scroll command.

### Claim 6 (depends on 4, 5)

A method according to claim 4 or claim 5, wherein said display manipulation command is generated as a direct result of a display of a downloaded image in a WWW browser.

### Claim 7 (depends on 4, 5)

A method according to any of claims 4-5, comprising reducing a difficulty of said motion vector generation by modifying said display manipulation commands.

### Claim 8 (depends on 7)

A method according to claim 7, wherein scroll-type display modification commands are limited to multiples of a compression block size dimension.

### Claim 9

A method of change detection for image stream compression, comprising:
providing a plurality of display commands;
identifying commands of said plurality of commands which affect image portions in said image stream;
identifying said image portions; and
determining if a change in one of said image portions requires image information to be included in said image stream.

### Claim 10 (depends on 9)

A method according to claim 9 wherein said determining comprises determining if said change is greater than a change threshold.

### Claim 11

A method of generating a compressed video stream, comprising:
providing a plurality of display commands;
directly setting values in a transform space, responsive to said display commands; and
creating a compressed video stream utilizing said set values.

### Claim 12 (depends on 11)

A method according to claim 11, wherein said transform space comprises a DCT space.

### Claim 13 (depends on 11)

A method according to claim 11, wherein directly setting comprises utilizing transform coefficients from a look-up table.

### Claim 14 (depends on 11)

A method according to claim 11, wherein directly setting comprises copying transform coefficients from a look-up table of transformed graphical primitives.

### Claim 15 (depends on 11)

A method according to claim 11, wherein directly setting comprises copying transform coefficients from a cache of transformed image portions.

### Claim 16 (depends on 11)

A method according to claim 11, wherein directly setting comprises rendering said display command into said transform space utilizing an analytical formula associating a graphical primitive defined by said display command with said transform space.

### Claim 17 (depends on 11)

A method according to claim 11, comprising, modifying at least one of said display commands so that a graphical primitive defined by said at least one display command does not straddle an image block, for which block a set of transform coefficients are defined according to a compression protocol utilized by said compressed image stream.

### Claim 18

A method of generating a catalog, comprising:
providing a plurality of display elements;
generating a compressed video representation of each one of said elements; and storing said compressed representations in a memory, indexed by element.

### Claim 19 (depends on 18)

A method according to claim 18, wherein said display elements comprise GUI (graphic user interface) objects.

### Claim 20

A method of generating a catalog, comprising:
receiving, during interaction with an Internet a plurality of objects to be displayed;
generating a compressed video representation of each one of said objects; and
storing said compressed representations in a cache of objects, whereby when one of said objects is received again from said Internet at a later time, a earlier stored compressed representation is used for display generation.

### Claim 21 (depends on 20)

A method according to claim 20, wherein said stored objects comprise at least one JPEG image.

### Claim 22 (depends on 20)

A method according to claim 20, wherein said stored objects comprise at least one uncompressed image.

### Claim 23 (depends on 20)

A method according to claim 20, wherein said stored objects comprise a WWW page.

### Claim 24

A method of generating a compressed video stream, comprising:
generating a plurality of display commands which represents a display, by a program;
modifying at least one of said display commands, independently of said program; and
generating a compressed video stream utilizing said display commands.

### Claim 25 (depends on 24)

A method according to claim 24, wherein said modifying comprises modifying said commands to responsive to limitations of a display device on which said compressed video stream is to be displayed.

### Claim 26 (depends on 24)

A method according to claim 24, wherein modifying comprises modifying said commands to reduce bandwidth requirements of said compressed video stream.

### Claim 27 (depends on 24)

A method according to claim 24, wherein modifying comprises modifying said commands to reduce resource requirements for compressing said compressed video stream.

### Claim 28 (depends on 24, 25, 26, 27)

A method according to any of claims 24-27, wherein at least one of said display commands represents an object and wherein said modifying comprises changing said at least one display command such that the object is moved relative to its original display position.

### Claim 29 (depends on 28)

A method according to claim 28, wherein said compressed video stream comprises utilizes blocks having boundaries and wherein moving comprises moving said object to match at least one compression block boundary.

### Claim 30 (depends on 24, 25, 26, 27)

A method according to any of claims 24-27, wherein at least one of said display commands represents an object and wherein said modifying comprises replacing said object with a different object.

### Claim 31 (depends on 30)

A method according to claim 30, wherein replacing said object comprises replacing said object with a compressed representation of said different object.

### Claim 32 (depends on 30)

A method according to claim 30, wherein said object comprises a text object.

### Claim 33 (depends on 30)

A method according to claim 30, wherein said object comprises a background of said display.

### Claim 34 (depends on 30)

A method according to claim 30, comprising analyzing said object to determine a closest suitable replacement object.

### Claim 35 (depends on 24, 25, 26, 27)

A method according to any of claims 24-27, wherein modifying said object comprises changing a font definition for said object.

### Claim 36 (depends on 24, 25, 26, 27)

A method according to any of claims 24-27, wherein modifying said object comprises modifying at least one color of said object.

### Claim 37 (depends on 36)

A method according to claim 36, wherein modifying a color comprises reducing a spatial resolution of said colors.

### Claim 38 (depends on 36)

A method according to claim 36, wherein modifying a color comprises reducing a color range resolution of said colors.

### Claim 39 (depends on 24, 25, 26, 27)

A method according to any of claims 24-27, wherein modifying said object comprises reducing a spatial resolution of said object.

### Claim 40 (depends on 24, 25, 26, 27)

A method according to any of claims 24-27, wherein at least one of said display commands comprises a scrolling command and wherein said modifying comprises increasing a granularity of said scrolling.

### Claim 41 (depends on 40)

A method according to claim 40, wherein said increasing a granularity comprises limiting said scrolling command to multiples of compression blocks size of said compressed stream.

### Claim 42

A method of generating a compressed video stream, comprising:
generating a plurality of display commands which represents a display;
generating a plurality of transform coefficients from said display commands, wherein said transform coefficients are quantized and wherein said quantization is modified responsive to limitations a display device on which said compressed video stream is to be displayed; and
creating a compressed video stream utilizing said coefficients.

### Claim 43 (depends on 42)

A method according to claim 42, wherein said coefficients are quantized at a lower resolution in a vertical direction of said display.

### Claim 44

A method of generating a compressed video stream, comprising:
providing a display definition including a compressed object; and
converting said display definition into a compressed video stream, wherein converting includes transcribing at least a portion of said compressed object, without decompressing said compressed object.

### Claim 45 (depends on 44)

A method according to claim 44, wherein transcribing comprises reducing a quality level of said compressed object.

### Claim 46 (depends on 44, 45)

A method according to claim 44 or claim 45, wherein said compressed stream and said compressed object are compressed utilizing a same transform and wherein transcribing comprises copying transform coefficients from said compressed object to said compressed video stream.

### Claim 47 (depends on 44, 45)

A method according to claim 44 or claim 45, wherein said compressed stream and said compressed object are compressed utilizing a motion estimation representation and wherein transcribing comprises copying motion estimation vectors from said compressed object to said compressed video stream.

### Claim 48 (depends on 44, 45)

A method according to claim 44 or claim 45, wherein converting said display definition comprises directly converting said display representation into said compressed video stream, without generating an intermediate image raster.

### Claim 49 (depends on 44, 45)

A method according to claim 44 or claim 45, wherein said compressed object comprises a second compressed video stream, covering only a portion of said display.

### Claim 50 (depends on 44, 45)

A method according to claim 44 or claim 45, wherein said compressed object comprises a compressed image, covering only a portion of said display.

### Claim 51 (depends on 44, 45)

A method according to claim 44 or claim 45, wherein said compressed object comprises a compressed audio stream.

### Claim 52

A method of image processing, comprising:
providing an image to be compressed for transmission;
simultaneously compressing and processing said image;
transmitting said image to a remote location; and
decompressing said processed image at said remote location.

### Claim 53 (depends on 52)

A method according to claim 52, wherein said simultaneously compressing and processing comprises processing said image by modifying a quantization of at least one transform coefficient of said image.

### Claim 54 (depends on 52)

A method according to claim 52, wherein said modifying comprises increasing a quantization granularity for vertical coefficients.

### Claim 55

A method of generating a compressed video stream, comprising:
accumulating a plurality of display commands;
identifying at least one display command of said plurality, whose effect is mooted by a later accumulated display command; and
converting said commands into said compressed video stream, ignoring the mooted display command.

### Claim 56 (depends on 55)

A method according to claim 55, wherein said mooted display command is ignored if its effect on said compressed video stream, in view of said later command, is below a threshold value.

### Claim 57 (depends on 55)

A method according to claim 55, wherein converting said commands comprises combining display commands.

### Claim 58 (depends on 55, 56, 57)

A method according to any of claims 55-57, wherein said converting comprises directly converting said commands into said compressed video stream without first generating an image raster.

### Claim 59

A method of video compression, comprising:
generating a display, which generation is refreshed at a generation refresh frame rate;
compressing, transmitting and decompressing said display; and
displaying said display at a second refresh rate other than said generation refresh rate and wherein said refresh is not synchronized to said display refresh rate.

### Claim 60 (depends on 59)

A method according to claim 59, wherein said compressing is un-synchronized with said generation refresh frame rate.

### Claim 61 (depends on 59)

A method according to claim 59, wherein said transmitting is un-synchronized with said compressing.

### Claim 62

A method of generating a compressed video stream, comprising:
generating a plurality of display commands;
accumulating at least one display command;
generating a compressed video stream from said accumulated display commands, wherein accumulating comprises accumulating a number of display commands responsive to instantaneous available resources.

### Claim 63

A method of generating a compressed video stream, comprising:
generating a plurality of display commands;
accumulating at least one display command;
generating a compressed video stream from said accumulated display commands, wherein accumulating comprises accumulating a number of display commands responsive to a desired output frame rate.

### Claim 64 (depends on 62, 63)

A method according to claim 62 or claim 63, wherein generating a compressed video stream is un-synchronized relative to said generation of display commands.

### Claim 65 (depends on 64)

A method according to claim 64, wherein said display commands are generated in sets, each set associated with a display frame and wherein said accumulating, in a single accumulation set, display commands originating from different sets, wherein said single accumulation set is utilized for generating a single frame of said compressed video stream.

### Claim 66 (depends on 62, 63)

A method according to claim 62 or claim 63, comprising assigning a priority to a display command.

### Claim 67 (depends on 66)

A method according to claim 66, comprising reordering said accumulated display commands, responsive to said priority, wherein generating a compressed video stream comprises generating a first frame in said compressed video stream utilizing an originally later display command, prior to generating a frame in said stream utilizing an originally earlier display command.

### Claim 68

A method of generating a compressed video stream, comprising:
generating a plurality of display commands;
generating a compressed image portion responsive to said display commands; and
assembling a compressed video stream from said compressed image portion and from at least one previously compressed image portion, created prior to generation of said display commands.

### Claim 69 (depends on 68)

A method according to claim 68, wherein said previously compressed image portion comprises an image portion created for a previous image of said video stream.

### Claim 70

A method of asynchronous video stream compression, comprising:
providing a first data source and a second data source;
converting said first data source into a first portion of a compressed video stream;
converting said second data source into a second portion of a compressed video stream; and
mixing said first portion and said second portions to create a single compressed video stream, wherein said first portion and said second portion temporally overlap in said single compressed video stream.

### Claim 71 (depends on 70)

A method according to claim 70, wherein said first portion and said second portion utilize different frame rates.

### Claim 72 (depends on 70)

A method according to claim 70, wherein said first portion and said second portion occupy different spatial areas in a display defined by said single compressed video stream.

### Claim 73 (depends on 70, 71, 72)

A method according to any of claims 70-72, wherein said first and said second data sources comprise sets of display commands.

### Claim 74 (depends on 73)

A method according to claim 73, wherein converting said first data source and converting said second data source comprises directly converting said data sources without generating an intermediate image raster.

### Claim 75 (depends on 70, 71, 72)

A method according to any of claims 70-72, wherein at least one of said first data source and said second data source comprises an image source.

### Claim 76 (depends on 70, 71, 72)

A method according to any of claims 70-72, wherein said first data source comprises a display portion affected by a command entered by a user who views said single compressed video source.

### Claim 77 (depends on 76)

A method according to claim 76, comprising identifying portions of a data display which are affected by said user command.

### Claim 78

A method of generating a compressed video stream, comprising:
executing a program, which program generates a set of display commands, representative of a display;
identifying changes in said display which are responsive to at least one type of user command; and
converting said display commands into a compressed video stream, wherein said changes are inserted into said compressed video stream at a higher frame rate than other changes in said display.

### Claim 79 (depends on 78)

A method according to claim 78, wherein said type of user command comprises a pointing device command.

### Claim 80 (depends on 78)

A method according to claim 78, wherein said changes comprises an indication of a selection of a GUI (graphical user interface) element.

### Claim 81 (depends on 78, 79, 80)

A method according to any of claims 78-80, comprising analyzing said user command to determine display commands which effect said identified changes.

### Claim 82

A method of display control, comprising:
transmitting a request, from a generator of a compressed video stream, to a source of display commands, which requests relates to said compression; and
modifying a generation of display commands, at said source, responsive to said request.

### Claim 83 (depends on 82)

A method according to claim 82, wherein said source of display commands comprises a WWW browser.
