# US 2012/0250980 A1 · Method, apparatus and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0250980 A1
- Document ID: US20120250980A1
- Status: Published application
- Kind: A1
- Application number: 13/418,987
- Filed: 2012-03-13
- Published: 2012-10-04
- Earliest claimed: 2011-03-29
- Assignees: Sony Corporation
- Inventors: Clive Henry Gillard; Robert Mark Stefan Porter; Stephen Mark Keating
- Web page: https://patentgrep.com/patent/US20120250980A1

## CPC

- H04N 21/234: Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- A63F 13/52: involving aspects of the displayed game scene
- H04N 5/2226: {Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10)}
- H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay
- H04N 19/162: User input
- H04N 19/17: the unit being an image region, e.g. an object
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 23/635: {Region indicators; Field of view indicators}
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- A63F 2300/1093: using visible light
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

A method of providing, over a network, an image for recreation in a device, the image containing a background and a foreground object and the method comprising: detecting the position of the foreground object in the image and generating position information on dependence thereon; removing the foreground object from the image; and transferring to the device i) the image with the foreground object removed, ii) the removed foreground object and iii) the position information.

## Claims

### Claim 1

A method of providing, over a network, an image for recreation in a device, the image containing a background and a foreground object and the method comprising: detecting the position of the foreground object in the image and generating position information on dependence thereon; removing the foreground object from the image; and transferring to the device i) the image with the foreground object removed, ii) the removed foreground object and iii) the position information.

### Claim 2 (depends on 1)

A method according to claim 1, comprising scaling the image with the foreground object removed therefrom; and dividing the scaled image into n segments, where n is an integer.

### Claim 3 (depends on 2)

A method according to claim 2, wherein each of the n image segments is 1920×1080 pixels.

### Claim 4 (depends on 2)

A method according to claim 2, comprising encoding each of the n segments, wherein for an area located at a position defined by the position information, encoding is performed at a higher bitrate compared with the remainder of the segment.

### Claim 5 (depends on 2)

A method according to claim 2, comprising providing a supplemental area in at least one of the segments, the supplemental area being blank and inserting the removed foreground object into the supplemental area prior to transmission to the device.

### Claim 6 (depends on 1)

A method according to claim 1, further comprising transferring to the device a depth map identifying the depth of each pixel relative to the camera position at which the image was captured.

### Claim 7

A method of recreating an image containing a background and a foreground object, the method comprising: receiving over a network i) the image with the foreground object removed, ii) the removed foreground object and iii) position information identifying the position of the foreground object in the image and inserting the foreground object into the image with the foreground object removed at a position defined by the position information.

### Claim 8 (depends on 7)

A method according to claim 7, comprising receiving the image with the foreground object removed as n segments where n is an integer and stitching the n segments together.

### Claim 9 (depends on 8)

A method according to claim 8, wherein each of the n image segments is 1920×1080 pixels.

### Claim 10 (depends on 7)

A method according to claim 7, comprising receiving at least one of the segments with a supplemental area provided therein, the supplemental area having the removed foreground object inserted therein.

### Claim 11 (depends on 7)

A method according to claim 7 further comprising receiving a depth map identifying the depth of each pixel in the image, generating a disparity at the pixel position defined by the position information on the basis of the depth at that pixel position and the dimensions of the screen upon which the image to be recreated, and generating a stereoscopic image formed of two images whereby in one of the images, a foreground object is inserted at a position defined by the position information and in the other image forming the stereoscopic image, inserting a copy of the foreground object into the image at a position horizontally displaced from the position defined by the position information.

### Claim 12

A method of providing, over a network, an image for recreation in a device, the image containing a background and a foreground object and the method comprising: detecting the position of the foreground object in the image and generating position information in dependence thereon; scaling the image; dividing the scaled image into n segments, where n is an integer; encoding each of the n segments, wherein for an area located at a position defined by the position information, the encoding is performed at a higher bitrate compared with the remainder of the segment; and transmitting to the device the encoded image.

### Claim 13 (depends on 12)

A method according to claim 12, wherein each of the n image segments is 1440×540 pixels.

### Claim 14 (depends on 12, 13)

A method according to either one of claim 12 or 13, further comprising transmitting to the device a depth map identifying the depth of each pixel relative to the camera position at which the image was captured.

### Claim 15 (depends on 12)

A method of recreating an image containing a background and a foreground object, the method comprising: receiving over a network a scaled image produced by the method of claim 12 divided into n segments, where n is an integer and stitching the n segments together.

### Claim 16 (depends on 15)

A method according to claim 15, comprising receiving a depth map identifying the depth of each pixel relative to the camera position at which the image was captured and generating a stereoscopic image for display by displacing in the horizontal direction of the viewable display each pixel in the stitched image by a distance determined by the depth map.

### Claim 17

An apparatus for providing, over a network, an image for recreation in a device, the image containing a background and a foreground object and the apparatus comprising: a detector operable to detect the position of the foreground object in the image and generate position information on dependence thereon; a remover operable to remove the foreground object from the image; and an output device operable to transfer to the device for display i) the image with the foreground object removed, ii) the removed foreground object and iii) the position information.

### Claim 18 (depends on 17)

An apparatus according to claim 17, comprising a scaler operable to scale the image with the foreground object removed therefrom; and dividing the scaled image into n segments, where n is an integer.

### Claim 19 (depends on 18)

An apparatus according to claim 18, wherein each of the n image segments is 1920×1080 pixels.

### Claim 20 (depends on 18)

An apparatus according to claim 18, comprising an encoder operable to encode each of the n segments, wherein for an area located at a position defined by the position information, encoding is performed at a higher bitrate compared with the remainder of the segment.

### Claim 21 (depends on 18)

An apparatus according to claim 18, comprising a provider operable to provide a supplemental area in at least one of the segments, the supplemental area being blank and inserting the removed foreground object into the supplemental area prior to transmission to the device.

### Claim 22 (depends on 17)

An apparatus according to claim 17, further wherein the output device is further operable to transfer to the device a depth map identifying the depth of each pixel relative to the camera position at which the image was captured.

### Claim 23

An apparatus for recreating an image containing a background and a foreground object, the apparatus comprising: a receiver operable to receive over a network i) the image with the foreground object removed, ii) the removed foreground object and iii) position information identifying the position of the foreground object in the image and an inserter operable to insert the foreground object into the image with the foreground object removed at a position defined by the position information.

### Claim 24 (depends on 23)

An apparatus according to claim 23, wherein the receiver is operable to receive the image with the foreground object removed as n segments where n is an integer and stitching the n segments together.

### Claim 25 (depends on 24)

An apparatus according to claim 24, wherein each of the n image segments is 1920×1080 pixels.

### Claim 26 (depends on 23)

An apparatus according to claim 23, wherein the receiver is operable to comprise receiving at least one of the segments with a supplemental area provided therein, the supplemental area having the removed foreground object inserted therein.

### Claim 27 (depends on 23)

An apparatus according to claim 23 wherein the receiver is operable to receive a depth map identifying the depth of each pixel in the image, and the apparatus comprises a generator operable to generate a disparity at the pixel position defined by the position information on the basis of the depth at that pixel position and the dimensions of the screen upon which the image to be recreated, and to generate a stereoscopic image formed of two images whereby in one of the images, a foreground object is inserted at a position defined by the position information and in the other image forming the stereoscopic image, and an inserter operable to insert a copy of the foreground object into the image at a position horizontally displaced from the position defined by the position information.

### Claim 28

An apparatus for providing, over a network, an image for recreation in a device, the image containing a background and a foreground object and the apparatus comprising: a detector operable to detect the position of the foreground object in the image and generating position information in dependence thereon; a scalar operable to scale the image; a divider operable to divide the scaled image into n segments, where n is an integer; an encoder operable to encode each of the n segments, wherein for an area located at a position defined by the position information, the encoding is performed at a higher bitrate compared with the remainder of the segment; and a transferring device operable to transfer the encoded image to the device.

### Claim 29 (depends on 28)

An apparatus according to claim 28, wherein each of the n image segments is 1440×540 pixels.

### Claim 30 (depends on 28)

An apparatus according to claim 28, wherein the transferring device is operable to transfer to the device a depth map identifying the depth of each pixel relative to the camera position at which the image was captured.

### Claim 31 (depends on 28)

An apparatus for recreating an image containing a background and a foreground object, the apparatus comprising: a receiver operable to receive over a network a scaled image produced by the apparatus of claim 28 divided into n segments, where n is an integer and stitching the n segments together.

### Claim 32 (depends on 31)

An apparatus according to claim 31, wherein the receiver is operable to receive a depth map identifying the depth of each pixel relative to the camera position at which the image was captured and generating a stereoscopic image for display by displacing in the horizontal direction of the viewable display each pixel in the stitched image by a distance determined by the depth map.

### Claim 33 (depends on 23)

An apparatus according to claim 23, wherein said apparatus is a games console.

### Claim 34 (depends on 31, 32)

An apparatus according to either claim 31 or 32, wherein said apparatus is a hand-held device.

### Claim 35 (depends on 17, 23)

A system comprising an apparatus according to claim 17 connected to a network and operable to communicate with an apparatus according to claim 23.

### Claim 36 (depends on 28, 31)

A system comprising an apparatus according to claim 28 connected to a network and operable to communicate with an apparatus according to claim 31.

### Claim 37 (depends on 1)

A computer program containing computer readable instructions which, when loaded onto a computer, configure the computer to perform a method according to claim 1.

### Claim 38 (depends on 37)

A storage medium configured to store the computer program of claim 37 therein or thereon.
