# US 2020/0267375 A1 · Efficient image population from captured scene

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0267375 A1
- Document ID: US20200267375A1
- Status: Published application
- Kind: A1
- Application number: 16/276,831
- Filed: 2019-02-15
- Published: 2020-08-20
- Earliest claimed: 2019-02-15
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Ashraf A. Michail; Yang You; Michael G. Boulton
- Web page: https://patentgrep.com/patent/US20200267375A1

## CPC

- G06T 15/205: {Image-based rendering}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- H04N 13/275: from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals
- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- H04N 19/513: Processing of motion vectors
- H04N 2013/0085: {Motion estimation from stereoscopic image signals}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for efficiently populating a display is provided. The method can include identifying a point at which a world ray intersects a capture surface defined by capture points of a scene, identifying a capture point closest to the identified point, generating a motion vector based on the motion vectors for each of two directly adjacent capture points, identifying a vector in the generated motion vector at a location at which the world ray intersects an image surface, and providing a pixel value from the image data of the capture point, the pixel value corresponding to a location in the image surface at which a vector of the generated motion vector points to the location at which the world ray intersects the image surface within a threshold distance or after a specified number of iterations.

## Claims

### Claim 1

A virtual reality (VR) system comprising:
a memory device including, for each capture point of capture points of a scene, image data, two directly adjacent capture points, and motion vectors for each of the two directly adjacent capture points; and
processing circuitry to perform operations for populating a VR display, the operations comprising:
identifying a point at which a world ray intersects a capture surface defined by the capture points;
identifying a capture point closest to the identified point;
generating a motion vector based on the motion vectors for each of the two directly adjacent capture points;
identifying a vector in the generated motion vector at a location at which the world ray intersects an image surface; and
providing a pixel value from the image data of the capture point, the pixel value corresponding to a location in the image surface at which a vector of the generated motion vector points to the location at which the world ray intersects the image surface within a threshold distance.

### Claim 2 (depends on 1)

The VR system of claim 1, wherein the memory further includes data identifying a cell and an associated capture point identifying the capture point closest to all points in the cell, and wherein identifying the capture point closest to the identified point includes identifying the cell to which the identified point maps and the associated capture point.

### Claim 3 (depends on 1)

The VR system of claim 1, wherein the motion vectors for the two directly adjacent capture points form an orthogonal basis.

### Claim 4 (depends on 1)

The VR system of claim 1, wherein providing the pixel value from the image data of the capture point includes identifying a second location corresponding to the identified location minus the identified vector.

### Claim 5 (depends on 4)

The VR system of claim 4, wherein providing the pixel value from the image data of the capture point further includes identifying a second vector of the generated vectors at the second location.

### Claim 6 (depends on 5)

The VR system of claim 5, wherein providing the pixel value from the image data of the capture point further includes determining a difference between the location and a third location to which the second vector points.

### Claim 7 (depends on 6)

The VR system of claim 6, wherein providing the pixel value from the image data of the capture point further includes comparing the determined difference to a threshold distance and in response to determining the determined difference is less than the specified threshold returning the pixel of the image data corresponding to the second location.

### Claim 8

A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
identifying a point at which a world ray intersects a capture surface defined by capture points;
identifying a capture point closest to the identified point;
generating a motion vector based on motion vectors for each of the two directly adjacent capture points;
identifying a vector in the generated motion vector at a location at which the world ray intersects an image surface; and
providing a pixel value from the image data of the capture point, the pixel value corresponding to a location in the image surface at which a vector of the generated motion vector points to the location at which the world ray intersects the image surface within a threshold distance.

### Claim 9 (depends on 8)

The non-transitory machine-readable medium of claim 8, wherein identifying the capture point closest to the identified point includes identifying a cell to which the identified point maps and the associated capture point.

### Claim 10 (depends on 8)

The non-transitory machine-readable medium of claim 8, wherein the motion vectors for the two directly adjacent capture points form an orthogonal basis.

### Claim 11 (depends on 8)

The non-transitory machine-readable medium of claim 8, wherein providing the pixel value from the image data of the capture point includes identifying a second location corresponding to the identified location minus the identified vector.

### Claim 12 (depends on 11)

The non-transitory machine-readable medium of claim 11, wherein providing the pixel value from the image data of the capture point further includes identifying a second vector of the generated vectors at the second location.

### Claim 13 (depends on 12)

The non-transitory machine-readable medium of claim 12, wherein providing the pixel value from the image data of the capture point further includes determining a difference between the location and a third location to which the second vector points.

### Claim 14 (depends on 13)

The non-transitory machine-readable medium of claim 13, wherein providing the pixel value from the image data of the capture point further includes comparing the determined difference to a threshold distance and in response to determining the determined difference is less than the specified threshold returning the pixel of the image data corresponding to the second location.

### Claim 15

A method for populating a virtual reality (VR) headset display, the method comprising:
identifying a point at which a world ray intersects a capture surface defined by capture points of a scene;
identifying a capture point closest to the identified point;
generating a motion vector based on the motion vectors for each of two directly adjacent capture points;
identifying a vector in the generated motion vector at a location at which the world ray intersects an image surface; and
providing a pixel value from the image data of the capture point, the pixel value corresponding to a location in the image surface at which a vector of the generated motion vector points to the location at which the world ray intersects the image surface within a threshold distance or after a specified number of iterations.

### Claim 16 (depends on 15)

The method of claim 15, wherein providing the pixel value from the image data of the capture point includes identifying a second location corresponding to the identified location minus the identified vector.

### Claim 17 (depends on 16)

The method system of claim 16, wherein providing the pixel value from the image data of the capture point further includes identifying a second vector of the generated vectors at the second location.

### Claim 18 (depends on 17)

The method of claim 17, wherein providing the pixel value from the image data of the capture point further includes determining a difference between the location and a third location to which the second vector points.

### Claim 19 (depends on 18)

The method of claim 18, wherein providing the pixel value from the image data of the capture point further includes comparing the determined difference to a threshold distance and in response to determining the determined difference is less than the specified threshold returning the pixel of the image data corresponding to the second location.

### Claim 20 (depends on 15)

The method of claim 15, wherein identifying the capture point closest to the identified point includes identifying a cell to which the identified point maps and an associated capture point of the cell.
