# US 12,762,807 B2 · Head-mounted display with pass-through imaging

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,762,807 B2
- Document ID: US12762807B2
- Status: Granted
- Kind: B2
- Application number: 18/617,452
- Filed: 2024-03-26
- Published: 2024-08-08
- Granted: 2026-10-13
- Earliest claimed: 2019-04-23
- Assignees: Valve Corporation
- Applicants: Valve Corporation
- Inventors: Charles N. Lohr; Gordon Wayne Stoll
- Web page: https://patentgrep.com/patent/US12762807B2

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G02B 27/0172: {characterised by optical features}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06T 15/20: Perspective computation
- G06T 2200/04: involving 3D image data
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

This application describes a head-mounted display (HMD) for use in virtual-reality (VR) environments. The systems and methods described herein may determine information about a real-world environment surrounding the user, a location of the user within the real-world environment, and/or a pose of the user within the real-world environment. Such information may allow the HMD to display images of the real-world environment in a pass-through manner and without detracting the user from the VR environment. In some instances, the HMD may pass-through images of the real-world environment based on one or more triggering events.

## Claims

### Claim 1

A system comprising:
a head-mounted display comprising:
a display; and
a camera;
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:
determining an offset of the camera relative to an eye of a user;
generating a depth map;
modifying points within the depth map based at least in part on the offset to obtain a modified depth map;
generating image data representing a real-world environment based at least in part on the modified depth map; and
displaying the image data via the display.

### Claim 2 (depends on 1)

The system of claim 1, wherein:
the camera is a first camera;
the offset is a first offset;
the eye is a first eye;
the head-mounted display further comprises a second camera;
the acts further comprise determining a second offset of the second camera relative to a second eye of the user; and
the modifying of the points within the depth map is further based on the second offset.

### Claim 3 (depends on 1)

The system of claim 1, wherein:
the generating of the image data comprises generating first image data based at least in part on superimposing, overlaying, or projecting second image data captured via the camera onto the modified depth map; and
the displaying of the image data comprises displaying the first image data.

### Claim 4 (depends on 1)

The system of claim 1, wherein the depth map represents distances between the user and objects within the real-world environment.

### Claim 5 (depends on 1)

The system of claim 1, wherein:
the head-mounted display further comprises one or more sensors; and
the one or more sensors are utilized to generate the depth map.

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the camera is a first camera;
the head-mounted display further comprises a second camera; and
stereoscopic camera imaging is utilized to generate the depth map.

### Claim 7 (depends on 1)

The system of claim 1, wherein the offset corresponds to at least one of:
a horizontal displacement of the camera relative to the eye;
a vertical displacement of the camera relative to the eye; or
a depth wise displacement of the camera relative to the eye.

### Claim 8

A method comprising:
determining, by a processor, an offset of a camera of a head-mounted display relative to an eye of a user;
generating, by the processor, a depth map;
modifying, by the processor, points within the depth map based at least in part on the offset to obtain a modified depth map;
generating, by the processor, image data representing a real-world environment based at least in part on the modified depth map; and
displaying the image data on the head-mounted display.

### Claim 9 (depends on 8)

The method of claim 8, wherein:
the camera is a first camera;
the offset is a first offset;
the eye is a first eye;
the method further comprises determining, by the processor, a second offset of a second camera of the head-mounted display relative to a second eye of the user; and
the modifying of the points within the depth map is further based on the second offset.

### Claim 10 (depends on 8)

The method of claim 8, wherein:
the generating of the image data comprises generating first image data based at least in part on superimposing, overlaying, or projecting second image data captured via the camera onto the modified depth map; and
the displaying of the image data comprises displaying the first image data on the head-mounted display.

### Claim 11 (depends on 8)

The method of claim 8, wherein the depth map represents a portion of the real-world environment.

### Claim 12 (depends on 8)

The method of claim 8, further comprising receiving an indication of a trigger event, and wherein the displaying of the image data is based at least in part on the receiving of the indication.

### Claim 13 (depends on 12)

The method of claim 12, wherein the indication corresponds to:
detecting that the user is approaching a boundary of the real-world environment; or
detecting that a visitor is within the real-world environment.

### Claim 14 (depends on 8)

The method of claim 8, wherein the image data is displayed by overlaying the image data on virtual content associated with a virtual reality environment presented on the head-mounted display.

### Claim 15

A system comprising:
a head-mounted display;
a camera disposed on the head-mounted display;
one or more processors; and
memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:
determining a location of the camera relative to an eye of a user;
generating a depth map;
modifying points within the depth map based at least in part on the location to obtain a modified depth map;
generating image data representing a real-world environment based at least in part on the modified depth map; and
causing the image data to be displayed on the head-mounted display.

### Claim 16 (depends on 15)

The system of claim 15, wherein:
the camera is a first camera;
the location is a first location;
the eye is a first eye;
the system further comprises a second camera disposed on the head-mounted display;
the acts further comprise determining a second location of the second camera relative to a second eye of the user; and
the modifying of the points within the depth map is further based on the second location.

### Claim 17 (depends on 15)

The system of claim 15, wherein:
the generating of the image data comprises generating first image data based at least in part second image data captured via the camera; and
the causing of the image data to be displayed comprises causing the first image data to be displayed on the head-mounted display.

### Claim 18 (depends on 17)

The system of claim 17, wherein:
the acts further comprise generating a 3D mesh of a portion of the real-world environment based at least in part on the location; and
the generating of the image data comprises generating the first image data based at least in part on superimposing, overlaying, or projecting the second image data onto the 3D mesh.

### Claim 19 (depends on 17)

The system of claim 17, wherein:
the generating of the image data comprises generating the first image data based at least in part on superimposing, overlaying, or projecting the second image data onto the modified depth map.

### Claim 20 (depends on 15)

The system of claim 15, wherein the location corresponds to a displacement of the camera relative to the eye.