# US 12,603,091 B2 · Immersive collaboration of remote participants via media displays

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,603,091 B2
- Document ID: US12603091B2
- Status: Granted
- Kind: B2
- Application number: 18/555,129
- Filed: 2022-04-14
- Published: 2024-07-18
- Granted: 2026-04-14
- Earliest claimed: 2017-04-26
- Assignees: View Operating Corporation
- Applicants: View, Inc.
- Inventors: Tanya Makker; Nitesh Trikha; Brian Lee Smith; Keivan Ebrahimi; Todd Daniel Antes; Aditya Dayal; Amit Sarin
- Web page: https://patentgrep.com/patent/US12603091B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- E06B 3/6722: {with adjustable passage of light}
- E06B 9/24: Screens or other constructions affording protection against light, especially against sunshine; Similar screens for privacy or appearance; {Slat blinds}(operating, guiding or securing devices or arrangements for roll-type closures E06B9/56; free-hanging flexible screens A47H23/00)
- G02F 1/163: Operation of electrochromic cells, e.g. electrodeposition cells; Circuit arrangements therefor
- G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- E06B 2009/2464: {featuring transparency control by applying voltage, e.g. LCD, electrochromic panels}
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- E06B: FIXED OR MOVABLE CLOSURES FOR OPENINGS IN BUILDINGS, VEHICLES, FENCES OR LIKE ENCLOSURES IN GENERAL, e.g. DOORS, WINDOWS, BLINDS, GATES (shades or blinds for greenhouses A01G9/22; curtains A47H; lids for car boots or bonnets B62D25/10; skylights E04B7/18; sunshades, awnings E04F10/00)
- G02F: OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An immersive digital experience for video conferencing simulates common presence of a virtual participant in a local environment. Such simulation may include (i) using a transparent media display having a portion of its pixels projecting the virtual participant's body image while keeping at least a portion of the background transparent (e.g., to visible light), (ii) disposing sensor(s) (e.g., camera) behind the transparent media display at the gaze of the participant, and/or (iii) using added virtual overlays (e.g., of plants, memorabilia, and/or furniture) to the virtual image (e.g., that are consistent with the local environment), e.g., to provide a sense of depth ranging from the overlays to the virtual participant projection and to the background showing through the transparent media display.

## Claims

### Claim 1

An apparatus for digital collaboration, the apparatus comprising:
a digital collaboration unit having a media display disposed within a first location; and
one or more controllers comprising circuitry;
wherein:
the media display comprises an electrochromic display that is at least partially transparent to visible light;
the one or more controllers are configured to display, on the media display, a media stream received from a remote processor, the media stream including a depiction of a remotely located remote user, to redact a background portion of the media stream from being displayed on the media display, the redacted portion comprising a region around the depiction of the remote user, and to enable a local user to view, through the redacted portion, at least a portion of the first location behind the media display; and
the one or more controllers are further configured to display, within the redacted portion, a virtual overlay based on the first location, the virtual overlay comprising a depiction of an object, and wherein the depiction of the object appears continuous with a physical object at the first location.

### Claim 2 (depends on 1)

The apparatus for digital collaboration of claim 1 wherein the one or more controllers are further configured to adjust a transparency of the electrochromic display based, at least in part, on an element selected from the group consisting of: a position of a sun, a weather condition, images displayed on the media display, and a reading from one or more sensors.

### Claim 3 (depends on 1)

The apparatus for digital collaboration of claim 1 wherein the electrochromic display comprises a camera configured for capturing an image of the local user.

### Claim 4 (depends on 3)

The apparatus for digital collaboration of claim 3 wherein the camera is configured for a focus that is greater than the distance between the camera and the electrochromic display to avoid capturing images displayed on the media display.

### Claim 5 (depends on 1)

The apparatus for digital collaboration of claim 1 wherein the electrochromic display comprises a wireless charger.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein a position of the depiction of the remotely located remote user is based on the depiction of the object.

### Claim 7

A method for digital collaboration, the method comprising:
receiving a media stream, the media stream including a depiction of a remote user;
redacting a background portion of the media stream, the redacted portion comprising a region around the depiction of the remote user;
rendering a portion of an electrochromic display transparent, the transparent portion corresponding to at least a portion of the redacted portion of the media stream;
displaying, on the electrochromic display, the redacted media stream;
enabling a local user to view, through the transparent portion of the electrochromic display, at least a portion of a first location behind the electrochromic display; and
displaying, within the redacted portion, a virtual overlay based on the first location, the virtual overlay comprising a depiction of an object, and wherein the depiction of the object appears continuous with a physical object at the first location.

### Claim 8 (depends on 7)

The method for digital collaboration of claim 7 further comprising displaying on the electrochromic display, within the redacted portion, a virtual overlay based on the first location.

### Claim 9 (depends on 7)

The method for digital collaboration of claim 7 further comprising adjusting a transparency of the electrochromic display based, at least in part, on an element selected from the group consisting of: a position of a sun, a weather condition, images displayed on the media display, and a reading from one or more sensors.

### Claim 10 (depends on 7)

The method for digital collaboration of claim 7 further comprising capturing an image of the local user with a camera built into the electrochromic display.

### Claim 11 (depends on 10)

The method for digital collaboration of claim 10 further comprising setting the camera with a focus that is greater than the distance between the camera and the electrochromic display to avoid capturing images displayed on the electrochromic display.