# US 2024/0196096 A1 · Merging webcam signals from multiple cameras

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0196096 A1
- Document ID: US20240196096A1
- Status: Published application
- Kind: A1
- Application number: 18/347,827
- Filed: 2023-07-06
- Published: 2024-06-13
- Earliest claimed: 2020-08-24
- Assignees: Owl Labs, Inc.
- Applicants: Owl Labs Inc.
- Inventors: Tom BUSHMAN; Ilya MOSKOVKO; Howard BROWN
- Web page: https://patentgrep.com/patent/US20240196096A1

## CPC

- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- H04N 5/06: Generation of synchronising signals
- H04N 23/611: where the recognised objects include parts of the human body
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- H04N 7/142: {Constructional details of the terminal equipment, e.g. arrangements of the camera and the display}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system includes a camera for capturing a first panorama view. The system determines a first bearing of a person within the first panorama view, and a first gaze direction of the person within the first panorama view. The system receives, from an external source, a second panorama view, a second bearing of the person within the second panorama view, and a second gaze direction of the person within the second panorama view. The system selects, by comparing the first gaze direction and the second gaze direction, a selected panorama view and a selected bearing of the person. The system forms a localized subscene video signal based on the selected panorama view along the selected bearing of the person. The system generates a stage view signal based on the localized subscene video signal and composites a composited signal comprising the stage view signal.

## Claims

### Claim 1

A system comprising:
a processor;
a camera operatively coupled to the processor and configured to capture a first panorama view;
a first communication interface operatively coupled to the processor; and
a memory storing computer-readable instructions that, when executed by the processor, cause the system to:
determine a first bearing of a person within the first panorama view;
determine a first gaze direction of the person within the first panorama view;
receive, from an external source and via the first communication interface, a second panorama view;
receive, from the external source and via the first communication interface, a second bearing of the person within the second panorama view;
receive, from the external source and via the first communication interface, a second gaze direction of the person within the second panorama view;
compare the first gaze direction and the second gaze direction;
select, based on comparing the first gaze direction and the second gaze direction, a selected panorama view from between the first panorama view and the second panorama view;
select, based on the selected panorama view, a selected bearing of the person from between the first bearing of the person and the second bearing of the person;
form a localized subscene video signal based on the selected panorama view along the selected bearing of the person;
generate a stage view signal based on the localized subscene video signal;
generate a scaled panorama view signal based on the first panorama view or the second panorama view;
composite a composited signal comprising the scaled panorama view signal and the stage view signal;
and
transmit the composited signal.

### Claim 2 (depends on 1)

The system of claim 1, comprising a second communication interface operatively coupled to the processor, the second communication interface being different from the first communication interface, and wherein the composited signal is transmitted via the second communication interface.

### Claim 3 (depends on 2)

The system of claim 2, wherein the first communication interface is a wireless interface and the second communication interface is a wired interface.

### Claim 4 (depends on 1)

The system of claim 1, comprising an audio sensor operatively coupled to the processor and configured to capture audio corresponding to the first panorama view, and wherein determining the first bearing of the person within the first panorama view is based on information from the audio sensor.

### Claim 5 (depends on 4)

The system of claim 4, wherein the computer-readable instructions, when executed by the processor, cause the system to:
receive audio corresponding to the second panorama view;
establish a common coordinate system of the camera and the external source;
determine an offset of a relative orientation between the camera and the external source in the common coordinate system; and
determine, based on the offset, that the first bearing of the person within the first panorama view is directed to a same location as the second bearing of the person in the second panorama view.

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the first gaze direction is determined as a first angle of a gaze of the person away from the camera;
the second gaze direction is a measurement of a second angle of the gaze of the person away from a video sensor of the external source; and
selecting the selected panorama view based on comparing the first gaze direction and the second gaze direction comprises selecting the first panorama view as the selected panorama view when the first angle is smaller than the second angle, or selecting the second panorama view as the selected panorama view when the second angle is smaller than the first angle.

### Claim 7 (depends on 1)

The system of claim 1, wherein:
the first gaze direction is determined as a first angle of a gaze of the person away from the camera;
the second gaze direction is a measurement of a second angle of the gaze of the person away from a video sensor of the external source; and
selecting the selected panorama view based on comparing the first gaze direction and the second gaze direction comprises selecting the first panorama view as the selected panorama view when the second angle is invalid and the first angle is smaller than a minimum threshold value, or selecting the second panorama view as the selected panorama view when the first angle is invalid and the second angle is smaller than the minimum threshold value.

### Claim 8 (depends on 1)

The system of claim 1, wherein:
the first gaze direction is determined as a first angle of a gaze of the person away from the camera;
the second gaze direction is a measurement of a second angle of the gaze of the person away from a video sensor of the external source; and
selecting the selected panorama view based on comparing the first gaze direction and the second gaze direction comprises selecting the first panorama view as the selected panorama view when a detected audio from the person is closer to the camera than the video sensor of the external source, or selecting the second panorama view as the selected panorama view when the detected audio of the person is closer to the video sensor of the external source than the camera.

### Claim 9 (depends on 1)

The system of claim 1, wherein the computer-readable instructions, when executed by the processor, cause the system to:
determine that the person is a speaker.

### Claim 10 (depends on 1)

The system of claim 1, wherein the computer-readable instructions, when executed by the processor, cause the system to:
determine a first coordinate map of the first panorama view;
receive, from the external source and via the first communication interface, a second coordinate map of the second panorama view;
determine a coordinate instruction associated with the first coordinate map of the first panorama view and the second coordinate map of the second panorama view;
determine, based on the coordinate instruction, a coordinate of a designated view in the first panorama view or the second panorama view; and
composite the designated view with the composited signal.

### Claim 11 (depends on 1)

The system of claim 1, wherein
the camera is configured to capture the first panorama view with a horizontal angle of 360 degrees; and
the second panorama view has a horizontal angle of 360 degrees.

### Claim 12

A method comprising:
capturing a first panorama view with a camera;
determining a first bearing of a person within the first panorama view;
determining a first gaze direction of the person within the first panorama view;
receiving, from an external source and via a first communication interface, a second panorama view;
receiving, from the external source via the first communication interface, a second bearing of the person within the second panorama view;
receiving, from the external source via the first communication interface, a second gaze direction of the person within the second panorama view;
comparing the first gaze direction and the second gaze direction;
selecting, based on comparing the first gaze direction and the second gaze direction, a selected panorama view from between the first panorama view and the second panorama view;
selecting, based on the selected panorama view, a selected bearing of the person from between the first bearing of the person and the second bearing of the person;
forming a localized subscene video signal based on the selected panorama view along the selected bearing of the person;
generating a stage view signal based on the localized subscene video signal;
generating a scaled panorama view signal based on the first panorama view or the second panorama view;
compositing a composited signal comprising the scaled panorama view signal and the stage view signal; and
transmitting the composited signal.

### Claim 13 (depends on 12)

The method of claim 12, wherein determining the first bearing of the person within the first panorama view is based on information from an audio sensor.

### Claim 14 (depends on 12)

The method of claim 12, wherein:
the first gaze direction is determined as a first angle of a gaze of the person away from the camera;
the second gaze direction is a measurement of a second angle of the gaze of the person away from a video sensor of the external source; and
selecting the selected panorama view based on comparing the first gaze direction and the second gaze direction comprises selecting the first panorama view as the selected panorama view when the first angle is smaller than the second angle, or selecting the second panorama view as the selected panorama view when the second angle is smaller than the first angle.

### Claim 15 (depends on 12)

The method of claim 12, wherein:
the first gaze direction is determined as a first angle of a gaze of the person away from the camera;
the second gaze direction is a measurement of a second angle of the gaze of the person away from a video sensor of the external source; and
selecting the selected panorama view based on comparing the first gaze direction and the second gaze direction comprises selecting the first panorama view as the selected panorama view when the second angle is invalid and the first angle is smaller than a minimum threshold value, or selecting the second panorama view as the selected panorama view when the first angle is invalid and the second angle is smaller than the minimum threshold value.

### Claim 16 (depends on 12)

The method of claim 12, wherein:
the first gaze direction is determined as a first angle of a gaze of the person away from the camera;
the second gaze direction is a measurement of a second angle of the gaze of the person away from a video sensor of the external source; and
selecting the selected panorama view based on comparing the first gaze direction and the second gaze direction comprises selecting the first panorama view as the selected panorama view when a detected audio from the person is closer to the camera than the video sensor of the external source, or selecting the second panorama view as the selected panorama view when the detected audio of the person is closer to the video sensor of the external source than the camera.

### Claim 17 (depends on 12)

The method of claim 12, comprising:
determining a first coordinate map of the first panorama view;
receiving, from the external source, a second coordinate map of the second panorama view via the first communication interface;
determining a coordinate instruction associated with the first coordinate map of the first panorama view and the second coordinate map of the second panorama view;
determining a coordinate of a designated view in the first panorama view or the second panorama view based on the coordinate instruction; and
further compositing the designated view with the composited signal.

### Claim 18

One or more non-transitory computer-readable media storing instructions that, when executed by at least one computing device, cause the at least one computing device to perform a method comprising:
capturing a first panorama view with a camera;
determining a first bearing of a person within the first panorama view;
determining a first gaze direction of the person within the first panorama view;
receiving, from an external source and via a first communication interface, a second panorama view;
receiving, from the external source via the first communication interface, a second bearing of the person within the second panorama view;
receiving, from the external source via the first communication interface, a second gaze direction of the person within the second panorama view;
comparing the first gaze direction and the second gaze direction;
selecting, based on comparing the first gaze direction and the second gaze direction, a selected panorama view from between the first panorama view and the second panorama view;
selecting, based on the selected panorama view, a selected bearing of the person from between the first bearing of the person and the second bearing of the person;
forming a localized subscene video signal based on the selected panorama view along the selected bearing of the person;
generating a stage view signal based on the localized subscene video signal;
generating a scaled panorama view signal based on the first panorama view or the second panorama view;
compositing a composited signal comprising the scaled panorama view signal and the stage view signal; and
transmitting the composited signal.

### Claim 19 (depends on 18)

The one or more non-transitory computer readable media of claim 18, wherein the method comprises determining the first bearing of the person within the first panorama view is based on information from an audio sensor.

### Claim 20 (depends on 18)

The one or more non-transitory computer readable media of claim 18, wherein:
the first gaze direction is determined as a first angle of a gaze of the person away from the camera;
the second gaze direction is a measurement of a second angle of the gaze of the person away from a video sensor of the external source; and
selecting the selected panorama view based on comparing the first gaze direction and the second gaze direction comprises selecting the first panorama view as the selected panorama view when the first angle is smaller than the second angle, or selecting the second panorama view as the selected panorama view when the second angle is smaller than the first angle.
