# US 2013/0258036 A1 · Motion Responsive Video Capture During a Video Conference

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0258036 A1
- Document ID: US20130258036A1
- Status: Published application
- Kind: A1
- Application number: 13/438,016
- Filed: 2012-04-03
- Published: 2013-10-03
- Earliest claimed: 2012-04-03
- Assignees: Cisco Technology, Inc.
- Inventors: Aaron Tong; Sung Fong Solomon Wu; Martin Rudolf Eppel
- Web page: https://patentgrep.com/patent/US20130258036A1

## CPC

- H04N 7/15: Conference systems
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- G06F 2200/1637: Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A mobile communication device that is outfitted with motion detection mechanisms is used to construct a viewport on audio-visual data. As the communication device is moved along a spatial trajectory, the viewport is positioned accordingly. A virtual space can be constructed such that the spatial trajectory is mapped into the virtual space and the user can navigate the virtual space by suitable movement of the communication device. Accordingly, a user can use natural body movements to navigate the virtual space through the viewport.

## Claims

### Claim 1

A method comprising:
generating audio-visual (AV) frames;
establishing a viewport to span a portion of each of the AV frames;
detecting motion of a user device along a spatial trajectory;
updating the portion spanned by the viewport in consecutive AV frames so as to correspond with the spatial trajectory; and
displaying the updated viewport of the consecutive AV frames on the user device.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the AV frames includes:
receiving a plurality of AV streams of source AV frames;
ordering the source AV frames of each of the AV streams in a spatial order; and
combining the source AV frames into the AV frames in accordance with the spatial order.

### Claim 3 (depends on 2)

The method of claim 2, wherein generating the AV frames includes:
determining an absence of dynamic data in any of the AV streams; and
excluding the AV frames of the AV streams absent the dynamic data from the combining.

### Claim 4 (depends on 3)

The method of claim 3 further comprising:
mapping AV data in the AV frames to a virtual space;
determining a trajectory in the virtual space from the spatial trajectory; and
determining an offset of the viewport for each of the consecutive AV frames according to the trajectory in the virtual space.

### Claim 5 (depends on 4)

The method of claim 4, wherein the user device is a handheld mobile communication device, wherein determining the trajectory in the virtual space includes forming the spatial trajectory in accordance with any body movement of a user holding the communication device.

### Claim 6 (depends on 5)

The method of claim 5, wherein determining the trajectory in the virtual space includes:
detecting rotation of the user device about one of orthogonal axes; and
mapping the rotation of the user device into a rotation of the viewport in the virtual space.

### Claim 7 (depends on 5)

The method of claim 5, further comprising:
dimensioning the viewport to correspond to boundaries of a display of the communication device.

### Claim 8 (depends on 1)

The method of claim 1, wherein generating comprises generating AV frames that are part of a video conference, and wherein detecting comprises detecting motion of the user device that is an endpoint of a participant in a video conference and the AV frames are video frames that contain one or more other participants in the video conference.

### Claim 9

An apparatus comprising:
a network interface device configured to enable communications over a network; and
a processor coupled to the network interface device and configured to:
generate audio-visual (AV) frames;
establish a viewport to span a portion of each of the AV frames;
receive signals representing motion detected of a user device along a spatial trajectory;
update the portion spanned by the viewport in consecutive AV frames so as to correspond with the spatial trajectory; and
generate display data for the updated viewport of the consecutive AV frames.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the processor is configured to:
receive a plurality of AV streams of source AV frames;
order the source AV frames of each of the AV streams in a spatial order; and
combine the source AV frames into the AV frames in accordance with the spatial order.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the processor is configured to:
determine an absence of dynamic data in any of the AV streams; and
exclude the AV frames of the AV streams absent the dynamic data from the combined source AV frames.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the processor is configured to:
map AV data in the AV frames to a virtual space;
determine a trajectory in the virtual space from the spatial trajectory; and
determine an offset of the viewport for each of the consecutive AV frames according to the trajectory in the virtual space.

### Claim 13 (depends on 12)

A system comprising the apparatus of claim 12, further comprising:
motion sensors configured to produce a signal responsive to movement of the user device along the spatial trajectory in accordance with any body movement of a user holding the user device.

### Claim 14 (depends on 13)

The system of claim 13, wherein the motion sensors are configured to generate the signal responsive to rotation of the user device about one of two orthogonal axes, and the user device further configured to map the rotation of the enclosure into a rotation of the viewport in the virtual space.

### Claim 15 (depends on 13)

The system of claim 13, further comprising:
a display associated with the user device, the viewport being dimensioned to correspond to boundaries of the display.

### Claim 16 (depends on 12)

The apparatus of claim 12, further comprising:
a plurality of cameras distributed about a space to generate the AV streams, the AV data being spatially ordered in the virtual space in accordance with the spatial distribution of the cameras.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the processor is configured to communicate in accordance with a teleconferencing protocol.

### Claim 18

A tangible computer readable medium having instructions encoded thereon that, when executed by a processor, are operable to:
generate audio-visual (AV) frames;
establish a viewport to span a portion of each of the AV frames;
receive signals representing motion detected of a user device along a spatial trajectory;
update the portion spanned by the viewport in consecutive AV frames so as to correspond with the spatial trajectory; and
generate display data for the updated viewport of the consecutive AV frames on the user device.

### Claim 19 (depends on 18)

The tangible computer readable medium of claim 18, further comprising instructions encoded thereon that, when executed by the processor, are operable to:
map AV data in the AV frames to a virtual space;
determine a trajectory in the virtual space from the spatial trajectory; and
determine an offset of the viewport for each of the consecutive AV frames according to the trajectory in the virtual space.

### Claim 20 (depends on 18)

The tangible computer readable medium of claim 18, further comprising instructions encoded thereon that, when executed by the processor, are operable to:
receive a plurality of AV streams of source AV frames;
order the source AV frames of each of the AV streams in a spatial order; and
combine the source AV frames into the AV frames in accordance with the spatial order.
