# US 10,484,669 B2 · Inertial measurement unit progress estimation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,484,669 B2
- Document ID: US10484669B2
- Status: Granted
- Kind: B2
- Application number: 16/225,242
- Filed: 2018-12-19
- Published: 2019-05-16
- Granted: 2019-11-19
- Earliest claimed: 2017-05-22
- Assignees: Fyusion, Inc.
- Applicants: Fyusion, Inc.
- Inventors: Alexander Jay Bruen Trevor; Chris Beall; Stefan Johannes Josef Holzer; Radu Bogdan Rusu
- Web page: https://patentgrep.com/patent/US10484669B2

## CPC

- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- G01S 3/00: Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic or electromagnetic waves, or particle emission, not having a directional significance, are being received (position-fixing by co-ordinating a plurality of determinations of direction or position lines G01S5/00)
- G06T 7/74: {involving reference images or patches}
- G06T 15/205: {Image-based rendering}
- H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation}
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N 23/10: for generating image signals from different wavelengths
- H04N 23/611: where the recognised objects include parts of the human body
- H04N 23/63: by using electronic viewfinders
- H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory}
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/30244: Camera pose
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. In particular, a multi-view interactive digital media representation can be generated from live images captured from a camera. The live images can include an object. An angular view of the object captured in the live images can be estimated using sensor data from an inertial measurement unit. The multi-view interactive digital media representation can include a plurality of images where each of the plurality of images includes the object from a different camera view. When the plurality of images is output to a display, the object can appear to undergo a 3-D rotation through the determined angular view where the 3-D rotation of the object is generated without a 3-D polygon model of the object.

## Claims

### Claim 1

A method comprising:
on a mobile device including a processor, a memory, a camera, an inertial measurement unit, a microphone, a GPS sensor and a touchscreen display:
receiving a request to generate a multi-view interactive digital media representation of a car;
receiving a sequence of live images from the camera on the mobile device, the live images including 2-D pixel data, the sequence of live images being captured as the camera moves along a path, an orientation of the camera varying along the path such that the car in the sequence of the live images is captured from a plurality of camera views;
based upon sensor data from the inertial measurement unit, determining angular changes in the orientation of the camera along the path;
based upon the angular changes, determining an angular view of the car captured in the sequence of the live images; and
generating from the sequence of the live images the multi-view interactive digital media representation, the multi-view interactive digital media representation including a plurality of images, each of the plurality of images including the car from a different camera view such that when the plurality of images is output to the touchscreen display the car appears to undergo a 3-D rotation through the angular view, the 3-D rotation of the car being generated without a 3-D polygon model of the car; and
outputting a value of the angular view of the car captured in the multi-view interactive digital media representation.

### Claim 2 (depends on 1)

The method of claim 1, wherein the angular changes are determined about an axis.

### Claim 3 (depends on 2)

The method of claim 2, wherein the axis is aligned with Earth's gravity vector.

### Claim 4 (depends on 2)

The method of claim 2, further comprising: receiving coordinate information which define a direction of the axis in an orthogonal coordinate system associated with Earth's gravity vector.

### Claim 5 (depends on 2)

The method of claim 2, further comprising based upon the orientation of the camera along the path, determining a first direction of rotation of the camera about the axis.

### Claim 6 (depends on 5)

The method of claim 5, wherein only angular changes in the first direction are used to determine the angular view of the car captured in the sequence of the live images.

### Claim 7 (depends on 6)

The method of claim 6, further comprising determining a first angular change in a second direction opposite the first direction, determining a subsequent second angular change in the first direction, the second angular change being determined based upon the orientation of the camera prior to the first angular change.

### Claim 8 (depends on 5)

The method of claim 5, further comprising determining angular changes in the first direction of rotation and in a second direction opposite the first direction wherein the angular view of the cared captured in the sequence of the live images is determined using both the angular changes in the first direction and the second direction.

### Claim 9 (depends on 2)

The method of claim 2, further comprising determining a plane perpendicular to the axis, projecting the orientation of the camera into the plane perpendicular to the axis, determining the angular changes in the plane perpendicular to the axis.

### Claim 10 (depends on 2)

The method of claim 2, further comprising, determining a first direction of rotation of the camera about the axis.

### Claim 11 (depends on 10)

The method of claim 10, wherein only angular changes in the first direction are used to determine the angular view of the car captured in the sequence of the live images.

### Claim 12 (depends on 11)

The method of claim 11, further comprising determining a first angular change in a second direction opposite the first direction, determining a subsequent second angular change in the first direction, wherein the second angular change is determined based upon the orientation of the camera prior to the first angular change.

### Claim 13 (depends on 10)

The method of claim 10, further comprising determining angular changes in the first direction of rotation and in a second direction opposite the first direction wherein the angular view of the cared captured in the sequence of the live images is determined using both the angular changes in the first direction and the second direction.

### Claim 14 (depends on 2)

The method of claim 2, further comprising receiving a specification of a first direction of rotation of the camera around the axis along the path.

### Claim 15 (depends on 1)

The method of claim 1, further comprising, 1) prior to receiving the sequence of the live images, receiving a specification of a first value of the angular view, 2) as the sequence of the live images are received and based upon the determined angular changes, determining a current value of the angular view and 3) determining whether the current value of the angular view is greater than the first value of the angular view.

### Claim 16 (depends on 15)

The method of claim 15, further comprising outputting to the display the current value of the angular view.

### Claim 17 (depends on 15)

The method of claim 15, further comprising determining the current value of the angular is greater than the first value of the angular view, outputting to the display a notification the first value of the angular view is exceeded and ending capture of the sequence of the live images.

### Claim 18 (depends on 15)

The method of claim 15, wherein the first value of the angular view is three hundred and sixty degrees.

### Claim 19 (depends on 1)

The method of claim 1, wherein the multi-view interactive digital media representation of the car includes a three dimensional representation of one or more components of the components of the car.

### Claim 20 (depends on 1)

The method of claim 1, wherein the angular changes are determined based upon one or more of accelerometer data, gyroscopic data, magnetometer data or combinations thereof.