# US 7,333,133 B2 · Recursive least squares approach to calculate motion parameters for a moving camera

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 7,333,133 B2
- Document ID: US7333133B2
- Status: Granted
- Kind: B2
- Application number: 10/814,711
- Filed: 2004-03-31
- Published: 2004-12-23
- Granted: 2008-02-19
- Earliest claimed: 2004-03-31
- Assignees: Spatial Integrated Systems, Inc.
- Inventors: Samuel Henry Chang; J. Joseph Fuller; Ali Farsaie; Leslie Ray Elkins
- Web page: https://patentgrep.com/patent/US7333133B2

## CPC

- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method and system for estimating camera motion parameters and determining a filter to correct for camera motion errors. The estimating method and system includes obtaining an observation point set including a plurality of observed point vectors, computing a plurality of motion output vectors by performing a recursive least squares (RLS) process based on a plurality of motion parameter vectors, and comparing the plurality of motion output vectors to the plurality of observed point vectors. The filter determining method and system includes determining a plurality of desired motion point vectors, computing a plurality of estimated motion point vectors by means of an RLS algorithm, and computing the filter based on a difference between the plurality of estimated motion point vectors and the plurality of desired motion point vectors.

## Claims

### Claim 1

A method for estimating camera motion parameters, the method comprising:
obtaining an observation point set including a plurality of observed point vectors;
computing a plurality of motion output vectors by performing a recursive least squares (RLS) process based on a plurality of motion parameter vectors; and,
comparing the plurality of motion output vectors to the plurality of observed point vectors;
wherein the computing a plurality of motion output vectors and the comparing the plurality of motion output vectors to the plurality of observed point vectors comprise:
initializing a first covariance matrix to a positive value; setting a first motion parameter estimate vector to an initial value;
determining each of the plurality of motion parameter vectors, the determining comprising: computing a current covariance matrix;
computing a current motion parameter estimate vector based on the current covariance matrix; computing a current motion output vector and a current state variable matrix based on the current covariance matrix; and,
comparing the current motion output vector to the observation point set; and, repeating the determining a predetermined number of times;
wherein: the computing a current covariance matrix comprises computing Pt=Pt−1+Pt−1φt[I+φt′Pt−1φt]−1φt−1′Pt−2, Pt being the current covariance matrix, Pt−1 being the prior current covariance matrix, Pt−2 being the second prior current covariance matrix, φt being the current state variable matrix, φt−1 being the prior current state variable matrix, and I being the identity matrix; and,
the computing a current motion parameter estimate vector comprises computing {circumflex over (θ)}t={circumflex over (θ)}t−1+[Pt−1φt[I+φ1′Pt−1φt]−1][yt−φt{circumflex over (θ)}t−1], {circumflex over (θ)}t being the current motion parameter estimate, {circumflex over (θ)}t−1 being the prior current motion parameter estimate, and y, being the current motion output vector.

### Claim 2 (depends on 1)

The method as set forth in claim 1, wherein the obtaining includes obtaining m number of points and n number of observations for each of the m number of points; and the repeating includes repeating the determining n times.

### Claim 3 (depends on 2)

The method as set forth in claim 2, wherein the setting a first motion parameter estimate vector comprises setting the first motion parameter estimate vector equal to [1 0 0 0 0 0 0]′.

### Claim 4 (depends on 1)

The method as set forth in claim 1, wherein: the determining further includes computing a random noise vector of the form e=[e1 e2 e3];
the computing a current motion parameter estimate vector includes computing the current motion parameter estimate vector which includes a plurality of noise vector transformation parameters; and,
the computing a current motion output vector includes computing the current motion output vector which includes a transformation of the random noise vector.

### Claim 5 (depends on 4)

The method as set forth in claim 4, wherein the obtaining includes obtaining m number of points and n number of observations for each of the m number of points; and the repeating includes repeating the determining n times.

### Claim 6 (depends on 5)

The method as set forth in claim 5, wherein the setting a first motion parameter estimate vector comprises setting the first motion parameter estimate vector equal to [1 0 0 0 0 0 0 0 0 0]′.

### Claim 7 (depends on 6)

The method as set forth in claim 6, wherein:
the computing a current covariance matrix comprises computing Pt=Pt−1+Pt−1φt[I+φt′Pt−1φt]−1φt−1 ′Pt−2, Pt being the current covariance matrix, Pt−1 being the prior current covariance matrix, Pt−2 being the second prior current covariance matrix, φt being the current state variable matrix including one of the plurality of random noise vectors, φt−1 being the prior current state variable matrix, and I being the identity matrix; and,
the computing a current motion parameter estimate vector comprises computing {circumflex over (θ)}t={circumflex over (θ)}t−1+[Pt−1φt[I+φt′Pt−1φt]−1][yt−φt{circumflex over (θ)}t−1], {circumflex over (θ)}t being the current motion parameter estimate including the transformed one of the plurality of noise vectors, {circumflex over (θ)}t−1 being the prior current motion parameter estimate, and yt being the current motion output vector.

### Claim 8

A method for determining a filter to correct for camera motion errors, the method comprising:
determining a plurality of desired motion point vectors;
computing a plurality of estimated motion point vectors by means of an RLS algorithm; and,
computing the filter based on a difference between the plurality of estimated motion point vectors and the plurality of desired motion point vectors,
wherein:
the determining a plurality of desired motion point vectors comprises determining the desired motion as described by Xt+1=RdXt+Td, where Xt is the motion output matrix at time t, Xt−1 is the motion output matrix at time t+1, Rd is the desired rotation matrix, and Td is the desired translation vector;
the computing a plurality of estimated motion point vectors comprises computing the estimated motion as described by Xt−1=ReXt+Te+cee, where Re is the estimated rotation matrix, Te is the estimated translation vector, ce is a noise transformation matrix, and e is a random noise vector; and,
the computing the filter comprises:
computing a filter rotation matrix according to Rf=Re −Rd; and,
computing a filter translation vector according to Tf=Te−Td.

### Claim 9 (depends on 8)

The method as set forth in claim 8, further comprising computing a corrected output according to Xt−1=RfXt+Tf+cee.

### Claim 10 (depends on 9)

The method as set forth in claim 9, wherein the RLS algorithm comprises:
obtaining an observation point set including a plurality of observed vectors;
initializing a first covariance matrix to a positive value;
setting a first motion parameter estimate vector to an initial value;
determining each of a plurality of motion parameter vectors, the determining each of a plurality of motion parameter vectors comprising:
computing a current covariance matrix;
computing a random noise vector;
computing a current motion parameter estimate vector based on the current covariance matrix, the current motion parameter estimate vector including the noise transformation matrix;
computing a current motion output vector and a current state variable matrix based on the current covariance matrix, the current motion output vector including the transformed random noise vector; and,
comparing the current motion output vector to the observation point set; and,
repeating the determining a predetermined number of times.

### Claim 11 (depends on 10)

The method as set forth in claim 10, wherein the obtaining includes obtaining m number of points and n number of observations for each of the m number of points; and the repeating includes repeating the determining n times.

### Claim 12 (depends on 11)

The method as set forth in claim 11, wherein:
the initializing a first covariance matrix comprises initializing the first covariance matrix to a large positive value; and,
the setting a first motion parameter estimate vector comprises setting the first motion parameter estimate vector to [1 0 0 0 0 0 0 0 0 0]′.

### Claim 13 (depends on 12)

The method as set forth in claim 12, wherein:
the computing a current covariance matrix comprises computing Pt=Pt−1+Pt−1φt[I+φt′Pt−1φt]−1φt−1′Pt−2, Pt being the current covariance matrix, Pt−1 being the prior current covariance matrix, Pt−2 being the second prior current covariance matrix, φt being the current state variable matrix, φt−1 being the prior current state variable matrix, and I being the identity matrix; and,
the computing a current motion parameter estimate vector comprises computing {circumflex over (θ)}t={circumflex over (θ)}t−1+[Pt−1φt[I+φt′Pt−1φt]−1][yt−φt{circumflex over (θ)}t−1], {circumflex over (θ)}t being the current motion parameter estimate, {circumflex over (θ)}t−1 being the prior current motion parameter estimate, and yt being the current motion output vector.

### Claim 14

A system for estimating and filtering camera motion parameters, the system comprising:
a movable digital camera for generating a plurality of 2-dimensional images of a scene or object;
a control means for translating and rotating the camera along a predetermined trajectory;
a computer system for receiving and processing the plurality of 2-dimensional images, the computer system including:
a user interface for receiving instructions from a user and for providing output to a user;
an image input means for receiving the plurality of 2-dimensional images from the camera;
a storage means for storing programs and the plurality of images;
a program to determine a desired motion, the desired motion described by Xt+1=RdXt+Td, where Xt is the motion output matrix at time t, Xt+1 is the motion output matrix at time t+1, Rd is the desired rotation matrix, and Td is the desired translation vector; and,
a program to compute an estimated motion by means of an RLS program, the estimated motion described by Xt+1=ReXt+Te+cee, where Re is the estimated rotation matrix, Te is the estimated translation vector, ce is a noise transformation matrix, and e is a random noise vector.

### Claim 15 (depends on 14)

The system as set forth in claim 14, further comprising:
a program to compute a filter rotation matrix according to Rf=Re−Rd;
a program to compute a filter translation vector according to Tf=Te−Td; and,
a program to compute a corrected output according to Xt+1=RfXt+Tf+cee.

### Claim 16 (depends on 14)

The system as set forth in claim 14, wherein the RLS program includes:
a program to obtain an observation point set including a plurality of observed vectors;
a program to initialize a first covariance matrix to a positive value;
a program to set a first motion parameter estimate vector to an initial value;
a program to determine each of a plurality of motion parameter vectors, including:
a program to compute a current covariance matrix;
a program to compute a random noise vector;
a program to compute a current motion parameter estimate vector based on the current covariance matrix, the current motion parameter estimate vector including the noise transformation matrix;
a program to compute a current motion output vector and a current state variable matrix based on the current covariance matrix, the current motion output vector including the transformed random noise vector; and,
a program to compare the current motion output vector to the observation point set; and,
a program to repeatedly use the program to determine each of a plurality of motion parameter vectors a predetermined number of times.

### Claim 17 (depends on 16)

The method as set forth in claim 16, wherein:
the RLS program is configured to obtain m number of points and n number of observations for each of the m number of points;
the program to repeatedly use the program to determine each of a plurality of motion parameter vectors is configured to repeatedly use the program to determine each of a plurality of motion parameter vectors n times;
the program to set a first motion parameter estimate vector is configured to set the first motion parameter estimate vector to [1 0 0 0 0 0 0 0 0 0]′;
the program to compute a current covariance matrix is configured to compute Pt=Pt−1+Pt−1φt[I+φt′Pt−1φt]−1φt−1Pt−2, Pt being the current covariance matrix, Pt−1 being the prior current covariance matrix, Pt−2 being the second prior current covariance matrix, φt being the current state variable matrix, φt−1 being the prior current state variable matrix, and I being the identity matrix; and,
the program to compute a current motion parameter estimate vector is configured to compute {circumflex over (θ)}t={circumflex over (θ)}t−1+[Pt−1φt[I+φt′Pt−1φt]−1][yt−φt{circumflex over (θ)}t−1], {circumflex over (θ)}t being the current motion parameter estimate, {circumflex over (θ)}t−1 being the prior current motion parameter estimate, and yt being the current motion output vector.