US 7,289,138 B2 · Intersection detection in panoramic video

Abstract

Provides a system for detecting an intersection between more than one panoramic video sequence and detecting the orientation of the sequences forming the intersection. Video images and corresponding location data are received. If required, the images and location data is processed to ensure the images contain location data. An intersection between two paths is then derived from the video images by deriving a rough intersection between two images, determining a neighborhood for the two images, and dividing each image in the neighborhood into strips. An identifying value is derived from each strip to create a row of strip values which are then converted to the frequency domain. A distance measure is taken between strips in the frequency domain, and the intersection is determined from the images having the smallest distance measure between them. The orientation between the two paths may also be determined in the frequency domain by using the phases of signals representing the images in the Fourier domain or performing a circular cross correlation of two vectors representing the images.

Claims36

Dependent claims

A method for detecting an intersection as displayed in two intersecting panoramic video frame sequences, the method comprising:

receiving data representing two intersecting panoramic video frame sequences of an identical location captured at different times and from different directions, the sequences each including a plurality of frames depicting an image of the intersection between the video frame sequences;
detecting an intersection frame from each of the panoramic video frame sequences using only the data representing the two panoramic video frame sequences, the intersection comprised of the intersection frame from each panoramic video frame sequence;
wherein detecting the intersection frame from each of the panoramic video frame sequences includes detecting frame images that are most similar from each of the panoramic video frame sequences;
and
displaying one of the two intersecting panoramic video frame sequences up to and including the intersection frame, then displaying the other of the two intersecting panoramic video frame sequences starting from the intersection frame.

2 more elements

A method for detecting an intersection as displayed in two intersecting panoramic video frame sequences, the method comprising:

receiving data representing two intersecting panoramic video frame sequences, the sequences each including a plurality of frames depicting an image of the intersection between the video frame sequences;
detecting an intersection frame from each of the panoramic video frame sequences using only the data representing the two panoramic video frame sequences, the intersection comprised of the intersection frame from each panoramic video frame sequence;
wherein detecting the intersection frame from each of the panoramic video frame sequences includes detecting frame images that are most similar from each of the panoramic video frame sequences, and wherein detecting frame images that are most similar includes:
segmenting each frame into at least one strip, wherein the frames are divided into rows and at least one strip is associated with each row;
determining an identifying value for each strip;
and
determining a distance measurement between strips in corresponding rows in each frame to determine which frames from the two sequences are closest to each other, wherein taking the distance measure includes processing the identifying value of each strip;
wherein taking the distance measurement includes:
receiving Fourier spectra data associated with the strips in a corresponding row of each frame;
and
comparing the Fourier spectra data to derive a distance measurement between the frames of each panoramic video frame sequence;
wherein the distance measurement is one of a Euclidean, cosine, and normalized distance;
wherein the distance measurement is weighted;
wherein the weighted distance measurement includes weights to reduce a high and low band of the Fourier spectra;
and
displaying one of the two intersecting panoramic video frame sequences up to and including the intersection frame, then displaying the other of the two intersecting panoramic video frame sequences starting from the intersection frame.

13 more elements

A method for detecting the orientation between two intersecting panoramic video frame sequences at an intersection as displayed in the two video sequences, the method comprising:

receiving data representing two intersecting panoramic video frame sequences, the sequences each including a plurality of frames depicting an image of the intersection between the panoramic video frame sequences;
detecting an intersection frame from each panoramic video sequence, the intersection frames containing frame images that are most similar between the two intersecting panoramic video frame sequences, the intersection frames detected using only the data representing the two panoramic video frame sequences;
detecting a relative orientation between the intersection frames of the panoramic video sequence using only the data representing the two panoramic video frame sequences;
wherein detecting a relative orientation includes:
segmenting a frame into at least one strip, wherein the frames are divided into rows and at least one strip is associated with each row;
determining an identifying value for each strip;
and
processing the identifying values to determine the relative orientation between images;
wherein processing the identifying values includes:
receiving Fourier spectra data associated with the identifying values of the strips;
and
determining the relative orientation by deriving the maximum of the cross correlation between the identifying data of the first and second panoramic video images, the cross correlation values derived from the Fourier spectra;
wherein deriving the maximum cross correlation includes:
processing the Fourier spectra using complex conjugation to derive a set of data;
and
taking the inverse DFT of the set of data;
and
displaying one of the two intersecting panoramic video frame sequences up to and including the intersection frame, then displaying the other of the two intersecting panoramic video frame sequences starting from the intersection frame.

15 more elements

A method for detecting the orientation between two intersecting panoramic video frame sequences at an intersection as displayed in the two video sequences, the method comprising:

receiving data representing two intersecting panoramic video frame sequences, the sequences each including a plurality of frames depicting an image of the intersection between the panoramic video frame sequences;
detecting an intersection frame from each panoramic video sequence, the intersection frames containing frame images that are most similar between the two intersecting panoramic video frame sequences, the intersection frames detected using only the data representing the two panoramic video frame sequences;
detecting a relative orientation between the intersection frames of the panoramic video sequence using only the data representing the two panoramic video frame sequences;
wherein detecting a relative orientation includes:
segmenting a frame into at least one strip, wherein the frames are divided into rows and at least one strip is associated with each row;
determining an identifying value for each strip;
and
processing the identifying values to determine the relative orientation between images;
wherein processing the identifying values includes:
receiving Fourier spectra data associated with the identifying values of the strips;
and
determining the relative orientation by deriving the maximum of the cross correlation between the identifying data of the first and second panoramic video images, the cross correlation values derived from the Fourier spectra;
wherein deriving the maximum cross correlation includes:
processing the Fourier spectra using complex conjugation to derive a set of data;
and
taking the inverse DET of the set of data;
wherein the distance measurement is weighted;
and
displaying one of the two intersecting panoramic video frame sequences up to and including the intersection frame, then displaying the other of the two intersecting panoramic video frame sequences starting from the intersection frame.

16 more elements

A method for detecting the orientation between two intersecting panoramic video frame sequences at an intersection as displayed in the two video sequences, the method comprising:

receiving data representing two intersecting panoramic video frame sequences, the sequences each including a plurality of frames depicting an image of the intersection between the panoramic video frame sequences;
detecting an intersection frame from each panoramic video sequence, the intersection frames containing frame images that are most similar between the two intersecting panoramic video frame sequences, the intersection frames detected using only the data representing the two panoramic video frame sequences;
detecting a relative orientation between the intersection frames of the panoramic video sequence using only the data representing the two panoramic video frame sequences;
wherein detecting a relative orientation includes:
segmenting a frame into at least one strip, wherein the frames are divided into rows and at least one strip is associated with each row;
determining an identifying value for each strip;
and
processing the identifying values to determine the relative orientation between images;
wherein processing the identifying values includes:
receiving Fourier spectra data associated with the identifying values of the strips;
and
determining the relative orientation by deriving the maximum of the cross correlation between the identifying data of the first and second panoramic video images, the cross correlation values derived from the Fourier spectra;
wherein deriving the maximum cross correlation includes:
processing the Fourier spectra using complex conjugation to derive a set of data;
and
taking the inverse DFT of the set of data;
wherein the distance measurement is weighted;
wherein the weighted distance measurement includes weights to reduce a high and low band of the Fourier spectra;
and
displaying one of the two intersecting panoramic video frame sequences up to and including the intersection frame, then displaying the other of the two intersecting panoramic video frame sequences starting from the intersection frame.

17 more elements

A method for determining an intersection and orientation between two intersecting panoramic video frame sequences, the method comprising:

receiving a first set of data representing two intersecting panoramic video frame sequences, the first set of data including pixel data and location data, the location data associated with at least one frame from each sequence, the sequences each including a plurality of frames depicting an image of the intersection between the video frame sequences;
determining location data for any frame not already associated with location data;
determining a rough estimate of the intersection between the two panoramic video frame sequences, the rough intersection determined by comparing the location data for each frame;
determining a neighborhood having a set of neighborhood frames, the neighborhood including the rough intersection, the neighborhood frames including frames from each of the panoramic video sequences within a parameter of the rough estimate of the intersection;
segmenting the neighborhood frames into at least one strip, wherein the frames are divided into rows and at least one strip is associated with each row, each strip comprised of a plurality of pixels;
determining an identifying value for each strip;
receiving Fourier spectra data associated with the strips in a corresponding row of each frame;
comparing the Fourier spectra data to derive a distance measurement between the frames of the panoramic video sequences;
determining the relative orientation between the frames having the shortest distance measurement using the Fourier spectra data;
and
displaying one of the two intersecting panoramic video frame sequences up to and including the intersection frame, then displaying the other of the two intersecting panoramic video frame sequences starting from the intersection frame.

8 more elements

A computer readable medium stored thereon a computer program, when executed by a computer, the computer program to determine an intersection and orientation between two intersecting panoramic video frame sequences, comprising:

computer code for receiving a first set of data representing two intersecting panoramic video frame sequences, the first set of data including pixel data and location data, the location data associated with at least one frame from each sequence, the sequences each including a plurality of frames depicting an image of the intersection between the video frame sequences;
computer code for determining location data for any frame not already associated with location data;
computer code for determining a rough estimate of the intersection between the two panoramic video frame sequences, the rough intersection determined by comparing the location data for each frame;
computer code for determining a neighborhood having a set of neighborhood frames, the neighborhood including the rough intersection, the neighborhood frames including frames from each of the panoramic video sequences within a parameter of the rough estimate of the intersection;
computer code for segmenting the neighborhood frames into at least one strip, wherein the frames are divided into rows and at least one strip is associated with each row, each strip comprised of a plurality of pixels;
computer code for determining an identifying value for each strip;
computer code for receiving Fourier spectra data associated with the strips in a corresponding row of each frame;
computer code for comparing the Fourier spectra data associated with the corresponding rows of the frames to derive a distance measurement between the frames of the panoramic video sequences;
and
computer code for determining the relative orientation between the frames having the shortest distance measurement using the Fourier spectra data.

7 more elements

A computer readable medium stored thereon a computer program, when executed by a computer, the computer program to detect the orientation between two intersecting panoramic video frame sequences at an intersection as displayed in the two video sequences, comprising:

computer code for receiving data representing two intersecting panoramic video frame sequences, the sequences each including a plurality of frames depicting an image of the intersection between the panoramic video frame sequences;
computer code for detecting an intersection frame from each panoramic video sequence, the intersection frames containing frame images that are most similar between the intersecting panoramic video frame sequences, the intersection frames detected using only the data representing the two panoramic video frame sequences;
computer code for segmenting a frame into at least one strip, wherein the frames are divided into rows and at least one strip is associated with each row;
computer code for determining an identifying value for each strip;
computer code for receiving Fourier spectra data associated with the identifying values of the strips;
and
computer code for determining the relative orientation between the intersection frames using the Fourier spectra data.

4 more elements

An apparatus for determining an intersection and orientation between two intersecting panoramic video frame sequences, comprising:

a processor;
and
a processor readable storage medium coupled to the processor;
said processor readable storage medium containing program code for programming the apparatus to perform a method for determining an intersection and orientation between two intersecting panoramic video frame sequences, the method comprising the steps of:
receiving a first set of data representing two intersecting panoramic video frame sequences, the first set of data including pixel data and location data, the location data associated with at least one frame from each sequence, the sequences each including a plurality of frames depicting an image of the intersection between the video frame sequences;
determining location data for any frame not already associated with location data;
determining a rough estimate of the intersection between the two panoramic video frame sequences, the rough intersection determined by comparing the location data for each frame;
determining a neighborhood having a set of neighborhood frames, the neighborhood including the rough intersection, the neighborhood frames including frames from each of the panoramic video sequences within a parameter of the rough estimate of the intersection;
segmenting the neighborhood frames into at least one strip, wherein the frames are divided into rows and at least one strip is associated with each row, each strip comprised of a plurality of pixels;
determining an identifying value for each strip;
receiving Fourier spectra data associated with the strips in a corresponding row of each frame;
comparing the Fourier spectra data associated with the corresponding rows of the frames to derive a distance measurement between each frame;
and
determining the relative orientation between the panoramic video frames having the shortest distance measurement between them, the relative orientation determined using the Fourier spectra data.

11 more elements

Dependencies are inferred from the claim text.

Intersection detection in panoramic video

Life of this patent

Intersection detection in panoramic video

Life of this patent

Abstract

Claims36