# US 2014/0198184 A1 · Stereo assist with rolling shutters

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0198184 A1
- Document ID: US20140198184A1
- Status: Published application
- Kind: A1
- Application number: 14/156,096
- Filed: 2014-01-15
- Published: 2014-07-17
- Earliest claimed: 2013-01-15
- Assignees: Mobileye Technologies Ltd.
- Inventors: Gideon Stein; Efim Belman
- Web page: https://patentgrep.com/patent/US20140198184A1

## CPC

- H04N 25/531: by controlling rolling shutters in CMOS SSIS
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- H04N 5/04: Synchronising (for television systems using pulse code modulation H04N7/56)
- H04N 13/25: using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- H04N 13/296: Synchronisation thereof; Control thereof
- H04N 23/60: Control of cameras or camera modules
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- B60R 2300/105: using multiple cameras
- B60R 2300/303: using joined images, e.g. multiple camera images
- B60R 2300/8093: for obstacle warning
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR

## Abstract

An imaging system for a vehicle may include a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter. The imaging system may also include a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter. As a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image corresponds with a second overlap portion of the second image. The first image capture device has a first scan rate associated with acquisition of the first series of image scan lines that is different from a second scan rate associated with acquisition of the second series of image scan lines, such that the first image capture device acquires the first overlap portion of the first image over a period of time during which the second overlap portion of the second image is acquired.

## Claims

### Claim 1

An imaging system for a vehicle, the system comprising:
a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter;
a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter;
wherein, as a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image corresponds with a second overlap portion of the second image; and
wherein the first image capture device has a first scan rate associated with acquisition of the first series of image scan lines that is different from a second scan rate associated with acquisition of the second series of image scan lines, such that the first image capture device acquires the first overlap portion of the first image over a period of time during which the second overlap portion of the second image is acquired, wherein the period of time is associated with a ratio between the first scan rate and the second scan rate.

### Claim 2 (depends on 1)

The system of claim 1, wherein the second scan rate is greater than the first scan rate.

### Claim 3 (depends on 1)

The system of claim 1, further including an image processing device configured to sample only a subset of scan lines from the second image capture device within the second overlap portion of the second image.

### Claim 4 (depends on 1)

The system of claim 1, wherein:
the first image capture device has a first horizontal blanking period associated with acquisition of the first series of image scan lines;
the second image capture device has a second horizontal blanking period associated with acquisition of the second series of image scan lines;
the second horizontal blanking period is less than the first horizontal blanking period.

### Claim 5 (depends on 1)

The system of claim 1, wherein:
the first image capture device has a first vertical blanking period associated with acquisition of the first series of image scan lines;
the second image capture device has a second vertical blanking period associated with acquisition of the second series of image scan lines;
the first vertical blanking period is less than the second vertical blanking period.

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the first image capture device has a first total frame time associated with acquisition of the first series of image scan lines;
the second image capture device has a second total frame time associated with acquisition of the second series of image scan lines; and wherein
the first total frame time is equal to the second total frame time.

### Claim 7 (depends on 1)

The system of claim 1, wherein the second field of view is narrower than the first field of view.

### Claim 8 (depends on 1)

The system of claim 1, wherein at least a center of the second field of view substantially corresponds to at least a center the first field of view.

### Claim 9 (depends on 1)

The system of claim 1, wherein the ratio of the first field of view to the second field of view ranges between 1.5 to 2.0.

### Claim 10 (depends on 1)

The system of claim 1, wherein the ratio of the first field of view to the second field of view ranges between 1.25 to 2.25.

### Claim 11 (depends on 1)

The system of claim 1, wherein the system further comprises:
at least one processing device configured to:
perform a periodic check to determine whether the first image capture device and the second image capture device are synchronized.

### Claim 12 (depends on 11)

The system of claim 11, wherein the at least one processing device is further configured to perform resynchronization of the first image capture device and the second image capture device if the periodic check indicates that the first image capture device and the second image capture device have fallen out of synch.

### Claim 13 (depends on 1)

The system of claim 1, wherein:
the first image capture device has a first pixel delay associated with acquisition of the first series of image scan lines;
the second image capture device has a second pixel delay associated with acquisition of the second series of image scan lines;
the first pixel delay is more than the second pixel delay.

### Claim 14 (depends on 1)

The system of claim 1, wherein the second overlap portion of the second image corresponds to an entire area of overlap between the first field of view and the second field of view.

### Claim 15 (depends on 1)

The system of claim 1, wherein at least a center of the second field of view falls within a lower half of the first field of view.

### Claim 16

A vehicle, comprising:
a body;
an imaging system, the system comprising:
a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter;
a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter;
wherein, as a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image corresponds with a second overlap portion of the second image; and
wherein the first image capture device has a first scan rate associated with acquisition of the first series of image scan lines that is different from a second scan rate associated with acquisition of the second series of image scan lines, such that the first image capture device acquires the first overlap portion of the first image over a period of time during which the second overlap portion of the second image is acquired, wherein the period of time is associated with a ratio between the first scan rate and the second scan rate.

### Claim 17

An imaging system for a vehicle, the system comprising:
a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter;
a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter;
wherein, as a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image will correspond to a second overlap portion of the second image;
at least one processing device configured to:
receive the first image from the first image capture device;
receive the second image from the second image capture device;
correlate at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image, wherein correlation of the at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image includes:
selection of a first scan line associated with acquisition of the first image to be synchronized with a second scan line associated with acquisition of the second image; and
synchronizing of the first scan line with the second scan line;
the at least one processing device further configured to:
select a third scan line associated with acquisition of a third image acquired by the first image capture device to be synchronized with a fourth scan line associated with acquisition of a fourth image acquired by the second image capture device, in place of synchronizing the first scan line with the second scan line, wherein the third scan line is different from the first scan line and the third scan line is different from the fourth scan line; and
synchronize the third scan line with the fourth scan line.

### Claim 18 (depends on 17)

The system of claim 17, wherein the second field of view fully overlaps with the first field of view.

### Claim 19 (depends on 17)

The system of claim 17, wherein the second field of view is narrower than the first field of view.

### Claim 20 (depends on 17)

The system of claim 17, wherein the first image capture device and the second image capture device have differing focal lengths.

### Claim 21

(canceled)

### Claim 22 (depends on 17)

The system of claim 17, wherein correlation of the at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image further includes:
performance of a dense depth map computation in a horizontal strip within the first overlap portion of the first image and within a corresponding region of the second overlap portion of the second image.

### Claim 23

(canceled)

### Claim 24 (depends on 17)

The system of claim 17, wherein correlation of the at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image further includes:
computation of dense optical flow between the first overlap portion of the first image and the second overlap portion of the second image; and
use of the dense optical flow to warp at least one of the first or second images.

### Claim 25

An imaging system for a vehicle, the system comprising:
a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter;
a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter;
wherein, as a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image will correspond to a second overlap portion of the second image;
at least one processing device configured to:
receive the first image from the first image capture device;
receive the second image from the second image capture device; and
correlate at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image, wherein correlation of the at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image includes prewarping at least one of the first image or the second image according to a speed dependent function, the prewarping being achieved by:
computing a homography matrix associated with motion of the vehicle; and
computing a homography associated with skew of lines of at least one of the first image or the second image.
