# US 12,003,839 B2 · Imaging apparatus using ghost imaging

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,003,839 B2
- Document ID: US12003839B2
- Status: Granted
- Kind: B2
- Application number: 17/506,277
- Filed: 2021-10-20
- Published: 2022-02-03
- Granted: 2024-06-04
- Earliest claimed: 2019-04-22
- Assignees: Koito Manufacturing Co., Ltd.
- Applicants: Koito Manufacturing Co., Ltd.
- Inventors: Masato Gomi; Shintaro Sugimoto; Yuta Haruse; Teruaki Torii
- Web page: https://patentgrep.com/patent/US12003839B2

## CPC

- H04N 23/56: provided with illuminating means
- G01S 7/4815: {using multiple transmitters}
- G01S 7/4816: {of receivers alone}
- G01S 17/89: for mapping or imaging
- G01S 17/931: of land vehicles
- G02B 26/0833: {the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD (G02B26/0825 takes precedence; micromechanical devices in general B81B)}
- G03B 15/05: Combinations of cameras with electronic flash apparatus; Electronic flash units
- G03B 15/06: Special arrangements of screening, diffusing, or reflecting devices, e.g. in studio
- G03B 17/00: Details of cameras or camera bodies; Accessories therefor (lens hoods or caps G03B11/04)
- G03B 30/00: Camera modules comprising integrated lens units and imaging units, specially adapted for being embedded in other devices, e.g. mobile phones or vehicles
- G06T 5/70: Denoising; Smoothing
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06V 10/141: Control of illumination
- G06V 10/143: Sensing or illuminating at different wavelengths
- G06V 10/145: Illumination specially adapted for pattern recognition, e.g. using gratings
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
- H04N 23/80: Camera processing pipelines; Components thereof
- H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio
- F21S 41/675: by moving reflectors
- G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- H04N 23/74: by influencing the scene brightness using illuminating means
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- F21S: NON-PORTABLE LIGHTING DEVICES; SYSTEMS THEREOF; VEHICLE LIGHTING DEVICES SPECIALLY ADAPTED FOR VEHICLE EXTERIORS

## Abstract

An illumination apparatus irradiates first reference light having a first intensity distribution and second reference light having a second intensity distribution that has a complementary relation with the first intensity distribution. A photodetector measures reflected light from an object. A processing device executes a first correlation calculation using a first detection intensity based on the output of the photodetector in a state in which the first reference light is irradiated and the second intensity distribution, and executes a second correlation calculation using a second detection intensity based on the output of the photodetector in a state in which the second reference light is irradiated and the second intensity distribution.

## Claims

### Claim 1

An imaging apparatus comprising:
an illumination apparatus structured to irradiate an object with first reference light having a first intensity distribution Ir(x,y) and second reference light having a second intensity distribution I{circumflex over ( )}r(x,y) that has a complementary (x,y) relation with the first intensity distribution Ir(x,y), wherein r (=1, 2, . . . M) represents r-th irradiation;
a photodetector structured to measure reflected light from the object; and
a processing device structured to perform first correlation calculation represented by expression (1) based on a result of irradiation of the first reference light and second correlation calculation represented by expression (2) based on a result of irradiation of the second reference light, wherein br represents a value of a detection intensity measured when the object is irradiated with the first reference light having the first intensity distribution Ir(x,y), b{circumflex over ( )}r represents a value of a detection intensity measured when the object is irradiated with the second reference light having the second intensity distribution I{circumflex over ( )}r(x,y), and M represents a number of irradiations
G
1
(
x
,
y
)
=
1
M
⁢
∑
r
=
1
M
[
{
b
r
-
〈
b
〉
}
·
I
r
(
x
,
y
)
]
(
1
)
〈
b
〉
=
1
M
⁢
∑
r
=
1
M
b
r
G
2
(
x
,
y
)
=
1
M
⁢
∑
r
=
1
M
[
{
b
r
^
-
〈
b
^
〉
}
·
I
r
^
(
x
,
y
)
]
(
2
)
〈
b
^
〉
=
1
M
⁢
∑
r
=
1
M
b
r
^
.

### Claim 2 (depends on 1)

The imaging apparatus according to claim 1, wherein the processing device is structured to generate a final reconstructed image G(x,y) based on expression (3)
G
⁡
(
x
,
y
)
=
G
1
(
x
,
y
)
+
G
2
(
x
,
y
)
=
1
M
⁢
∑
r
=
1
M
[
{
b
r
-
〈
b
〉
}
·
I
r
(
x
,
y
)
+
{
b
r
^
-
〈
b
^
〉
}
·
I
r
^
(
x
,
y
)
]
.
(
3
)

### Claim 3 (depends on 1)

The imaging apparatus according to claim 1, wherein the processing device is structured to output a first reconstructed image G1(x,y) of the object by the first correlation calculation represented by the expression (1), and to reconstruct a second reconstructed image G2(x,y) of the object by the second correlation calculation represented by the expression (2).

### Claim 4 (depends on 1)

The imaging apparatus according to claim 1, wherein the first reference light and the second reference light have a same wavelength, and are irradiated in a time-sharing manner.

### Claim 5 (depends on 1)

The imaging apparatus according to claim 1, wherein the first reference light has a first wavelength, and the second reference light has a second wavelength that differs from the first wavelength,
and wherein the photodetector comprises:
a first detector that is sensitive to the first wavelength, and not sensitive to the second wavelength, and
a second detector that is sensitive to the second wavelength, and not sensitive to the first wavelength.

### Claim 6 (depends on 5)

The imaging apparatus according to claim 5, wherein the first reference light and the second reference light are irradiated at a same time.

### Claim 7 (depends on 5)

The imaging apparatus according to claim 5, wherein the processing device is structured to correct an output of the first detector and an output of the second detector.

### Claim 8 (depends on 7)

The imaging apparatus according to claim 7, wherein a coefficient required for correction in the processing device is generated based on a first detection intensity obtained with respect to reflected light from a given object in a state in which the first reference light is irradiated with a uniform intensity distribution and a second detection intensity obtained with respect to reflected light from the same object in a state in which the second reference light is irradiated with a uniform intensity distribution.

### Claim 9 (depends on 1)

The imaging apparatus according to claim 1, wherein the illumination apparatus comprises:
a digital micromirror device (DMD) comprising a plurality of micromirrors that correspond to a plurality of pixels, and each structured to be tilted independently around a hinge axis in a first direction and a second direction;
a first light source structured to irradiate a first input beam to the DMD; and
a second light source structured to irradiate a second input beam to the DMD,
wherein a part of the first input beam incident to each micromirror of the plurality of micromirrors tilted in the first direction is projected to the object as a first output beam, and the part of the first input beam incident to each micromirror of the plurality of micromirrors tilted in the second direction is not projected to the object,
and wherein a part of the second input beam incident to each micromirror tilted in the second direction is projected to the object as a second output beam, and the part of the second input beam incident to each micromirror tilted in the first direction is not projected to the object.

### Claim 10 (depends on 1)

An automotive lamp comprising the imaging apparatus according to claim 1.

### Claim 11 (depends on 1)

An automobile comprising the imaging apparatus according to claim 1.

### Claim 12

An imaging method comprising:
irradiating an object with first reference light having a first intensity distribution Ir(x,y);
irradiating the object with second reference light having a second intensity distribution I{circumflex over ( )}r(x,y) that has a complementary (x,y) relation with the first intensity distribution Ir(x,y), wherein r (=1, 2, . . . M) represents r-th irradiation;
measuring reflected light from the object in a state in which the first reference light having the first intensity distribution Ir(x,y) is irradiated, so as to generate a first detection intensity br;
measuring reflected light from the object in a state in which the second reference light having the second intensity distribution I{circumflex over ( )}r(x,y) is irradiated, so as to generate a second detection intensity b{circumflex over ( )}r; and
executing a first correlation calculation represented by expression (1) and a second correlation calculation represented by expression (2)
G
1
(
x
,
y
)
=
1
M
⁢
∑
r
=
1
M
[
{
b
r
-
〈
b
〉
}
·
I
r
(
x
,
y
)
]
(
1
)
〈
b
〉
=
1
M
⁢
∑
r
=
1
M
b
r
G
2
(
x
,
y
)
=
1
M
⁢
∑
r
=
1
M
[
{
b
r
^
-
〈
b
^
〉
}
·
I
r
^
(
x
,
y
)
]
(
2
)
〈
b
^
〉
=
1
M
⁢
∑
r
=
1
M
b
r
^
.

### Claim 13 (depends on 12)

The imaging method according to claim 12, further comprising generating a final reconstructed image G(x,y) represented in expression (3)
G
⁡
(
x
,
y
)
=
G
1
(
x
,
y
)
+
G
2
(
x
,
y
)
=
1
M
⁢
∑
r
=
1
M
[
{
b
r
-
〈
b
〉
}
·
I
r
(
x
,
y
)
+
{
b
r
^
-
〈
b
^
〉
}
·
I
r
^
(
x
,
y
)
]
.
(
3
)