# US 11,029,452 B2 · Vehicle and 3D object information capturing system and method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,029,452 B2
- Document ID: US11029452B2
- Status: Granted
- Kind: B2
- Application number: 16/257,355
- Filed: 2019-01-25
- Published: 2019-07-25
- Granted: 2021-06-08
- Earliest claimed: 2018-01-25
- Assignees: TDK Taiwan Corp.
- Applicants: TDK Taiwan Corp.
- Inventors: Sin-Jhong Song
- Web page: https://patentgrep.com/patent/US11029452B2

## CPC

- G03B 5/02: Lateral adjustment of lens
- F03G 7/0614: {using shape memory elements}
- F03G 7/06143: {Wires}
- G02B 3/04: with continuous faces that are rotationally symmetrical but deviate from a true sphere {, e.g. so called "aspheric" lenses}
- G02B 3/12: Fluid-filled or evacuated lenses
- G02B 3/14: of variable focal length
- G02B 5/005: {Diaphragms (for cameras G03B9/02)}
- G02B 6/0025: {Diffusing sheet or layer; Prismatic sheet or layer}
- G02B 6/0066: {characterised by the light source being coupled to the light guide}
- G02B 7/023: {permitting adjustment}
- G02B 7/08: adapted to co-operate with a remote control mechanism
- G02B 7/09: adapted for automatic focusing or varying magnification
- G02B 7/102: {controlled by a microcomputer (cameras with interchangeable lenses G03B17/14)}
- G02B 7/1805: {for prisms (G02B7/181 takes precedence)}
- G02B 7/182: for mirrors
- G02B 7/1821: {for rotating or oscillating mirrors}
- G02B 7/1828: {using magnetic means}
- G02B 13/0065: {having a beam-folding prism or mirror}
- G02B 13/0075: {having an element with variable optical properties}
- G02B 13/009: {having zoom function}
- G02B 26/004: {based on a displacement or a deformation of a fluid}
- G02B 26/0816: {by means of one or more reflecting elements}
- G02B 26/0883: {the refracting element being a prism}
- G02B 27/0068: {having means for controlling the degree of correction, e.g. using phase modulators, movable elements (controlling the phase of light using moving or deformable elements G02B26/06)}
- G02B 27/0955: {Lenses (lenses per se G02B3/00)}
- G02B 27/0972: {Prisms (prisms per se G02B5/04)}
- G02B 27/0977: {Reflective elements}
- G02B 27/646: {compensating for small deviations, e.g. due to vibration or shake (movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68)}
- G03B 5/00: Adjustment of optical system relative to image or object surface other than for focusing
- G03B 9/04: Single movable plate with two or more apertures of graded size, e.g. sliding plate or pivoting plate
- G03B 9/06: Two or more co-operating pivoted blades, e.g. iris type (shutters functioning as diaphragms by limiting extent of opening movement G03B9/08)
- G03B 9/10: Blade or disc rotating or pivoting about axis normal to its plane
- G03B 9/14: Two separate members moving in opposite directions
- G03B 13/36: Autofocus systems
- G03B 17/17: with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
- G03B 30/00: Camera modules comprising integrated lens units and imaging units, specially adapted for being embedded in other devices, e.g. mobile phones or vehicles
- G06F 1/1626: {with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]}
- G06F 1/1686: {the I/O peripheral being an integrated camera}
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)}
- G06T 15/08: Volume rendering
- G09G 5/003: {Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto (specific for a CRT G09G1/165; for a flat panel G09G3/2092)}
- H02K 41/0356: {moving along a straight path}
- H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
- H04N 23/51: Housings
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- H04N 23/55: Optical parts specially adapted for electronic image sensors; Mounting thereof
- H04N 23/56: provided with illuminating means
- H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
- H04N 23/73: by influencing the exposure time
- G02B 5/003: {Light absorbing elements}
- G02B 13/004: {having four lenses}
- G03B 5/04: Vertical adjustment of lens; Rising fronts
- G03B 2205/0007: Movement of one or more optical elements for control of motion blur
- G03B 2205/0015: by displacing one or more optical elements normal to the optical axis
- G03B 2205/0023: by tilting or inclining one or more optical elements with respect to the optical axis
- G03B 2205/0069: using electromagnetic actuators, e.g. voice coils
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10048: Infrared image
- G09G 2354/00: Aspects of interface with display user
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- F03G: SPRING, WEIGHT, INERTIA OR LIKE MOTORS; MECHANICAL-POWER PRODUCING DEVICES OR MECHANISMS, NOT OTHERWISE PROVIDED FOR OR USING ENERGY SOURCES NOT OTHERWISE PROVIDED FOR (arrangements in connection with power supply in vehicles from force of nature B60K16/00; electric propulsion with power supply in vehicles from force of nature B60L8/00)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H02K: DYNAMO-ELECTRIC MACHINES (dynamo-electric relays H01H53/00; conversion of DC or AC input power into surge output power {H03K3/53})
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A 3D object information capturing system is provided, including a camera module, a distance measuring module, and a processing module. The camera module captures image information of an object, and the distance measuring module captures distance information of the object. The processing module receives the image information and the distance information respectively from the camera module and the distance measuring module, and constructs a 3D model of the object according to the image information and the distance information.

## Claims

### Claim 1

A 3D object information capturing method, comprising:
providing a camera module and capturing a 2D image of an object using the camera module;
analyzing the 2D image and generating first outline information of the object according to the 2D image;
providing a distance-measuring module and capturing 2D distance matrix information of the object's surface using the distance-measuring module;
analyzing the 2D distance matrix information and generating second outline information of the object according to the 2D distance matrix information;
establishing a 3D model of the object according to the first outline information and the second outline information;
providing an infrared sensing module and sensing the object with the infrared sensing module to obtain an infrared image of the object;
analyzing the infrared image and generating third outline information according to the infrared image; and
establishing the 3D model of the object according to the first outline information, the second outline information, and the third outline information.

### Claim 2 (depends on 1)

The 3D object information capturing method as claimed in claim 1, further comprising:
generating the first outline information by performing binarization on the 2D image.

### Claim 3 (depends on 1)

The 3D object information capturing method as claimed in claim 1, further comprising:
generating the second outline information by calculating the differences between adjacent elements of the 2D distance matrix information.

### Claim 4 (depends on 1)

The 3D object information capturing method as claimed in claim 1, further comprising:
transmitting infrared light to the object with the distance-measuring module; and receiving the infrared light that is reflected by the object using the infrared sensing module.

### Claim 5 (depends on 4)

The 3D object information capturing method as claimed in claim 4, further comprising:
providing a light measuring module to measure environmental light; and
transmitting infrared light to the object with the distance-measuring module when the illumination by environmental light is lower than a predetermined value, and receiving the infrared light that is reflected by the object using the infrared sensing module.

### Claim 6

A 3D object information capturing method, comprising:
providing a camera module and capturing a 2D image of an object using the camera module;
analyzing the 2D image and generating first outline information of the object according to the 2D image;
providing a distance-measuring module and capturing 2D distance matrix information of the object's surface using the distance-measuring module;
analyzing the 2D distance matrix information and generating second outline information of the object according to the 2D distance matrix information; and
establishing a 3D model of the object according to the first outline information and the second outline information;
capturing a plurality of 2D images of the object from different locations using the camera module, and generating a plurality of first outline informations of the object according to the 2D images;
capturing a plurality of 2D distance matrix informations of the object from different locations using the distance-measuring module, and generating a plurality of second outline informations of the object according to the 2D distance matrix informations; and
establishing the 3D model of the object according to the first outline information and the second outline information.

### Claim 7

A 3D object information capturing method, comprising:
providing a camera module and capturing a 2D image of an object using the camera module;
analyzing the 2D image and generating first outline information of the object according to the 2D image;
providing a distance-measuring module and capturing 2D distance matrix information of the object's surface using the distance-measuring module;
analyzing the 2D distance matrix information and generating second outline information of the object according to the 2D distance matrix information; and
establishing a 3D model of the object according to the first outline information and the second outline information;
capturing a plurality of 2D images of the object from different angles using the camera module, and generating a plurality of first outline informations of the object according to the 2D images;
capturing a plurality of 2D distance matrix informations of the object from different angles using the distance-measuring module, and generating a plurality of second outline informations of the object according to the 2D distance matrix informations; and
establishing the 3D model of the object according to the first and second outline informations.

### Claim 8 (depends on 1)

The 3D object information capturing method as claimed in claim 1, further comprising:
providing a GPS module and capturing location information of the camera module and the distance measuring module relative to the object; and
establishing the 3D model of the object according to the location information and the first and second outline informations.

### Claim 9 (depends on 1)

The 3D object information capturing method as claimed in claim 1, further comprising:
providing an inertial sensor GPS module and capturing posture information of the camera module and the distance measuring module relative to the object.

### Claim 10 (depends on 1)

The 3D object information capturing method as claimed in claim 1, wherein the 2D image includes color information of the object.

### Claim 11 (depends on 1)

The 3D object information capturing method as claimed in claim 1, further comprising:
providing a plurality of 3D object information capturing systems, wherein each of the 3D object information capturing systems comprises the camera module and the distance measuring module; and
detecting the object using the 3D object information capturing systems from different angles at the same time, to establish the 3D model of the object.

### Claim 12

A 3D object information capturing system, comprising:
a camera module, capturing a plurality of 2D images of an object from different angles;
a distance measuring module, capturing a plurality of 2D distance matrix informations of the object from different angles;
a processing unit, receiving the image information and the distance information respectively from the camera module and the distance measuring module, generating a plurality of first outline informations of the object according to the 2D images, generating second outline information of the object according to the 2D distance matrix information, and constructing a 3D model of the object according to the first and second outline informations.

### Claim 13 (depends on 12)

The 3D object information capturing system as claimed in claim 12, further comprising an infrared sensing module for sensing the object with infrared light.

### Claim 14 (depends on 12)

The 3D object information capturing system as claimed in claim 12, further comprising a light measuring module to measure environmental light.

### Claim 15 (depends on 12)

The 3D object information capturing system as claimed in claim 12, further comprising a GPS module to capture location information of the camera module and the distance measuring module relative to the object.

### Claim 16 (depends on 12)

The 3D object information capturing system as claimed in claim 12, further comprising an inertial sensor to capture posture information of the camera module and the distance measuring module relative to the object.

### Claim 17 (depends on 12)

The 3D object information capturing system as claimed in claim 12, further comprising a temperature sensor for sensing the temperature around the 3D object information capturing system.

### Claim 18 (depends on 12)

The 3D object information capturing system as claimed in claim 12, further comprising a magnetic field sensor for sensing the magnetic field around the 3D object information capturing system.

### Claim 19 (depends on 12)

A vehicle, comprising:
a 3D object information capturing system as claimed in claim 12; and
a computing unit, wherein the 3D object information capturing system transmits the 3D model of the object to the computing unit, and the computing unit establishes a moving path of the vehicle according to the 3D model of the object.