# US 2019/0026878 A1 · Image-stitching for dimensioning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0026878 A1
- Document ID: US20190026878A1
- Status: Published application
- Kind: A1
- Application number: 16/140,953
- Filed: 2018-09-25
- Published: 2019-01-24
- Earliest claimed: 2014-10-10
- Assignees: Hand Held Products, Inc.
- Applicants: Hand Held Products, Inc.
- Inventors: Brian L. Jovanovski; Jingquan Li
- Web page: https://patentgrep.com/patent/US20190026878A1

## CPC

- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G01B 11/2522: {the position of the object changing and being recorded}
- G05B 19/402: characterised by control arrangements for positioning, e.g. centring a tool relative to a hole in the workpiece, additional detection means to correct position (G05B19/19 takes precedence)
- H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00)
- H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects
- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- B65G 1/1371: {with data records}
- B65G 1/1375: {the orders being assembled on a commissioning stacker-crane or truck}
- G05B 2219/31319: Use data groups as inventory control value, adapt inventory need to new data
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D)

## Abstract

Dimensioning systems may automate or assist with determining the physical dimensions of an object without the need for a manual measurement. A dimensioning system may project a light pattern onto the object, capture an image of the reflected pattern, and observe changes in the imaged pattern to obtain a range image, which contains 3D information corresponding to the object. Then, using the range image, the dimensioning system may calculate the dimensions of the object. In some cases, a single range image does not contain 3D data sufficient for dimensioning the object. To mitigate or solve this problem, the present invention embraces capturing a plurality of range images from different perspectives, and then combining the range images (e.g., using image-stitching) to form a composite range-image, which can be used to determine the object's dimensions.

## Claims

### Claim 1

A method, comprising:
projecting, using a pattern projector, a light pattern into a field-of-view of a range camera of a dimensioning system, the field-of-view comprising a portion of an object;
capturing, using the range camera of the dimensioning system, a range image of the field-of-view, wherein each pixel of the range image represents a distance from the range camera to a respective point in the range camera's field-of-view;
moving the dimensioning system and/or the object so that there is relative movement between the dimensioning system and the object, and the range camera's field-of-view contains a different portion of the object;
repeating the capturing and the moving until a plurality of range images have been captured, wherein in each range image of the plurality of range images, each pixel of the range image represents a distance from the range camera to a respective point in the range camera's field-of-view;
gathering information as the spatial relationship between the range camera and the object is changed, and the information is selected from the group consisting of tracking information and mapping information;
combining the plurality of range images to create a composite range-image; and
dimensioning the object using the composite range-image.

### Claim 2 (depends on 1)

The method of claim 1, comprising combining the plurality of range images to create a composite range-image using the gathered information.

### Claim 3 (depends on 1)

The method of claim 1, wherein range images in the plurality of range images have partially overlapping fields of view.

### Claim 4 (depends on 1)

The method of claim 1, wherein the field-of-view of the range camera and a field-of-view of the pattern projector overlap.

### Claim 5 (depends on 4)

The method of claim 4, wherein the field-of-view of the range camera and the field-of-view of the pattern projector do not have the same shape and/or size.

### Claim 6 (depends on 1)

The method of claim 1, comprising generating messages to help a user change the spatial relationship between the range camera and the object using the gathered information.

### Claim 7 (depends on 1)

The method of claim 1, wherein the messages comprise instructions to take action, and the instructions to take action are selected from the group consisting of instructions to (i) move the dimensioning system or the object in a particular direction, (ii) move the dimensioning system or the object at a particular speed, and (iii) cease moving the dimensioning system or the object.

### Claim 8 (depends on 1)

The method of claim 1, wherein the light pattern comprises points of light arranged in a pseudo-random pattern.

### Claim 9 (depends on 1)

The method of claim 1, wherein the light pattern comprises points of light having different sizes.

### Claim 10 (depends on 1)

The method of claim 1, comprising:
determining a range value for each point of light in the projected light pattern to generate a point-cloud image; and
interpolating range values between points of light in the point-cloud image to create a range image.

### Claim 11 (depends on 1)

The method of claim 1, wherein dimensioning the object using the composite range-image comprises creating a minimum bounding box that surrounds the object, and the method comprises returning the dimensions of the minimum bounding box as dimensions of the object.

### Claim 12 (depends on 1)

The method of claim 1, comprising moving the dimensioning system and/or the object so that there is a constant range between the dimensioning system and the object.

### Claim 13 (depends on 1)

The method of claim 1, comprising moving the dimensioning system and/or the object along a certain path having a particular starting point and ending point.

### Claim 14 (depends on 13)

The method of claim 13, comprising determining the certain path using an expected object size, an expected object shape, or an expected object size and shape.

### Claim 15

A dimensioning system, comprising:
a pattern projector configured to project a light pattern onto an object;
a range camera having a field of view and configured to (i) capture an image of a reflected light-pattern in the field of view, (ii) generate 3D data from the reflected light-pattern, and (iii) create a range image using the 3D data, wherein the pattern projector and the range camera are spatially offset;
at least one device configured to move the dimensioning system and/or the object so that there is relative movement between the dimensioning system and the object, and the range camera's field-of-view contains a different portion of the object; and
a processor communicatively coupled to the pattern projector and the range camera, wherein the processor is configured by software to:
trigger the range camera to capture a plurality of range images, wherein in each range image of the plurality of range images, each pixel of the range image represents a distance from the range camera to a respective point in the range camera's field-of-view;
gather information as the spatial relationship between the range camera and the object is changed, and the information is selected from the group consisting of tracking information and mapping information;
combine the plurality of range images to create a composite range-image; and
dimension the object using the composite range-image.

### Claim 16 (depends on 15)

The dimensioning system of claim 15, wherein the processor is configured by software to adjust the spatial offset to change the resolution at which range differences may be detected.

### Claim 17 (depends on 15)

The dimensioning system of claim 15, wherein the processor is configured by software to combine the plurality of range images to create a composite range-image by image-stitching the plurality of range images, wherein the image-stitching comprises simultaneous localization and mapping (SLAM).

### Claim 18 (depends on 17)

The dimensioning system of claim 17, wherein the simultaneous localization and mapping comprises:
detecting features in a captured range image; and
comparing the detected features to features detected in previously captured range image to update a position of the dimensioning system.

### Claim 19 (depends on 18)

The dimensioning system of claim 18, wherein the processor is configured by software to use the updated position of the dimensioning system to combine the plurality of range images.

### Claim 20 (depends on 15)

The dimensioning system of claim 15, wherein the dimensioning system is handheld.
