# US 2021/0233209 A1 · Image processing device, image processing system, imaging device, image processing method, and recording medium storing program code

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0233209 A1
- Document ID: US20210233209A1
- Status: Published application
- Kind: A1
- Application number: 17/158,113
- Filed: 2021-01-26
- Published: 2021-07-29
- Earliest claimed: 2020-01-29
- Assignees: Ricoh Company, Ltd.
- Inventors: Kazuhiro YOSHIDA; Takamichi KATOH; Takashi TADA; Hiroyuki KAWAMOTO; Makoto USAMI; Kenichi ONO
- Web page: https://patentgrep.com/patent/US20210233209A1

## CPC

- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- G06T 3/12: Panospheric to cylindrical image transformations
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/32: using correlation-based methods
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06T 2200/32: involving image mosaicing
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing device, an image processing system, an imaging device, an image processing method, and a recording medium storing program code. The image processing device includes a receiver configured to receive a first image and a second image, a memory configured to store a joining position at each one of a plurality of portions between the first image and the second image received by the receiver, and circuitry configured to evaluate, for each of the plurality of portions, validity of the joining position in a past stored in the memory, and generate a composite image based on the first image and the second image received by the receiver and the joining position in the past stored in the memory on a site where the validity meets a criterion. The method includes reading a past joining position at each one of a plurality of portions between a pair of images.

## Claims

### Claim 1

An image processing device comprising:
a receiver configured to receive a first image and a second image;
a memory configured to store a joining position at each one of a plurality of portions between the first image and the second image received by the receiver; and
circuitry configured to
evaluate, for each of the plurality of portions, validity of the joining position in a past stored in the memory, and
generate a composite image based on the first image and the second image received by the receiver and the joining position in the past stored in the memory on a site where the validity meets a criterion.

### Claim 2 (depends on 1)

The image processing device according to claim 1,
wherein the joining position stored in the memory includes a previous joining position for each of the plurality of portions,
wherein the validity indicates a relevance ratio when the previous joining position is applied to the first image and the second image, and
wherein, when the validity meets the criterion, the previous joining position is used as the joining position in the past for generating the composite image.

### Claim 3 (depends on 1)

The image processing device according to claim 1, wherein the joining position stored in the memory includes, for each of the plurality of portions, a previous joining position and a basic joining position at a long distance detected within a prescribed period of time.

### Claim 4 (depends on 3)

The image processing device according to claim 3,
wherein the circuitry is configured to use a relevance ratio when the previous joining position is applied to the first image and the second image as a first evaluation value indicating the validity, and use a relevance ratio when the basic joining position is applied to the first image and the second image as a second evaluation value indicating the validity, and
wherein the criterion includes a first criterion and a second criterion, and when the first evaluation value does not satisfy the first criterion and the second evaluation value satisfies the second criterion, the basic joining position is used as the joining position in the past for generating the composite image.

### Claim 5 (depends on 3)

The image processing device according to claim 3, wherein the circuitry is configured to update, for each of the plurality of portions, the basic joining position stored in the memory with the joining position used for generating the composite image, when the joining position used for generating the composite image is at a longer distance than the basic joining position stored in the memory.

### Claim 6 (depends on 3)

The image processing device according to claim 3, wherein the circuitry is configured to update, for each of the plurality of portions, the basic joining position stored in the memory with the joining position used for generating the composite image, when the joining position used for generating the composite image is at a shorter distance than the basic joining position stored in the memory and a period of duration at the joining position approximately same as the joining position used for generating the composite image satisfies a prescribed criterion.

### Claim 7 (depends on 3)

The image processing device according to claim 3, wherein the circuitry is configured to set a search area on a site where the validity does not meet the criterion based on the basic joining position, and detect the joining position between the first image and the second image.

### Claim 8 (depends on 1)

The image processing device according to claim 1,
wherein the circuitry is configured to obtain distance-to-object information; and
wherein the circuitry is configured to set a search area on a site where the validity does not meet the criterion based on the distance-to-object information and detect the joining position between the first image and the second image.

### Claim 9 (depends on 1)

The image processing device according to claim 1,
wherein the circuitry is configured to generate the composite image using a conversion table generated for each of the plurality of portions based on the joining position,
wherein the circuitry is configured to update the conversion table when the joining position is changed at one or more portions of the plurality of portions and a number of the plurality of portions at which the joining position is changed is equal to or greater than a threshold, and
wherein the circuitry is configured not to update the conversion table when the validity meets the criterion at the plurality of portions whose number is equal to or greater than the threshold.

### Claim 10 (depends on 1)

The image processing device according to claim 1,
wherein each one of the first image, the second image, and the composite image makes up an image of one frame in video data, and
wherein the composite image makes up a spherical image, a panoramic image of 360 degrees, or a part of the spherical image or the panoramic image of 360 degrees.

### Claim 11 (depends on 1)

An image processing system comprising
the image processing device according to claim 1; and
circuitry configured to distribute the composite image.

### Claim 12 (depends on 1)

An imaging device comprising:
a plurality of optical systems;
an imaging unit configured to capture a plurality of partial-view images through each one of the plurality of optical systems; and
the image processing device according to claim 1,
wherein the first image and the second image are based on a first partial-view image and a second partial-view image each of which is captured through different one of the plurality of optical systems.

### Claim 13

A method of processing an image, the method comprising:
reading a past joining position at each one of a plurality of portions between a pair of images;
receiving a first image and a second image;
evaluating validity of the past joining position read in the reading for each of the plurality of portions; and
generating a composite image based on the first image and the second image and the past joining position read for a site where the validity meets a criterion.

### Claim 14

A computer-readable non-transitory recording medium storing a program for causing a computer to execute a method, the method comprising:
receiving a first image and a second image;
storing, in a memory, a joining position at each one of a plurality of portions between the first image and the second image;
evaluating, for each of the plurality of portions, validity of the joining position in a past stored in the memory; and
generating a composite image based on the first image and the second image and the joining position in the past stored in the memory on a site where the validity meets a criterion.
