# US 2017/0257596 A1 · Image processing device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0257596 A1
- Document ID: US20170257596A1
- Status: Published application
- Kind: A1
- Application number: 15/129,301
- Filed: 2015-01-20
- Published: 2017-09-07
- Earliest claimed: 2014-03-27
- Assignees: Noritsu Precision Co., Ltd.
- Applicants: Noritsu Precision Co., Ltd.
- Inventors: Yukiko Murata; Tomoo Nakano
- Web page: https://patentgrep.com/patent/US20170257596A1

## CPC

- H04N 5/915: for field- or frame-skip recording or reproducing
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06V 20/47: {Detecting features for summarising video content}
- G06V 20/48: {Matching video sequences}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)}
- G11B 27/3081: {used signal is a video-frame or a video-field (P.I.P)}
- H04N 5/93: Regeneration of the television signal or of selected parts thereof
- H04N 7/183: {for receiving images from a single remote source}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)

## Abstract

Provided is an image processing device for sorting frames included in one moving image into different channels, the frames included in the moving image being a mixture of frames belonging to the different channels. The image processing device includes an automatic sorting unit. The automatic sorting unit calculates a similarity degree of the plurality of frames included in the moving image by performing image processing on the frames, and sorts the plurality of frames into the plurality of channels based on the similarity degree.

## Claims

### Claim 1

An image processing device for sorting frames included in one moving image into different channels, the frames included in the moving image being a mixture of frames belonging to the different channels, comprising
an automatic seating unit configured to calculate a similarity degree of the plurality of frames included in the moving image by performing image processing on the frames, and sort the plurality of frames into the plurality of channels based on the similarity degree.

### Claim 2 (depends on 1)

The image processing device according to claim 1, wherein the automatic sorting unit includes:
a setting unit configured to set a specific frame included in The moving image or a frame that is obtained by combining two or more specific frames included in the moving image as a reference frame, and set another specific frame included in the moving image as a comparative frame;
a calculation unit configured to execute calculation processing of dividing the comparative frame and the reference frame into a plurality of sub-regions, calculating a local similarity degree between the comparative frame and the reference frame for each sub-region, and calculating a total similarity degree, which is a frame-overall similarity degree, between the comparative frame and the reference frame only using sub-regions having a high local similarity degree; and
a determination unit configured to execute determination processing of determining, based on the total similarity degree, whether or not the comparative frame belongs to the same channel as that of the reference frame or the frames combined into the reference frame.

### Claim 3 (depends on 1)

The image processing device according to claim 1, wherein the automatic sorting unit includes:
a setting unit configured to set a specific frame included in The moving image or a frame that is obtained by combining two or more specific frames included in the moving image as a reference frame, and set another specific frame included in the moving image as a comparative frame;
a calculation unit configured to perform calculation processing of detecting feature points from the comparative frame, setting a vicinity region for each feature point, and calculating a total similarity degree, which is a frame-overall similarity degree, between the comparative frame and the reference frame only using the vicinity regions; and
a determination unit configured to execute determination processing of determining, based on the total similarity degree, whether or not the comparative frame belongs to the same channel as that of the reference frame or the frames combined into the reference frame.

### Claim 4 (depends on 3)

The image processing device according to claim 3,
wherein the calculation unit is configured to divide the comparative frame into a plurality of sub-regions, and detect the feature points for each sub-region from the comparative frame.

### Claim 5 (depends on 3)

The image processing device according to claim 3,
wherein the calculation unit is configured to calculate, for each vicinity region, a local similarity degree between the comparative frame and the reference frame, and calculate the total similarity degree only using vicinity regions having a high local similarity degree.

### Claim 6 (depends on 2)

The image processing device according to claim 2,
wherein the setting unit is configured to accept, from a user, setting of an area in a frame screen, set an image in the area of the specific frame included in the moving image or a frame that is obtained by combining images in the area of the two or more specific frames included in the moving image, as the reference frame, and set an image in the area of the other specific frame included in the moving image as the comparative frame.

### Claim 7 (depends on 2)

The image processing device according to claim 2,
wherein the setting unit is capable of setting a plurality of reference frames, and
the calculation unit is configured to execute, if the plurality of reference frames are set, the calculation processing on each of the reference frames, and
the determination unit is configured to determine, in the determination processing, that the comparative frame and the reference frame or the frames combined into the reference frame that have the highest total similarity degree belong to the same channel.

### Claim 8 (depends on 7)

The image processing device according to claim 7,
wherein the setting unit is configured to sequentially select frames from the moving image, and set the first selected frame as the reference frame and a frame selected thereafter as the comparative frame,
each time the setting unit is configured to newly set a comparative frame, the calculation unit is configured to execute the calculation processing and the determination unit is configured to execute the determination processing, and
if it is determined, in the determination processing, that the comparative frame does not belong to the same channel as any of the reference frames or any of the frames combined into the reference frames, the setting unit again is configured to set that comparative frame as the reference frame.

### Claim 9 (depends on 7)

The image processing device according to claim 7,
wherein the setting unit is configured to sequentially select frames from the moving image, and set the first selected frame as the reference frame and a frame selected thereafter as the comparative frame,
each time the setting unit newly is configured to set a comparative frame, the calculation unit is configured to execute the calculation processing and the determination unit is configured to execute the determination processing, and
if it is determined, in the determination processing, that the comparative frame belongs to the same channel as any of the reference frames or any of the frames combined into the reference frames, the setting unit is configured to combine that comparative frame with the reference frame.

### Claim 10 (depends on 8)

The image processing device according to claim 8,
wherein, if the plurality of reference frames are set, after the final frame has been selected from the moving image and the calculation processing and the determination processing have ended,
the calculation unit is configured to calculate a similarity degree between the plurality of reference frames, and
the determination unit is configured to combine channels that correspond to reference frames that have a high similarity degree between the plurality of reference frames into one channel.

### Claim 11 (depends on 8)

The image processing device according to claim 8,
wherein, alter the final frame has been selected from the moving image and the calculation processing and the determination processing have ended, the determination unit is configured to delete the channel to which only a predetermined number or less of frames belong.

### Claim 12 (depends on 1)

The image processing device according to claim 1,
wherein the similarity degree is a correlation coefficient.

### Claim 13

A non-transitory computer readable medium storing an image
processing program for sorting frames included in one moving image into different channels,
the frames included in the moving image being a mixture of frames belonging to the
different channels, the image processing program causing a computer to execute
a sorting step of calculating a similarity degree of the plurality of frames included in the moving image by performing image processing on the frames, and sorting the plurality of frames into the plurality of channels based on the similarity degree.

### Claim 14 (depends on 13)

The non-transitory computer readable medium according to claim 13,
wherein the sorting step includes the steps of:
setting a specific frame included in the moving image or a frame that is obtained by combining two or more specific frames included in the moving image as a reference frame;
setting another specific frame included in the moving image as a comparative frame;
dividing the comparative frame and the reference frame into a plurality of sub-regions;
calculating a local similarity degree between the comparative frame and the reference frame for each sub-region;
calculating a total similarity degree, which is a frame-overall similarity degree, between the comparative frame and the reference frame only using sub-regions having a high local similarity degree; and
executing determination processing of determining, based on the total similarity degree, whether or sot The comparative frame belongs to the same channel as that of the reference frame or the frames combined into the reference frame.

### Claim 15 (depends on 13)

The image processing program non-transitory computer readable medium according to claim 13,
wherein the sorting step includes the steps of:
setting a specific frame included in the moving image or a frame that is obtained by combining two or more specific frames included in the moving image as a reference frame;
setting another specific frame included in the moving image as a comparative frame;
detecting feature points from the comparative frame;
setting a vicinity region for each feature point;
calculating a total similarity degree, which is a frame-overall similarity degree, between the comparative frame and the reference frame only using the vicinity regions; and
determining, based on the total similarity degree, whether or not the comparative frame belongs to the same channel as that of the reference frame or the frames combined into the reference frame.

### Claim 16

An image processing method for sorting frames included in one moving image into different channels, the frames included in the moving image being a mixture of frames belonging to the different channels, the method comprising the step of
calculating a similarity degree of the plurality of frames included in the moving image by performing image processing on the frames, and sorting the plurality of frames into the plurality of channels based on the similarity degree.

### Claim 17 (depends on 4)

The image processing device according to claim 4,
wherein the calculation unit is configured to calculate, for each vicinity region, a local similarity degree between the comparative frame and the reference frame, and calculate the total similarity degree only using vicinity regions having a high local similarity degree.

### Claim 18 (depends on 3)

The image processing device according to claim 3,
wherein the setting unit is configured to accept, from a user, setting of an area in a frame screen, set an image in the area of the specific frame included in the moving image or a frame that is obtained by combining images in the area of the two or more specific frames included in the moving image, as the reference frame, and set an image in the area of the other specific frame included in the moving image as the comparative frame.

### Claim 19 (depends on 4)

The image processing device according to claim 4,
wherein the setting unit is configured to accept, from a user, setting of an area in a frame screen, set an image in the area of the specific frame included in the moving image or a frame that is obtained by combining images in the area of the two or more specific frames included in the moving image, as the reference frame, and set an image in the area of the other specific frame included in the moving image as the comparative frame.

### Claim 20 (depends on 5)

The image processing device according to claim 5,
wherein the setting unit is configured to accept, from a user, setting of an area in a frame screen, set an image in the area of the specific frame included in the moving image or a frame that is obtained by combining images in the area of the two or more specific frames included in the moving image, as the reference frame, and set an image in the area of the other specific frame included in the moving image as the comparative frame.