# US 2013/0094831 A1 · Image processing apparatus, image processing method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0094831 A1
- Document ID: US20130094831A1
- Status: Published application
- Kind: A1
- Application number: 13/608,439
- Filed: 2012-09-10
- Published: 2013-04-18
- Earliest claimed: 2011-10-18
- Assignees: Sony Corporation
- Inventors: Hirotaka Suzuki
- Web page: https://patentgrep.com/patent/US20130094831A1

## CPC

- G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)}
- G11B 27/11: by using information not detectable on the record carrier
- H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
- H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments}
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing apparatus includes a display control unit displaying a segment map, in which segment-representing images representing video segments obtained by dividing content into video segments which are collections of frames of one or more temporally continuous frames, are arranged on a display apparatus and a reproduction control unit controlling reproduction of the video segments corresponding to the segment-representing image according to a reproduction operation of a user requesting reproduction with respect to the segment-representing image, in which, when, during reproduction of a video segment of content, a reproduction operation is performed with respect to a segment-representing image of another video segment of the content, the reproduction control unit reproduces the other video segment corresponding to the segment-representing image for which the reproduction operation was performed, while still maintaining the reproduction of the video segment being reproduced.

## Claims

### Claim 1

An image processing apparatus comprising:
a display control unit displaying a segment map, which is a map in which segment-representing images, which are images representing video segments obtained by dividing content into video segments which are collections of frames of one or more temporally continuous frames, are arranged, on a display apparatus; and
a reproduction control unit controlling reproduction of the video segments corresponding to the segment-representing image according to a reproduction operation of a user requesting reproduction with respect to the segment-representing image,
wherein, in a case where, during reproduction of a video segment of content, a reproduction operation is performed with respect to a segment-representing image of another video segment of content, the reproduction control unit reproduces another video segment, which is the video segment corresponding to the segment-representing image for which the reproduction operation was performed, while still maintaining reproduction of the video segment being reproduced.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, further comprising:
a map generating unit generating the segment map;
a clustering unit clustering each frame of the content into clusters of any one of a plurality of clusters;
a segment dividing unit dividing frames belonging to clusters into the video segments which are collections of one or more temporally continuous frames respectively for the plurality of clusters; and
a segment-representing image generating unit generating segment-representing images representing the video segments,
wherein, the map generating unit associates segment-representing images of a video segment belonging to a cluster with the cluster for each of the plurality of clusters, selects a cluster representation image representing the cluster from the segment-representing images associated with the cluster, and
arranges the segment-representing image selected as the cluster representation image on the segment map.

### Claim 3 (depends on 2)

The image processing apparatus according to claim 2, wherein the clustering unit determines a maximum likelihood state series in which characteristic amounts in chronological series of frames of the content are observed using a state transition model having states and state transitions, and
clusters each frame of the content into clusters corresponding to states in which characteristic amounts of the frames are observed in the maximum likelihood state series.

### Claim 4 (depends on 3)

The image processing apparatus according to claim 3, wherein the map generating unit generates the segment map by generating a model map which is a map in which the state transition model is projected on a two-dimensional flat surface or into three-dimensional space, and
arranging the segment-representing images selected as cluster representation images of clusters corresponding to the states at positions of each state on the model map.

### Claim 5 (depends on 4)

The image processing apparatus according to claim 4, wherein the map generating unit selects a segment-representing image of a video segment for which a number of frames is largest or a segment-representing image of a video segment for which a reproduction time is earliest among the segment-representing images associated with the cluster corresponding to the state, as the cluster representation image.

### Claim 6 (depends on 4)

The image processing apparatus according to claim 4, wherein the map generating unit arranges the segment-representing image selected as a cluster representation image of a cluster corresponding to the state at positions of each state on the model map at a size corresponding to an importance of the state.

### Claim 7 (depends on 6)

The image processing apparatus according to claim 6, wherein the map generating unit determines a largest number of frames of a video segment corresponding to the segment-representing images associated with the cluster corresponding to the state as the importance of the state.

### Claim 8 (depends on 4)

The image processing apparatus according to claim 4, further comprising:
a storage unit storing a state as a registered state according to a favoriting operation of a user requesting that the state be registered,
wherein the display control unit highlights and displays the segment-representing image of the segment map selected as the cluster representation image of a cluster corresponding to the registered state.

### Claim 9 (depends on 8)

The image processing apparatus according to claim 8,
wherein, when a reproduction operation is performed with respect to the segment-representing image selected as the cluster representation image of the cluster corresponding to the registered state, the reproduction control unit reproduces, in chronological order, the video segments corresponding to the segment-representing images associated with the clusters corresponding to all the registered states stored in the storage unit.

### Claim 10 (depends on 4)

The image processing apparatus according to claim 4,
wherein the display control unit sets the state on the segment map closest to an instructed position instructed by the user as a state of interest to be focused on and performs display control displaying, on the display apparatus, a tile image which is an image of a list of the segment-representing images associated with the clusters corresponding to the state of interest.

### Claim 11 (depends on 10)

The image processing apparatus according to claim 10,
wherein the display control unit sets the segment-representing image closest to the instructed position as an image of interest to be focused on from among the segment-representing images of the tile image and highlights and displays the image of interest.

### Claim 12 (depends on 11)

The image processing apparatus according to claim 11,
wherein, according to a predetermined operation of a user, the display control unit performs display control displaying, on the display apparatus, a timeline image in which, from among the segment-representing images associated with the state of interest, the image of interest and a predetermined number of the segment-representing images continuing before and after the image of interest are arranged in chronological order with the image of interest as a center.

### Claim 13 (depends on 12)

The image processing apparatus according to claim 12,
wherein the display control unit scrolls the segment-representing image arranged in the timeline image according to an operation of the user.

### Claim 14

An image processing method comprising:
displaying a segment map, which is a map in which segment-representing images, which are images representing video segments obtained by dividing content into video segments which are collections of frames of one or more temporally continuous frames, are arranged, on a display apparatus; and
controlling reproduction of the video segments corresponding to the segment-representing image according to a reproduction operation of a user requesting reproduction with respect to the segment-representing image,
wherein, in the controlling of the reproduction of the video segment, in a case where, during the reproduction of a video segment of content, a reproduction operation is performed with respect to a segment-representing image of another video segment of the content, another video segment, which is a video segment corresponding to a segment-representing image for which the reproduction operation was performed, is reproduced while still maintaining the reproduction of the video segment being reproduced.

### Claim 15

A program causing a computer to execute:
a displaying a segment map, which is a map in which segment-representing images, which are images representing video segments obtained by dividing content into video segments which are collections of frames of one or more temporally continuous frames, are arranged on a display apparatus; and
a controlling reproduction of video segments corresponding to the segment-representing image according to a reproduction operation of a user requesting reproduction with respect to the segment-representing image,
wherein, in a case where, during reproduction of a video segment of content, a reproduction operation is performed with respect to a segment-representing image of another video segment of content, another video segment, which is a video segment corresponding to the segment-representing image for which the reproduction operation was performed, is reproduced while still maintaining the reproduction of the video segment being reproduced.
