# US 2013/0156321 A1 · Video processing apparatus and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2013/0156321 A1 - Document ID: US20130156321A1 - Status: Published application - Kind: A1 - Application number: 13/705,399 - Filed: 2012-12-05 - Published: 2013-06-20 - Earliest claimed: 2011-12-16 - Assignees: Kabushiki Kaisha Toshiba - Inventors: Shigeru MOTOI; Koji Yamamoto - Web page: https://patentgrep.com/patent/US20130156321A1 ## CPC - G06V 20/47: {Detecting features for summarising video content} - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} - H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N 21/8549: Creating video summaries, e.g. movie trailer {(retrieval in video databases by using presentations in form of a video summary G06F16/739)} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In one embodiment, a method is disclosed for performing a video processing. The method can extract one or more common video segments. The method can select a common summarization segment based on a first summarization score. The method can extract one or more individual video segments in which the number of segments included therein is not more than a third threshold value that is less than a second threshold value. The method can select an individual summarization segment based on a second summarization score. In addition, the method can integrate the common summarization segment and the individual summarization segment to create the summary video. ## Claims ### Claim 1 A video processing method, comprising: acquiring a plurality of video streams each including moving picture data item and sound data item; extracting one or more feature values indicating a common feature between the video streams and changing in time series, by analyzing at least one of the moving picture data item and the sound data item for each video stream; generating a plurality of segments by dividing each of the video streams in accordance with changes in the one or more feature values, and generating associated segment groups by comparing the first feature values for each of the segments to obtain similarities and associating a plurality of segments in different video streams such that the segments included in each of the associated segment groups have a similarity not less than a first threshold value; extracting, from the associated segment groups, one or more common video segments in which the number of segments included therein is not less than a second threshold value; calculating a first summarization score indicating a degree of suitability for including the one or more common video segments in a summary video, the first summarization score being defined for each common video segment and varying with time and being based on the one or more feature values; selecting a common summarization segment to be used for the summary video from the common video segments based on the first summarization score; extracting, from the associated segment groups, one or more individual video segments in which the number of segments included therein is not more than a third threshold value that is less than the second threshold value; calculating a second summarization score indicating a degree of suitability for including the one or more individual video segments in the summary video, the second summarization score being defined for each individual video segment and varying with time and being based on the one or more feature values; selecting an individual summarization segment to be used for the summary video from the individual video segments based on the second summarization score; and integrating the common summarization segment and the individual summarization segment to create the summary video. ### Claim 2 (depends on 1) The method according to claim 1, wherein the calculating the second summarization score calculates a first difference between second feature values of the one or more common video segments and third feature values of the one or more individual video segments, and weights the second summarization score of the individual video segments to be higher as the first difference becomes larger. ### Claim 3 (depends on 1) The method according to claim 1, wherein the calculating the second summarization score calculates a second difference between the feature values of the one or more individual video segments, and weights the second summarization score of the individual video segments to be higher as the second difference becomes larger. ### Claim 4 (depends on 1) The method according to claim 1, wherein the calculating the second summarization score weights the second summarization score of the individual video segment to be higher as the number of segments associated with for the individual video segments becomes smaller. ### Claim 5 (depends on 1) The method according to claim 1, further comprising: calculating similarities between a feature value model of a typical pattern and the feature values of the one or more individual video segments to detect a first individual video segment having a similarity not less than a fourth threshold value, the typical pattern indicating a combination of video segments constituting a predetermined video sequence; and adding a correction value to the second summarization score of the first individual video segment to generate a corrected summarization score, wherein the selecting the individual summarization segment selects the individual summarization segment based on the second summarization score and the corrected summarization score. ### Claim 6 (depends on 1) The method according to claim 1, further comprising selecting a summarization object that is to be a main object of the summary video in each of the one or more common video segments and individual video segments, wherein the calculating the second summarization score weights the second summarization score of the individual video segment which includes the summarization object to be high. ### Claim 7 A non-transitory computer readable medium including computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform a method comprising: acquiring a plurality of video streams each including moving picture data item and sound data item; extracting one or more feature values indicating a common feature between the video streams and changing in time series, by analyzing at least one of the moving picture data item and the sound data item for each video stream; generating a plurality of segments by dividing each of the video streams in accordance with changes in the one or more feature values, and generating associated segment groups by comparing the first feature values for each of the segments to obtain similarities and associating a plurality of segments in different video streams such that the segments included in each of the associated segment groups have a similarity not less than a first threshold value; extracting, from the associated segment groups, one or more common video segments in which the number of segments included therein is not less than a second threshold value; calculating a first summarization score indicating a degree of suitability for including the one or more common video segments in a summary video, the first summarization score being defined for each common video segment and varying with time and being based on the one or more feature values; selecting a common summarization segment to be used for the summary video from the common video segments based on the first summarization score; extracting, from the associated segment groups, one or more individual video segments in which the number of segments included therein is not more than a third threshold value that is less than the second threshold value; calculating a second summarization score indicating a degree of suitability for including the one or more individual video segments in the summary video, the second summarization score being defined for each individual video segment and varying with time and being based on the one or more feature values; selecting an individual summarization segment to be used for the summary video from the individual video segments based on the second summarization score; and integrating the common summarization segment and the individual summarization segment to create the summary video. ### Claim 8 (depends on 7) The medium according to claim 7, wherein the calculating the second summarization score calculates a first difference between second feature values of the one or more common video segments and third feature values of the one or more individual video segments, and weights the second summarization score of the individual video segments to be higher as the first difference becomes larger. ### Claim 9 (depends on 7) The medium according to claim 7, wherein the calculating the second summarization score calculates a second difference between the feature values of the one or more individual video segments, and weights the second summarization score of the individual video segments to be higher as the second difference becomes larger. ### Claim 10 (depends on 7) The medium according to claim 7, wherein the calculating the second summarization score weights the second summarization score of the individual video segment to be higher as the number of segments associated with for the individual video segments becomes smaller. ### Claim 11 (depends on 7) The medium according to claim 7, further comprising: calculating similarities between a feature value model of a typical pattern and the feature values of the one or more individual video segments to detect a first individual video segment having a similarity not less than a fourth threshold value, the typical pattern indicating a combination of video segments constituting a predetermined video sequence; and adding a correction value to the second summarization score of the first individual video segment to generate a corrected summarization score, wherein the selecting the individual summarization segment selects the individual summarization segment based on the second summarization score and the corrected summarization score. ### Claim 12 (depends on 7) The medium according to claim 7, further comprising selecting a summarization object that is to be a main object of the summary video in each of the one or more common video segments and individual video segments, wherein the calculating the second summarization score weights the second summarization score of the individual video segment which includes the summarization object to be high. ### Claim 13 An image processing apparatus, comprising: an acquisition module configured to acquire one or more still picture groups each including a plurality of still pictures; a first extraction module configured to extract one or more feature values indicating a common feature between the still picture groups, by analyzing the one or more still picture groups; a generation module configured to generate a plurality of associated still picture groups by comparing the feature values for each of the still pictures to obtain similarities and associating a plurality of still pictures such that the still pictures included in each of the associated still picture groups have a similarity not less than a first threshold value; a second extraction module configured to extract, from the associated still picture groups, common still picture groups in which the number of still pictures included therein is not less than a second threshold value; a first calculation module configured to calculate a first summarization score indicating a degree of suitability for each still picture of the common still picture groups in a summary still picture, the first summarization score being defined for each common still picture group and varying with time and being based on the one or more feature values; a first selection module configured to select still pictures to be used for the summary still picture as common summarization still pictures from the common still picture groups based on the first summarization score; a third extraction module configured to extract, from the associated still picture groups, individual still picture groups in which the number of still pictures included therein is not less than a third threshold value that is less than the second threshold value; a second calculation module configured to calculate a second summarization score indicating a degree of suitability for each still picture of the individual still picture groups in the summary still picture, the second summarization score being defined for each individual still picture group and varying with time and being based on the one or more feature values; a second selection module configured to select still pictures to be used for the summary still picture as individual summarization still pictures from the individual still picture groups based on the second summarization score; and an integration module configured to integrate the common summarization still pictures and the individual summarization still pictures to create the summary still picture. ### Claim 14 (depends on 13) The apparatus according to claim 13, wherein the second calculation module calculates a first difference between first feature values of the common still picture groups and second feature values of the individual still picture groups, and weights the second summarization score of the individual still picture groups to be higher as the first difference becomes larger. ### Claim 15 (depends on 13) The apparatus according to claim 13, wherein the second calculation module calculates a second difference between the feature values of the one or more individual still picture groups, and weights the second summarization score of the individual still picture groups to be higher as the second difference becomes larger. ### Claim 16 (depends on 13) The apparatus according to claim 13, wherein the second calculation module weights the second summarization score of the individual still picture groups to be higher as the number of still pictures associated with for the individual still picture groups becomes smaller. ### Claim 17 (depends on 13) The apparatus according to claim 13, further comprising: a detection module configured to calculate similarities between a feature value model of a typical pattern and the feature values of the individual still picture groups, and to detect a first individual still picture group having a similarity not less than a fourth threshold value, the typical pattern indicating a combination of still pictures constituting a predetermined video sequence; and a correction module configured to add a correction value to the second summarization score of the first individual still picture group to generate a corrected summarization score, wherein the second selection module selects the individual summarization still pictures based on the second summarization score and the corrected summarization score. ### Claim 18 (depends on 13) The apparatus according to claim 13, further comprising a third selection module configure to select a summarization object that is to be a main object of the summary still picture in each of the common still picture groups and individual still picture groups, wherein the second calculation module weights the second summarization score of the individual still picture group including the summarization object to be high.