# US 10,373,648 B2 · Apparatus and method for editing content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,373,648 B2 - Document ID: US10373648B2 - Status: Granted - Kind: B2 - Application number: 14/969,366 - Filed: 2015-12-15 - Published: 2016-07-21 - Granted: 2019-08-06 - Earliest claimed: 2015-01-20 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Joon-gyu Sun; Han-min Bang; Seong-kook Shin - Web page: https://patentgrep.com/patent/US10373648B2 ## CPC - G11B 27/036: Insert-editing - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - G11B 27/3081: {used signal is a video-frame or a video-field (P.I.P)} - G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} - G06V 40/172: {Classification, e.g. identification} - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 25/51: for comparison or discrimination - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A system and a method for editing moving image content are provided. The method includes acquiring moving image content, mapping an image of a person included in the moving image content and speech data of the person, selecting at least one frame among frames included in the moving image content, and creating edited content of the moving image content using the mapped image and speech data, and the selected at least one frame. ## Claims ### Claim 1 A method of a content editing apparatus for editing moving image content, the method comprising: acquiring moving image content; generating a face database (DB) of face images based on face data extracted from the moving image content; generating a speech DB of speech based on speech data extracted from the moving image content; mapping an image of a person included in the face DB with speech data of the person included in the speech DB; selecting at least one frame among frames included in the moving image content; and creating edited image content of the moving image content using the mapped image and speech data, and the selected at least one frame, wherein the selecting of the at least one frame among the frames included in the moving image content comprises: determining at least one scene among scenes of the moving image content based on a voice level variation and a voice frequency variation of voice data in the moving image content, and selecting at least one frame among frames included in the determined scene, the at least one frame comprising fewer than all of the frames included in the determined scene, wherein the edited image content includes at least one image segment corresponding to the selected at least one frame, and wherein the image segment includes an image of the selected frame and speech data corresponding to the selected frame. ### Claim 2 (depends on 1) The method of claim 1, wherein the mapping of the image of the person included in the moving image content and speech data of the person comprises: classifying face images of persons in the face DB for each person; and mapping the classified face images to speech data of the persons included in the speech DB. ### Claim 3 (depends on 2) The method of claim 2, wherein the speech data of the person includes at least one of voice data of the person or text data converted from the voice data. ### Claim 4 (depends on 1) The method of claim 1, wherein the determining of the at least one scene among scenes of the moving image content comprises: detecting a predetermined voice in the moving image content; and determining a scene corresponding to a position at which a predetermined condition is satisfied based on a position of the detected voice. ### Claim 5 (depends on 4) The method of claim 4, wherein the predetermined voice includes a burst sound, and wherein the scene corresponding to the position at which the predetermined condition is satisfied is a scene ranging between a portion of the moving image content in which the burst sound is output and another portion of the moving image content spaced apart a certain distance in a reverse direction. ### Claim 6 (depends on 1) The method of claim 1, wherein the selecting of the at least one frame comprises selecting the at least one frame according to a level of noise included in the frames included in the determined scene. ### Claim 7 (depends on 1) The method of claim 1, wherein the selecting of the at least one frame comprises selecting the at least one frame based on at least one of a line-of-sight direction, a face direction, or a facial expression of persons included in the frames included in the determined scene. ### Claim 8 (depends on 1) The method of claim 1, wherein the creating of the edited image content further comprises: determining a template used to create the edited image content; and inserting an image of the selected frame into the determined template. ### Claim 9 (depends on 8) The method of claim 8, wherein text converted from a voice uttered by a person included in the image of the frame is displayed in the image inserted into the template. ### Claim 10 (depends on 8) The method of claim 8, wherein the determining of the template comprises determining a type of the edited image content and selecting a template corresponding to the determined type, and wherein the image inserted into the template is processed according to the type of the edited image content. ### Claim 11 (depends on 1) A non-transitory computer readable recording medium storing a computer program for executing by a processor the method of claim 1. ### Claim 12 A content editing apparatus for editing moving image content, the content editing apparatus comprising: a memory configured to: store a face database (DB) of face images included in the moving image content, store a speech DB of speech included in the moving image content, and store a mapping of an image of a person included in the face DB with speech data of the person included in the speech DB; and at least one processor configured to: generate the face DB based on face data extracted from the moving image content, generate the speech DB based on speech data extracted from the moving image content, map the image of a person included in the face DB with the speech data of the person included in the speech DB, select at least one frame among frames included in the moving image content, and create edited image content of the moving image content using the mapped image and speech data, and the selected at least one frame, wherein the at least one processor is further configured to: determine at least one scene among scenes of the moving image content based on a voice level variation and a voice frequency variation of voice data in the moving image content, and select at least one frame among frames included in the determined scene, the at least one frame comprising fewer than all of the frames included in the determined scene, wherein the edited image content includes at least one image segment corresponding to the selected at least one frame, and wherein the image segment includes an image of the selected frame and speech data corresponding to the selected frame. ### Claim 13 (depends on 12) The content editing apparatus of claim 12, wherein the at least one processor is further configured to: classify face images of persons in the face DB for each person, and store the mapping of the classified face images and speech data of the persons in the storage unit. ### Claim 14 (depends on 13) The content editing apparatus of claim 13, wherein the speech data of the person includes at least one of voice data of the person or text data converted from the voice data. ### Claim 15 (depends on 12) The content editing apparatus of claim 12, wherein the at least one processor is further configured to: detect a predetermined voice in the moving image content, and determine a scene corresponding to a position at which a predetermined condition is satisfied based on a position of the detected voice. ### Claim 16 (depends on 15) The content editing apparatus of claim 15, wherein the predetermined voice includes a burst sound, and wherein the scene corresponding to the position at which the predetermined condition is satisfied is a scene ranging between a portion of the moving image content in which the burst sound is output and another portion of the moving image content spaced apart a certain distance in a reverse direction. ### Claim 17 (depends on 12) The content editing apparatus of claim 12, wherein the at least one processor is further configured to select the at least one frame according to a level of noise included in the frames included in the determined scene. ### Claim 18 (depends on 12) The content editing apparatus of claim 12, wherein the at least one processor is further configured to select the at least one frames based on at least one of a line-of-sight direction, a face direction, or a facial expression of persons included in the frames included in the determined scene. ### Claim 19 (depends on 12) The content editing apparatus of claim 12, wherein the at least one processor is further configured to: determine a template used to create the edited image content, and insert an image of the selected frame into the determined template. ### Claim 20 (depends on 19) The content editing apparatus of claim 19, wherein text converted from a voice uttered by a person included in the image of the frame is displayed in the image inserted into the template. ### Claim 21 (depends on 19) The content editing apparatus of claim 19, wherein the at least one processor is further configured to: determine a type of the edited image content, and select a template corresponding to the determined type, and wherein the image inserted into the template is processed according to the type of the edited image content. ### Claim 22 (depends on 1) The method of claim 1, wherein the selection of the at least one frame comprises selecting the at least one frame from among the frames included in the moving image content based on features related to the eyes of a subject appearing in the moving content.