# US 9,436,358 B2 · Systems and methods for editing three-dimensional video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,358 B2 - Document ID: US9436358B2 - Status: Granted - Kind: B2 - Application number: 13/788,203 - Filed: 2013-03-07 - Published: 2014-09-11 - Granted: 2016-09-06 - Earliest claimed: 2013-03-07 - Assignees: Cyberlink Corp. - Applicants: Cyberlink Corp. - Inventors: Chia-Che Yang; Chun Ming Yeh - Web page: https://patentgrep.com/patent/US9436358B2 ## CPC - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Various embodiments are disclosed for editing a video sequence. One embodiment, among others, is a method for editing a video in a video editing system. The method comprises obtaining a video sequence and identifying objects in the video sequence with corresponding depth information, displaying the video sequence in a display panel in a user interface, and facilitating insertion of an object into the video sequence by displaying in the user interface, a timeline and a control panel including a graphical representation for each of the identified objects with corresponding depth information. The graphical representations are arranged relative to the timeline and based on the corresponding depth information. The method further comprises obtaining input from a user positioning the object into the video sequence relative to the identified objects with corresponding depth information. ## Claims ### Claim 1 A method for editing a video in a video editing system, comprising: obtaining a video sequence and identifying objects in a frame of the video sequence, the identified objects in the frame each having corresponding depth information; displaying the video sequence in a display panel in a user interface; facilitating insertion of a new object into the frame of the video sequence by displaying in the user interface, a timeline and a control panel including a graphical representation for each of the identified objects with corresponding depth information, wherein the graphical representations are arranged according to an axis corresponding to time and an axis corresponding to depth; and obtaining input from a user positioning the new object into the video sequence relative to the identified objects with corresponding depth information. ### Claim 2 (depends on 1) The method of claim 1, wherein the graphical representations comprise line objects, and wherein a length of each of the line objects corresponds to a duration of the object depicted in the video sequence. ### Claim 3 (depends on 2) The method of claim 2, wherein a location of each of the graphical representations corresponds to a relative depth of each corresponding object. ### Claim 4 (depends on 2) The method of claim 2, wherein the line objects extend in a horizontal direction along the axis corresponding to time. ### Claim 5 (depends on 1) The method of claim 1, wherein obtaining the user input positioning the object into the video sequence comprises: displaying a graphical representation of the inserted new object relative to the identified objects with corresponding depth information; obtaining a position of the inserted new object relative to the identified objects with corresponding depth information; and based on the obtained position, displaying the new object inserted into the video sequence relative to the identified objects with corresponding depth information. ### Claim 6 (depends on 5) The method of claim 5, wherein obtaining the user input positioning the new object into the video sequence further comprises obtaining a position of the inserted new object relative to the timeline. ### Claim 7 (depends on 5) The method of claim 5, wherein displaying the new object inserted into the video sequence relative to the identified objects with corresponding depth information based on the obtained position comprises displaying the inserted new object occluding or being occluded by the identified objects with corresponding depth information. ### Claim 8 (depends on 1) The method of claim 1, wherein the new object comprises a graphic or text. ### Claim 9 (depends on 1) The method of claim 1, wherein in response to hovering a pointer over a graphical representation in the control panel, the object corresponding to the graphical representation is highlighted to the user. ### Claim 10 (depends on 9) The method of claim 9, wherein the object is highlighted by displaying a graphic on or near the object in the display panel. ### Claim 11 (depends on 1) The method of claim 1, wherein in response to hovering a pointer over a graphical representation in the control panel, the object corresponding to the graphical representation is shown in the form of a thumbnail image in the control panel. ### Claim 12 (depends on 1) The method of claim 1, further comprising obtaining user input specifying a viewing angle for displaying the video sequence in the display panel. ### Claim 13 (depends on 12) The method of claim 12, wherein the user input for specifying the viewing angle comprises at least one of input defined via navigation of a pointing device, input via a touch panel, a keyboard input, and an audio input. ### Claim 14 (depends on 1) The method of claim 1, further comprising dividing the video sequence into additional video sequences according to scene changes, wherein a scene change corresponds to at least a change in depth of at least one object in the video sequence. ### Claim 15 A method for editing a frame in a video editing system, comprising: obtaining a frame within a video sequence and identifying objects in the frame with corresponding depth information; displaying the frame in a display panel in a user interface; facilitating insertion of a new object into the frame by displaying in the user interface, a control panel including a graphical representation for each of the identified objects with corresponding depth information, wherein the graphical representations are depicted based on object size and relative positioning of each of the identified objects, and based on their corresponding depth information wherein the graphical representations are arranged according to an axis corresponding to time and an axis corresponding to depth; and obtaining input from a user positioning the new object into the frame relative to the identified objects with corresponding depth information. ### Claim 16 A system for editing a video, comprising: a memory comprising program instructions; a processor coupled to the memory and configured by the program instructions to: obtain a video sequence; identify objects in a frame of the video sequence, the identified objects in the frame each having corresponding depth information; display the video sequence in a display panel in a user interface; display in the user interface, a timeline and a control panel including a graphical representation for each of the identified objects with corresponding depth information, wherein the graphical representations are arranged according to an axis corresponding to time and an axis corresponding to depth; and apply a special effect to the video sequence, wherein the special effect comprises positioning of a new object into the video sequence relative to the identified objects with corresponding depth information. ### Claim 17 (depends on 16) The system of claim 16, wherein the graphical representations comprise line objects, wherein a length of the line objects correspond to a duration of the object depicted in the video sequence. ### Claim 18 (depends on 17) The system of claim 17, wherein a location of the graphical representations correspond to a relative depth of each corresponding object. ### Claim 19 (depends on 16) The system of claim 16, wherein the user interface component is further configured to: display a graphical representation of the inserted new object relative to the identified objects with corresponding depth information; obtain a position of the inserted new object relative to the identified objects with corresponding depth information; and based on the obtained position, display the new object inserted into the video sequence relative to the identified objects with corresponding depth information. ### Claim 20 A non-transitory computer-readable medium embodying a program executable in a computing device, comprising: code that obtains a video sequence and identifies objects in a frame of the video sequence, the identified objects in the frame each having corresponding depth information; code that displays the video sequence in a display panel in a user interface; code that displays in the user interface, a timeline and a control panel including a graphical representation for each of the identified objects with corresponding depth information, wherein the graphical representations are arranged according to an axis corresponding to time and an axis corresponding to depth; and code that obtains input from a user positioning a new object into the video sequence relative to the identified objects with corresponding depth information. ### Claim 21 (depends on 20) The non-transitory computer-readable medium of claim 20, wherein the control panel corresponds to a top view of the objects with corresponding depth information. ### Claim 22 (depends on 21) The non-transitory computer-readable medium of claim 21, wherein spacing between the objects along a first axis represents a relative difference in depth between the objects. ### Claim 23 (depends on 22) The non-transitory computer-readable medium of claim 22, wherein locations and lengths of the objects along a second axis correspond to timing information.