# US 2017/0287346 A1 · System and methods to create multi-faceted index for instructional videos > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0287346 A1 - Document ID: US20170287346A1 - Status: Published application - Kind: A1 - Application number: 15/088,508 - Filed: 2016-04-01 - Published: 2017-10-05 - Earliest claimed: 2016-04-01 - Assignees: Yen4Ken Inc. - Applicants: Yen4Ken Inc. - Inventors: Kuldeep Yadav; Arijit Biswas; Ankit Gandhi; Sumit Negi; Om D. Deshmukh - Web page: https://patentgrep.com/patent/US20170287346A1 ## CPC - G09B 5/065: {Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems} - G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation - G06V 10/40: Extraction of image or video features - G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) - G06V 30/224: of printed characters having additional code marks or containing code marks - G09B 5/08: providing for individual presentation of information to a plurality of student stations - G11B 27/105: {of operating discs} - G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} - G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract Features are extracted from visual and audio modalities of a video to infer the location of figures/tables/equations/graphs/flow-charts determined as video anchor points which are highlighted on the video timeline to enable quick navigation and provide a quick summary of the video. A voice-based mechanism navigates to a point-of-interest in the video. In case of bandwidth-constrained settings, videos are often played at a very low resolution (quality), and often users need to increase video resolution manually to understand content presented in the figures. Using the automatic identification of these aforementioned anchored points, the resolution can be changed dynamically during streaming a video, which will provide a better viewing experience. ## Claims ### Claim 1 An indexing system for video content including a plurality of anchor points of selected interest to a user of the video content, comprising: an anchor point identifying processor configured to identify occurrence of selected ones of the anchor points based on high-level visual elements, the high-level visual elements are identified from multi-modal inputs, such as audio and visual data from a video and textual data obtained from the audio and visual data; a label assignation processor configured to assign labels to the identified anchor points; and an index generator configured to form and display in association with the video content a multi-faceted index, comprising a relative representation of position of a selected one of this anchor points within the video content, and a 2-dimensional timeline identifying a plurality of categories of the anchor points. ### Claim 2 (depends on 1) The system of claim 1, comprising a resolution changer configured to adjust displayed resolution of a one of the anchor points selected for display by the user via the index. ### Claim 3 (depends on 1) The system of claim 1, wherein the identifying processor comprises a segmentation of text and non-text regions. ### Claim 4 (depends on 3) The system of claim 3, wherein the identifying processor comprises optical character recognition of the text region for the anchor points. ### Claim 5 (depends on 3) The system of claim 3, wherein the identifying processor comprises feature extraction and classification of the non-text region for the anchor points. ### Claim 6 (depends on 5) The system of claim 5, wherein the feature extraction and classification comprises SIFT and SURF extraction. ### Claim 7 (depends on 5) The system of claim 5, wherein the feature extraction and classification comprises conventional neural network classification. ### Claim 8 (depends on 1) The system of claim 1, wherein the index generator comprises a timeline relative presentation of anchor point position in the video content. ### Claim 9 (depends on 1) The system of claim 1, wherein the index generator comprises a topic table. ### Claim 10 (depends on 1) The system of claim 1, including a voice based navigation processor configured to navigate to an identified anchor point. ### Claim 11 (depends on 1) The system of claim 1, wherein the label assignation processor references associated topic discussion to determine the assigned topic. ### Claim 12 (depends on 11) The system of claim 11, wherein the label assignation processor comprises at least one of stop-words removal, time specific relevance, many to one stemming and majority vote inversion. ### Claim 13 (depends on 1) The system of claim 1, further comprising: an index generator further configured to provide an overlay or interactive summary of anchor points over the video, wherein the anchor points provide a link to relevant section of the video.