# US 11,030,257 B2 · Automatically generating theme-based folders by clustering media items in a semantic space > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,030,257 B2 - Document ID: US11030257B2 - Status: Granted - Kind: B2 - Application number: 16/417,232 - Filed: 2019-05-20 - Published: 2020-11-26 - Granted: 2021-06-08 - Earliest claimed: 2019-05-20 - Assignees: Adobe Inc. - Applicants: Adobe Inc. - Inventors: Jonas Dahl; Sudheer Tumu; Nithyanand Kota; Mihir Naware; Maneesh Dewan; Jatin Chhugani; Ganesh Satish Mallya - Web page: https://patentgrep.com/patent/US11030257B2 ## CPC - G06F 3/0483: Interaction with page-structured environments, e.g. book metaphor - G06F 16/166: {File name conversion} - G06F 16/168: {Details of user interfaces specifically adapted to file systems, e.g. browsing and visualisation, 2d or 3d GUIs (query results presentation G06F16/156)} - G06F 16/188: Virtual file systems - G06F 16/906: Clustering; Classification - G06F 18/23213: with fixed number of clusters, e.g. K-means clustering - G06N 3/02: Neural networks - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06V 10/48: by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation - G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions} - G06V 10/764: using classification, e.g. of video objects - G06V 10/82: using neural networks - G06V 10/945: {User interactive design; Environments; Toolboxes} - G06V 20/35: {Categorising the entire scene, e.g. birthday party or wedding scene} - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - G06N 3/045: Combinations of networks - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The present disclosure relates to systems, methods, and non-transitory computer readable media for clustering media items in a semantic space to generate theme-based folders that organize media items by content theme. In particular, the disclosed systems can access media items that are stored in an original folder structure. The disclosed systems can generate content-based tags for each media item in a collection of media items. Based on the generated tags, the disclosed systems can map the collection of media items to a semantic space and cluster the collection of media items. The disclosed systems determine themes for the clusters based on the generated tags. The disclosed systems can present a media item navigation graphical user interface comprising the collection of media items organized by themes. The disclosed system can present the media item navigation graphical user interface without altering the original folder structure. ## Claims ### Claim 1 A non-transitory computer readable medium for generating a customized user interface for navigating files, the non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computing device to: generate tags and confidence scores for media items in a collection of media items, wherein the confidence scores indicate how strongly a given tag corresponds to a given media item; map the collection of media items to a semantic space based on the tags; cluster the collection of media items in the semantic space to create clusters; determine themes for the clusters based on the tags of the media items in the clusters; generate a reverse index that maps the tags to associated media items with associated confidence scores; generate a media item navigation graphical user interface comprising the collection of media items organized by the themes and a search query element; receive a search query via the search query element; based on determining that the search query is not in the reverse index, determine a nearest term to the search query; and provide media items associated with the nearest term based on corresponding confidence scores. ### Claim 2 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the instructions, when executed by the at least one processor, cause the computing device to generate the tags for the media items in the collection of media items by: generating, utilizing a neural network, a feature map for a media item; and identifying, utilizing the neural network, one or more tags for the media item based on the feature map. ### Claim 3 (depends on 1) The non-transitory computer readable medium of claim 1, further comprising instructions, that when executed by the at least one processor, cause the computing device to generate confidence scores for each tag associated with a media item. ### Claim 4 (depends on 3) The non-transitory computer readable medium of claim 3, wherein the instructions, when executed by the at least one processor, cause the computing device to map the collection of media items to the semantic space by generating semantic feature vectors for the media items in the collection of media items. ### Claim 5 (depends on 4) The non-transitory computer readable medium of claim 4, wherein generating the semantic feature vectors comprises, for a given media item, converting one or more tags associated with the given media item into a semantic feature vector using a word to vector algorithm. ### Claim 6 (depends on 5) The non-transitory computer readable medium of claim 5, wherein the instructions, when executed by the at least one processor, cause the computing device to cluster the collection of media items in the semantic space by using K-means clustering on the semantic feature vectors. ### Claim 7 (depends on 5) The non-transitory computer readable medium of claim 5, wherein the instructions, when executed by the at least one processor, cause the computing device to cluster the collection of media items in the semantic space by processing the collection of media items using Latent Dirichlet Allocation. ### Claim 8 (depends on 5) The non-transitory computer readable medium of claim 5, wherein the instructions, when executed by the at least one processor, cause the computing device to cluster the collection of media items in the semantic space by: identifying cluster centers for the clusters in the semantic space; determining distances between the cluster centers and the semantic feature vectors; determining that a distance between a cluster center and a set of semantic feature vectors falls below a threshold; and assigning media items associated with set of semantic feature vectors to the cluster. ### Claim 9 (depends on 8) The non-transitory computer readable medium of claim 8, further comprising instructions that, when executed by the at least one processor, cause the computing device to determine themes for the clusters by: identifying cluster tags for media items associated with the set of semantic feature vectors in the cluster; determining a number of times each tag of the cluster tags is in the cluster; identifying a most frequently occurring tag in the cluster; and associating a human-understandable name associated with the most frequently occurring tag as the theme. ### Claim 10 A system comprising: at least one processor; and a computer readable storage medium storing instructions that, when executed by the at least one processor, cause the system to: generate tags and confidence scores for media items in a collection of media items utilizing a neural network, wherein the confidence scores indicate how strongly a given tag corresponds to a given media item; map the collection of media items to a semantic space based on the tags by generating semantic feature vectors for the media items of the collection of media items from the tags using a word to vector algorithm; cluster the collection of media items in the semantic space to create clusters by grouping similar semantic feature vectors; assign the media items of the collection of media items to the clusters; determine themes for the clusters based on the tags of the media items assigned to the clusters; generate a reverse index that maps the tags to associated media items with associated confidence scores; generate a media item navigation graphical user interface comprising the collection of media items organized into folders corresponding to the clusters and titled with the themes and a search query element; based on receiving a search query via the search query element, determine media items associated with the search query in the reverse index by: determining if the search query is in the reverse index; based on determining that the search query is not in the reverse index, determining a nearest term to the search query; and providing the media items associated with the nearest term based on the associated confidence scores; and provide, for display via the media item navigation graphical user interface, the media items associated with the search query. ### Claim 11 (depends on 10) The system of claim 10, wherein the instructions, when executed by the at least one processor, cause the system to assign the media items of the collection of media items to the clusters by: identifying cluster centroids in the semantic space for the clusters; determining distances between the cluster centroids and the semantic feature vectors; determining that a distance between a cluster centroid for a cluster and a set of semantic feature vectors falls below a distance threshold; and assigning media items associated with the set of semantic feature vectors to the cluster. ### Claim 12 (depends on 10) The system of claim 10, further comprising instructions that, when executed by the at least one processor, cause the system to determine themes for the clusters by: identifying cluster tags for media items associated with a set of semantic feature vectors in the cluster; determining a number of times each tag of the cluster tags is in the cluster; identifying a most frequently occurring tag in the cluster; and associating a human-understandable name associated with the most frequently occurring tag as the theme. ### Claim 13 (depends on 10) The system of claim 10, further comprising instructions that, when executed by the at least one processor, cause the system to: receive, via the search query element, a set of model media items, wherein the search query comprises the set of model media items; and determine the media items associated with the search query by: generating tags for the set of model media items; and determining media items associated with the tags for the set of model media items. ### Claim 14 (depends on 10) The system of claim 10, further comprising instructions, that when executed by the at least one processor, cause the system to provide the media items associated with the search query by: determining if the search query is in the reverse index; and based on determining that search query is in the reverse index, providing the media items associated with the search query in the reverse index in descending order based on the associated confidence scores. ### Claim 15 (depends on 10) The system of claim 10, further comprising instructions, that when executed by the at least one processor, cause the system to provide the media items associated with the nearest term in descending order based on the associated confidence scores. ### Claim 16 (depends on 10) The system of claim 10, further comprising instructions, that when executed by the at least one processor, cause the system to suggest additional query terms by suggesting one or more tags more frequently associated with the media items associated with the nearest term. ### Claim 17 (depends on 13) The system of claim 13, further comprising instructions, that when executed by the at least one processor, cause the system to provide the media items associated with the search query in themed-based search folders. ### Claim 18 In a digital medium environment for searching and presenting digital content, a computer-implemented method for recommending related media items, the computer-implemented method comprising: generating tags and confidence scores for media items in a collection of media items, wherein the confidence scores indicate how strongly a given tag corresponds to a given media item; mapping the collection of media items to a semantic space based on the tags; clustering the collection of media items in the semantic space to create clusters; determining themes for the clusters based on the tags of the media items in the clusters; generating a reverse index that maps the tags to associated media items with associated confidence scores; generating a media item navigation graphical user interface comprising the collection of media items organized by the themes and a search query element; receiving a search query via the search query element; based on determining that the search query is not in the reverse index, determining a nearest term to the search query; and providing media items associated with the nearest term based on corresponding confidence scores. ### Claim 19 (depends on 18) The method of claim 18, further comprising: receiving a selection of a theme-based folder; and organizing a subset of the media items in the collection of media items associated with the theme-based folder in a plurality of theme-based subfolders. ### Claim 20 (depends on 18) The method of claim 18, wherein organizing the media items in the collection of media items in theme-based folders is performed without modifying an existing folder structure in which the collection of media items are stored.