# US 2021/0240757 A1 · Automatic Detection and Transfer of Relevant Image Data to Content Collections > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240757 A1 - Document ID: US20210240757A1 - Status: Published application - Kind: A1 - Application number: 17/170,697 - Filed: 2021-02-08 - Published: 2021-08-05 - Earliest claimed: 2016-07-25 - Assignees: Evernote Corporation - Applicants: Evernote Corporation - Inventors: Dylan Marriott; Daniel Nicolae; Ruben Bakker; Alexander Pashintsev; Zdzislaw Pawel Losvik; Eugene Livshitz; Vitaly Glazkov; Boris Gorbatov; Ilya Buryak - Web page: https://patentgrep.com/patent/US20210240757A1 ## CPC - G06F 16/51: Indexing; Data structures therefor; Storage structures - G06F 16/24565: {Triggers; Constraints} - G06F 16/24578: {using ranking} - G06F 16/40: of multimedia data, e.g. slideshows comprising image and additional audio data (retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70) - G06F 16/434: {using image data, e.g. images, photos, pictures taken by a user} - G06F 16/55: Clustering; Classification - G06F 16/5846: {using extracted text} - G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information} - G06Q 30/02: Marketing; Price estimation or determination; Fundraising - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract This application is directed to a method for automatically identifying and transferring relevant image data. The method includes obtaining a plurality of content items from a personal content collection and determining attributes based on the plurality of content items. The method includes generating a plurality of relevance rules. The method further includes obtaining unclassified content items and determining for a first unclassified content item a plurality of aggregate relevance scores using the plurality of relevance rules. The method include determining whether a first aggregate relevance score and/or a second aggregate relevance score satisfy threshold score. The method includes, in accordance with a determination that the first aggregate relevance score and the second aggregate relevance score do not satisfy the threshold score, forgoing determining the attributes corresponding to the first unclassified content item, and storing the first unclassified content item in a candidate list of the personal content collection. ## Claims ### Claim 1 A method comprising: at a server having one or more processors and memory storing programs executed by the one or more processors: obtaining a plurality of content items from a personal content collection of a user; determining one or more attributes based on the plurality of content items, wherein the one or more attributes include one or more of content categories, keywords, and user groupings; generating a plurality of relevance rules, wherein each relevance rule is based on a subset of the one or more attributes, and is configured to associate the one or more attributes with an unclassified content item; obtaining, from the user, one or more unclassified content items; determining for a first unclassified content item of the one or more unclassified content items a plurality of aggregate relevance scores using the plurality of relevance rules, the plurality of aggregate relevance scores including: a first aggregate relevance score based on a first relevance rule of the plurality of relevance rules, and a second aggregate relevance score based on a second relevance rule of the plurality of relevance rules; determining whether the first aggregate relevance score and/or the second aggregate relevance score satisfy threshold score; and in accordance with a determination that the first aggregate relevance score and the second aggregate relevance score do not satisfy the threshold score: forgoing determining the one or more attributes corresponding to the first unclassified content item, and storing the first unclassified content item in a candidate list of the personal content collection, the candidate list including previously stored unclassified content items. ### Claim 2 (depends on 1) The method of claim 1, further comprising in accordance with a determination that the first aggregate relevance score or the second aggregate relevance score satisfy the threshold score, storing the first unclassified content item in the personal content collection of the user according to a respective relevance rule satisfying the threshold score. ### Claim 3 (depends on 2) The method of claim 2, further comprising notifying the user about the storage of the first unclassified content item in the personal content collection of the user. ### Claim 4 (depends on 3) The method of claim 3, further comprising prompting the user for approval before storing the first unclassified content item in the personal content collection of the user. ### Claim 5 (depends on 2) The method of claim 2, further comprising after storing the first unclassified content item in the personal content collection of the user according to the respective relevance rule satisfying the threshold score, updating the plurality of relevance rules. ### Claim 6 (depends on 5) The method of claim 5, further comprising: after updating the plurality of relevance rules, obtaining, from the candidate list of the personal content collection, a second unclassified content item; determining for the second unclassified content item of the candidate list a plurality of aggregate relevance scores using the plurality of relevance rules; determining whether a third aggregate relevance score of the plurality of aggregate relevance scores satisfies the threshold score; and in accordance with a determination that the third aggregate relevance score of the plurality of aggregate relevance scores satisfies the threshold score, storing the second unclassified content item in the personal content collection of the user according to a relevance rule corresponding to the third aggregate relevance score. ### Claim 7 (depends on 1) The method of claim 1, further comprising in accordance with a determination that the first aggregate relevance score and the second aggregate relevance score satisfy the threshold score, storing the first unclassified content item in the personal content collection of the user according to a respective relevance rule with the highest aggregate relevance score. ### Claim 8 (depends on 1) The method of claim 1, wherein determining the one or more attributes based on the plurality of content items includes detecting text within one or more content items of the plurality of content items. ### Claim 9 (depends on 8) The method of claim 8, wherein detecting text within one or more content items of the plurality of content items includes: recognizing fully or partially the text within the one or more content items of the plurality of content items; generating search indexes based on the fully or partially recognized text; and including the search indexes in the one or more attributes. ### Claim 10 (depends on 9) The method of claim 9, wherein: determining for the first unclassified content item of the one or more unclassified content items a plurality of aggregate relevance scores using the plurality of relevance rules includes detecting text within the first unclassified content item; and wherein the plurality of aggregate relevance scores are further based on the detected text within the first unclassified content item. ### Claim 11 (depends on 10) The method of claim 10, wherein detecting text within the first unclassified content item includes recognizing fully or partially the text within the first unclassified content item. ### Claim 12 (depends on 1) The method of claim 1, wherein plurality of content items include image data. ### Claim 13 (depends on 12) The method of claim 12, wherein: the image data includes text, keywords, user preferences, image content, URLs, location, time, forms, receipts, and handwritten notes; and the one or more attributes are based on the image data. ### Claim 14 (depends on 1) The method of claim 1, further comprising: displaying a user interface on a client device connected to the server, the user interface including: a text entry pane for entering note information or a search query, and a content collection displaying one or more categories or user groupings of content items, the one or more categories or user groupings of content items displaying one or more content items, content item notes, or content attributes. ### Claim 15 A server system, comprising: one or more processors; and memory storing one or more instructions that, when executed by the one or more processors, cause the server system to perform operations including: obtaining a plurality of content items from a personal content collection of a user; determining one or more attributes based on the plurality of content items, wherein the one or more attributes include one or more of content categories, keywords, and user groupings; generating a plurality of relevance rules, wherein each relevance rule is based on a subset of the one or more attributes, and is configured to associate the one or more attributes with an unclassified content item; obtaining, from the user, one or more unclassified content items; determining for a first unclassified content item of the one or more unclassified content items a plurality of aggregate relevance scores using the plurality of relevance rules, the plurality of aggregate relevance scores including: a first aggregate relevance score based on a first relevance rule of the plurality of relevance rules, and a second aggregate relevance score based on a second relevance rule of the plurality of relevance rules; determining whether the first aggregate relevance score and/or the second aggregate relevance score satisfy threshold score; and in accordance with a determination that the first aggregate relevance score and the second aggregate relevance score do not satisfy the threshold score: forgoing determining the one or more attributes corresponding to the first unclassified content item, and storing the first unclassified content item in a candidate list of the personal content collection, the candidate list including previously stored unclassified content items. ### Claim 16 (depends on 15) The server system of claim 15, further comprising instructions that, when executed by the one or more processors, cause the server system to perform operations including in accordance with a determination that the first aggregate relevance score or the second aggregate relevance score satisfy the threshold score, storing the first unclassified content item in the personal content collection of the user according to a respective relevance rule satisfying the threshold score. ### Claim 17 (depends on 15) The server system of claim 15, further comprising instructions that, when executed by the one or more processors, cause the server system to perform operations including in accordance with a determination that the first aggregate relevance score and the second aggregate relevance score satisfy the threshold score, storing the first unclassified content item in the personal content collection of the user according to a respective relevance rule with the highest aggregate relevance score. ### Claim 18 A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for: obtaining a plurality of content items from a personal content collection of a user; determining one or more attributes based on the plurality of content items, wherein the one or more attributes include one or more of content categories, keywords, and user groupings; generating a plurality of relevance rules, wherein each relevance rule is based on a subset of the one or more attributes, and is configured to associate the one or more attributes with an unclassified content item; obtaining, from the user, one or more unclassified content items; determining for a first unclassified content item of the one or more unclassified content items a plurality of aggregate relevance scores using the plurality of relevance rules, the plurality of aggregate relevance scores including: a first aggregate relevance score based on a first relevance rule of the plurality of relevance rules, and a second aggregate relevance score based on a second relevance rule of the plurality of relevance rules; determining whether the first aggregate relevance score and/or the second aggregate relevance score satisfy threshold score; and in accordance with a determination that the first aggregate relevance score and the second aggregate relevance score do not satisfy the threshold score: forgoing determining the one or more attributes corresponding to the first unclassified content item, and storing the first unclassified content item in a candidate list of the personal content collection, the candidate list including previously stored unclassified content items. ### Claim 19 (depends on 18) The non-transitory computer readable storage medium of claim 18, wherein the one or more programs further comprise instructions for in accordance with a determination that the first aggregate relevance score or the second aggregate relevance score satisfy the threshold score, storing the first unclassified content item in the personal content collection of the user according to a respective relevance rule satisfying the threshold score. ### Claim 20 (depends on 18) The non-transitory computer readable storage medium of claim 18, wherein the one or more programs further comprise instructions for in accordance with a determination that the first aggregate relevance score and the second aggregate relevance score satisfy the threshold score, storing the first unclassified content item in the personal content collection of the user according to a respective relevance rule with the highest aggregate relevance score.