# US 2018/0025003 A1 · Automatic Detection and Transfer of Relevant Image Data to Content Collections

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0025003 A1
- Document ID: US20180025003A1
- Status: Published application
- Kind: A1
- Application number: 15/658,361
- Filed: 2017-07-24
- Published: 2018-01-25
- Earliest claimed: 2016-07-25
- Assignees: Evernote Corporation
- Applicants: Evernote Corporation
- Inventors: Dylan Marriott; Daniel Nicolae; Ruben Bakker; Alexander Pashintsev; Zdzislaw Pawel Losvik; Eugene Livshitz; Vitaly Glazkov; Boris Gorbatov; Ilya Buryak
- Web page: https://patentgrep.com/patent/US20180025003A1

## CPC

- G06F 16/51: Indexing; Data structures therefor; Storage structures
- G06F 16/24565: {Triggers; Constraints}
- G06F 16/24578: {using ranking}
- G06F 16/40: of multimedia data, e.g. slideshows comprising image and additional audio data (retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70)
- G06F 16/434: {using image data, e.g. images, photos, pictures taken by a user}
- G06F 16/55: Clustering; Classification
- G06F 16/5846: {using extracted text}
- G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information}
- G06Q 30/02: Marketing; Price estimation or determination; Fundraising
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

This application is directed to a method for automatically identifying and transferring relevant image data implemented at a server with a processor and memory including one or programs executable by the processor. The server obtains content items from a personal content collection of a user, analyzes the content items to identify attributes associated with the images, generates rule sets based on the identified attributes, applies the generated rule sets to a plurality of unclassified images, identifies a first set of one or more of the unclassified images, associates each image of the first set with the content items and stores the first set in the personal content collection. A second set of unclassified images is placed in a candidate list to be periodically, without user intervention, monitored and updated for storing in the personal content collection. The generated rule sets are updated after images are stored in the content collection.

## Claims

### Claim 1

A method comprising:
at a server having one or more processors and memory storing programs executed by the one or more processors,
obtaining content items from a personal content collection of a user, wherein the content items includes images and associated image information;
analyzing the content items to identify attributes associated with the images, wherein the attributes include one or more of image data, image categories, and image groupings;
generating rule sets based on the identified attributes to be used in determining one or more of a category, grouping, and image relevance scores specific to the user from images;
applying the generated rule sets to a plurality of unclassified images to determine for each of the unclassified images one or more of a category, grouping, and image relevance scores;
identifying a first set of one or more of the unclassified images with determined relevance scores satisfying a predetermined relevance criterion;
associating each image of the first set with the content items in accordance with the one or more determined category and grouping; and
storing the first set in the personal content collection.

### Claim 2 (depends on 1)

The method of claim 1, further comprising notifying the user about the storage of the first set.

### Claim 3 (depends on 2)

The method of claim 2, further comprising prompting the user for approval before storing the first set.

### Claim 4 (depends on 1)

The method of claim 1, further comprising analyzing the content items to detect text within the images.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
recognizing fully or partially the text within the respective images;
generating image search indexes with the fully or partially recognized text; and
including the search indexes in the generated rule sets.

### Claim 6 (depends on 5)

The method of claim 5, further comprising analyzing the plurality of unclassified images to detect text within the unclassified images.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
recognizing fully or partially the text within the unclassified images;
applying the generated rule sets to the plurality of unclassified images to determine for each of the plurality of unclassified images one or more of a category, grouping, and image relevance scores;

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
identifying a second set of one or more of the unclassified images with determined image relevance scores below the predetermined relevance criterion; and
placing the second set in a candidate list.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
periodically without user intervention monitoring the second set in the candidate list;
periodically without user intervention updating respective image relevance scores of the second set;
identifying a third set of one or more of the second set with determined relevance scores satisfying the predetermined relevance criterion;
associating each image of the third set with the content items in accordance with the one or more determined category and grouping; and
storing the third set in the personal content collection.

### Claim 10 (depends on 1)

The method of claim 1, further comprising updating the generated rule sets after associating each image of the first set with the content items and storing the first set.

### Claim 11 (depends on 9)

The method of claim 9, further comprising updating the generated rule sets after associating each image of the third set with the content items and storing the third set.

### Claim 12 (depends on 1)

The method of claim 1, wherein the associated image information includes text, keywords, user preferences, image content, URLs, location, time, forms, receipts, and handwritten notes.

### Claim 13 (depends on 1)

The method of claim 1, further comprising displaying a user interface on a client device connected to the server, the user interface including a text entry pane for entering note information or a search query, a content collection displaying one or more category or grouping of images, the one or more category or grouping of images displaying one or more of content items, content item notes, or content attributes.

### Claim 14

A server system, comprising:
one or more processors; and
memory storing one or more instructions that, when executed by the one or more processors, cause the computing system to perform operations including:
obtaining content items from a personal content collection of a user, wherein the content items includes images and associated image information;
analyzing the content items to identify attributes associated with the images, wherein the attributes include one or more of image data, image categories, and image groupings;
generating rule sets based on the identified attributes to be used in determining one or more of a category, grouping, and image relevance scores specific to the user;
applying the generated rule sets to a plurality of unclassified images to determine for each of the unclassified images one or more of a category, grouping, and image relevance scores;
identifying a first set of one or more of the unclassified images with determined relevance scores satisfying a predetermined relevance criterion;
associating each image of the first set with the content items in accordance with the one or more determined category and grouping; and
storing the first set in the personal content collection.

### Claim 15 (depends on 14)

The server system of claim 14, wherein the operations further comprise analyzing the content items to detect text within the images.

### Claim 16 (depends on 15)

The server system of claim 15, wherein the operations further comprise:
recognizing fully or partially the text within the respective images;
generating image search indexes with the fully or partially recognized text; and
including the search indexes in the generated rule sets.

### Claim 17 (depends on 14)

The server system of claim 14, wherein the operations further comprise:
identifying a second set of one or more of the unclassified images with determined image relevance scores below the predetermined relevance criterion; and
placing the second set in a candidate list.

### Claim 18 (depends on 17)

The server system of claim 17, wherein the operations further comprise:
periodically without user intervention monitoring the second set in the candidate list;
periodically without user intervention updating respective image relevance scores of the second set;
identifying a third set of one or more of the second set with determined relevance scores satisfying the predetermined relevance criterion;
associating each image of the third set with the content items in accordance with the one or more determined category and grouping; and
storing the third set in the personal content collection.

### Claim 19 (depends on 14)

The server system of claim 14, wherein the operations further comprise displaying a user interface on a client device connected to the server, the user interface including a text entry pane for entering note information or a search query, a content collection displaying one or more category or grouping of images, the one or more category or grouping of images displaying one or more of content items, content item notes, or content attributes.

### Claim 20

A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:
obtaining content items from a personal content collection of a user, wherein the content items includes images and associated image information;
analyzing the content items to identify attributes associated with the images, wherein the attributes include one or more of image data, image categories, and image groupings;
generating rule sets based on the identified attributes to be used in determining one or more of a category, grouping, and image relevance scores specific to the user;
applying the generated rule sets to a plurality of unclassified images to determine for each of the unclassified images one or more of a category, grouping, and image relevance scores;
identifying a first set of one or more of the unclassified images with determined relevance scores satisfying a predetermined relevance criterion;
associating each image of the first set with the content items in accordance with the one or more determined category and grouping; and
storing the first set in the personal content collection.