# US 2018/0025032 A1 · Image storage and retrieval based on eye movements

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0025032 A1
- Document ID: US20180025032A1
- Status: Published application
- Kind: A1
- Application number: 15/640,773
- Filed: 2017-07-03
- Published: 2018-01-25
- Earliest claimed: 2012-09-28
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Raanan Yehezkel; Michal Jacob
- Web page: https://patentgrep.com/patent/US20180025032A1

## CPC

- G06F 16/54: Browsing; Visualisation therefor
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 16/50: of still image data
- G06F 16/58: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Various embodiments are generally directed to creating and using an index based on eye movements of the human eye to store and retrieve images in an image database. An apparatus comprises a processor circuit and a storage communicatively coupled to the processor circuit and storing instructions operative on the processor circuit to receive a first eye movement data associated with a first image provided by the apparatus from an image database stored in the storage; determine a first identity of a first object at a first focus region in the first image indicated by the first eye movement data; search the image database for an image depicting the first object; and provide a second image depicting the first object from the image database. Other embodiments are described and claimed herein.

## Claims

### Claim 1

An apparatus comprising:
a processor circuit; and
a storage communicatively coupled to the processor circuit and arranged to store instructions operative on the processor circuit to:
receive an image data comprising a first image;
store the first image in an image database;
receive a first eye movement data associated with the first image;
determine a first identity of a first object at a first focus region in the first image indicated by the first eye movement data; and
augment the image database with an indication of the first identity of the first object at the first focus region.

### Claim 2 (depends on 1)

The apparatus of claim 1, the instructions operative on the processor circuit to:
determine a second identity of a second object at a second focus region in the first image indicated by the first eye movement data;
augment the image database with an indication of the second identity of the second object at the second focus region;
derive a possible relationship between the first and second objects from a plurality of saccades specified by the first eye movement data as extending between the first and second focus regions; and
augment the image database with an indication of the possible relationship between the first and second objects.

### Claim 3 (depends on 2)

The apparatus of claim 2, the image database comprising an index, and the instructions operative on the processor circuit to augment the image database.

### Claim 4 (depends on 1)

The apparatus of claim 1, comprising an interface operative to communicatively couple the apparatus to a network, and the instructions operative on the processor circuit to receive the image data from a computing device via the network.

### Claim 5 (depends on 1)

The apparatus of claim 1, the instructions operative on the processor circuit to:
receive tag data associated with the first image; and
augment the image database with the tag data.

### Claim 6 (depends on 5)

The apparatus of claim 5, the tag data comprising one of an indication of a date on which the first image was captured, a time of day at which the first image was captured, a place at which the first image was captured, or an image category of the first image is classified.

### Claim 7 (depends on 6)

The apparatus of claim 6, the instructions operative on the processor circuit to:
receive a search criterion, the search criterion comprising one of a defined date of image capture, a defined time of day of image capture, a defined place of image capture, or a defined image category;
search the image database for an image that meets the criterion; and
provide the first image in response to a match between the search criterion and the tag data associated with the first image.

### Claim 8 (depends on 7)

The apparatus of claim 7, the instructions operative on the processor circuit to:
receive a second eye movement data associated with the first image;
determine a second identity of a second object at a second focus region in the first image indicated by the second eye movement data;
search the image database for an image that depicts the second object; and
provide a second image that depicts the second object from the image database.

### Claim 9 (depends on 8)

The apparatus of claim 8, the instructions operative on the processor circuit to:
analyze the first image to identify the image category of the first image; and
augment the image database with the image category of the first image.

### Claim 10 (depends on 5)

The apparatus of claim 5, the tag data comprising a text description of the first object as depicted in the first image, and the instructions operative on the processor circuit to:
receive a search text that describes a search object;
search the image database for an image that depicts the search object; and
provide the first image in response to the tag data that indicates the search object as depicted in the first image.

### Claim 11 (depends on 10)

The apparatus of claim 10, the instructions operative on the processor circuit to:
receive a second eye movement data associated with the first image;
determine a second identity of a second object at a second focus region in the first image indicated by the second eye movement data;
search the image database for an indication of an image that depicts the second object; and
provide a second image that depicts the second object from the image database.

### Claim 12 (depends on 11)

The apparatus of claim 11, comprising a display, and the instructions operative on the processor circuit to visually present the second image on the display.

### Claim 13 (depends on 1)

The apparatus of claim 1, the instructions operative on the processor circuit to:
augment the image database with the eye movement data, the eye movement data that indicates a pattern of eye movement from a viewing of the first image; and
search the image database for an image indicated as comprising a similar pattern of eye movement.

### Claim 14

An apparatus comprising:
a processor circuit; and
a storage communicatively coupled to the processor circuit and arranged to store instructions operative on the processor circuit to:
receive a first eye movement data associated with a first image from an image database stored in the storage;
determine a first identity of a first object at a first focus region in the first image indicated by the first eye movement data;
search the image database for an image that depicts the first object; and
provide a second image that depicts the first object from the image database.

### Claim 15 (depends on 14)

The apparatus of claim 14, the instructions operative on the processor circuit to:
determine that the image database comprises no indication of the first focus region; and
augment the image database with an indication of the first identity of the first object at the first focus region.

### Claim 16 (depends on 14)

The apparatus of claim 14, comprising a display, and the instructions operative on the processor circuit to visually present the first and second images on the display.

### Claim 17 (depends on 14)

The apparatus of claim 14, comprising an interface operative to communicatively couple the apparatus to a network, and the instructions operative on the processor circuit to receive the first eye movement data from a capture device via the network.

### Claim 18 (depends on 14)

The apparatus of claim 14, the instructions operative on the processor circuit to:
receive an image data comprising the first image;
store the first image in the image database;
determine the first identity of the first object at a focus region in the first image indicated by an eye movement data of the image data; and
augment the image database with an indication of the first identity of the first object.

### Claim 19 (depends on 18)

The apparatus of claim 18, the instructions operative on the processor circuit to:
determine a second identity of a second object at a second focus region in the first image indicated by the eye movement data of the image data;
augment the image database with an indication of the second identity of the second object;
derive a possible relationship between the first and second objects from a plurality of saccades specified by the eye movement data of the image data; and
augment the image database with an indication of the possible relationship between the first and second objects.

### Claim 20 (depends on 19)

The apparatus of claim 19, the instructions operative on the processor circuit to:
search the image database for an image that depicts the second object in response to the indication of the possible relationship between the first and second objects; and
provide a third image that depicts the second object from the image database along with the second image.

### Claim 21 (depends on 20)

The apparatus of claim 20, comprising a display, and the instructions operative on the processor circuit to visually present the third image on the display.

### Claim 22

A computer-implemented method comprising:
receiving an image data comprising a first image;
storing the first image in an image database;
receiving tag data associated with the first image;
augmenting the image database with the tag data;
receiving a first eye movement data associated with the first image;
determining a first identity of a first object at a first focus region in the first image indicated by the first eye movement data; and
augmenting the image database with an indication of the first identity of the first object at the first focus region.

### Claim 23

The computer-implemented method of 22, comprising:
determining a second identity of a second object at a second focus region in the first image indicated by the first eye movement data;
augmenting the image database with an indication of the second identity of the second object at the second focus region;
deriving a possible relationship between the first and second objects from a plurality of saccades specified by the first eye movement data as extending between the first and second focus regions; and
augmenting the image database with an indication of the possible relationship between the first and second objects.

### Claim 24 (depends on 23)

The computer-implemented method of claim 23, the image database comprising an index, and augmenting the image database comprises augmenting the index.

### Claim 25

The computer-implemented method of 22, the tag data comprising one of an indication of a date on which the first image was captured, a time of day at which the first image was captured, and a place at which the first image was captured.

### Claim 26

The computer-implemented method of 25, comprising:
receiving a search criterion, the search criterion comprising one of a defined date of image capture, a defined time of day of image capture, and a defined place of image capture;
searching the image database for an image meeting the criterion; and
providing the first image in response to the search criterion matching the tag data associated with the first image.

### Claim 27

The computer-implemented method of 26, comprising:
receiving a second eye movement data associated with the first image;
determining a second identity of a second object at a second focus region in the first image indicated by the second eye movement data;
searching the image database for an image depicting the second object; and
providing a second image depicting the second object from the image database.

### Claim 28

The computer-implemented method of 22, the tag data comprising a text description of the first object as depicted in the first image, and the method comprising:
receiving a search text describing a search object;
searching the image database for an image depicting the search object; and
providing the first image in response to the tag data indicating the search object as depicted in the first image.

### Claim 29

The computer-implemented method of 28, comprising:
receiving a second eye movement data associated with the first image;
determining a second identity of a second object at a second focus region in the first image indicated by the second eye movement data;
searching the image database for an image depicting the second object; and
providing a second image depicting the second object from the image database.

### Claim 30

At least one machine-readable storage medium comprising instructions that when executed by a computing device, cause the computing device to:
receive a first eye movement data associated with a first image provided by the computing device from an image database stored in a storage of the computing device;
determine a first identity of a first object at a first focus region in the first image indicated by the first eye movement data;
search an image database of the image database for an indication of an image depicting the first object; and
provide a second image depicting the first object from the image database.

### Claim 31 (depends on 30)

The at least one machine-readable storage medium of claim 30, the computing device caused to:
determine that the image database comprises no indication of the first focus region; and
augment the image database with an indication of the first identity of the first object at the first focus region.

### Claim 32 (depends on 30)

The at least one machine-readable storage medium of claim 30, the computing device caused to:
receive an image data comprising the first image;
store the first image in the image database;
determine the first identity of the first object at a focus region in the first image indicated by an eye movement data of the image data; and
augment the image database with an indication of the first identity of the first object.

### Claim 33 (depends on 32)

The at least one machine-readable storage medium of claim 32, the computing device caused to:
determine a second identity of a second object at a second focus region in the first image indicated by the eye movement data of the image data;
augment the image database with an indication of the second identity of the second object;
derive a possible relationship between the first and second objects from a plurality of saccades specified by the eye movement data of the image data; and
augment the image database with an indication of the possible relationship between the first and second objects.

### Claim 34 (depends on 33)

The at least one machine-readable storage medium of claim 33, the computing device caused to:
search the image database for an image depicting the second object in response to the indication of the possible relationship between the first and second objects; and
provide a third image depicting the second object from the image database along with the second image.