# US 2015/0149428 A1 · Search Inquiry Method Using Contextual Annotation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0149428 A1
- Document ID: US20150149428A1
- Status: Published application
- Kind: A1
- Application number: 14/088,352
- Filed: 2013-11-23
- Published: 2015-05-28
- Earliest claimed: 2013-11-23
- Assignees: Sharp Laboratories of America, Inc.
- Applicants: Sharp Laboratories of America, Inc.
- Inventors: Dana S. Smith
- Web page: https://patentgrep.com/patent/US20150149428A1

## CPC

- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 16/53: Querying
- G06F 16/532: Query formulation, e.g. graphical querying
- G06F 16/535: Filtering based on additional data, e.g. user or group profiles
- G06F 16/9032: Query formulation
- G06F 16/24: Querying
- G06F 16/538: Presentation of query results
- G06F 16/951: Indexing; Web crawling techniques
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computer system and associated method are provided for formulating a search inquiry. A user interface (UI) accepts commands for selecting a first data object, and a display presents the first data object with a top plane as located within a geometric boundary representation of a virtual object-bearing front surface. The UI then accepts commands for selecting a first feature associated with the first data object. The UI next accepts either a flip object command or a flip surface command. In response to the flip command, the display presents a first data object bottom plane, opposite of the first data object top plane. The UI is then able to accept commands for selecting a second feature associated with the first data object, and construct a search inquiry in response to the first data object, the first feature, and the second feature, which is sent to a search engine.

## Claims

### Claim 1

A method for formulating a search inquiry using a computer enabled with a sequence of processor executable instructions stored in a non-transitory medium, the method comprising:
a computer user interface (UI) accepting commands for selecting a first data object;
a computer display presenting the first data object with a top plane as located within a geometric boundary representation of a virtual object-bearing front surface;
the computer UI accepting commands for selecting a first feature associated with the first data object;
the computer UI accepting a flip command selected from a group consisting of a flip object command and a flip surface command;
in response to the flip command, the computer display presenting a first data object bottom plane, opposite of the first data object top plane;
the computer UI accepting commands for selecting a second feature associated with the first data object;
the computer constructing a search inquiry in response to the first data object, the first feature, and the second feature; and,
the computer sending the search inquiry to a search engine.

### Claim 2 (depends on 1)

The method of claim 1 wherein the computer display presenting the first data object bottom plane includes presenting a reversed first data object top plane, representing a two-dimensional first data top plane image as reversed from left to right.

### Claim 3 (depends on 1)

The method of claim 1 wherein accepting the flip command includes accepting a flip surface command;
wherein presenting the first data object bottom plane includes presenting a geometric boundary representation of a virtual object-bearing transparent back surface, opposite of the front surface, through which the first data object bottom plane is visible; and,
wherein accepting commands for selecting the second feature includes associating the second feature with the first data object bottom plane.

### Claim 4 (depends on 1)

The method of claim 1 wherein accepting the flip command includes accepting a flip object command;
wherein presenting the first data object bottom plane includes presenting the first data object bottom plane overlying the front surface; and,
wherein accepting commands for selecting the second feature includes associating the second feature with the first data object bottom plane.

### Claim 5 (depends on 4)

The method of claim 4 further comprising:
the computer UI accepting an unflip object command;
a computer display presenting the first data object top plane as located on the front surface;
the computer UI accepting a flip surface command;
the computer display presenting a geometric boundary representation of a virtual object-bearing transparent hack surface, opposite of the front surface, through which the first data object bottom plane is visible; and,
the computer UI accepting commands for selecting a third feature associated with the first data object bottom surface.

### Claim 6 (depends on 1)

The method of claim 1 further comprising:
the computer UI accepting a shift command for relocating the first data object on the front surface;
in response to the shift command, the computer display presenting a geometric boundary representation of a virtual object-bearing occluded surface underlying the first data object;
the computer UI accepting commands for selecting a third feature, associated with the first data object, to the occluded surface; and,
wherein the computer constructing the search inquiry includes constructing the search inquiry in response to the first data object, first feature, second feature, and third feature.

### Claim 7 (depends on 1)

The method of claim 1 wherein the computer constructing the search inquiry includes the computer differentially weighting the first data object, first feature, and second feature.

### Claim 8 (depends on 1)

The method of claim 1 wherein the computer UI accepting commands for selecting the first data object includes the first data object being selected from a group consisting of preexisting still images, preexisting video, preexisting text documents, preexisting sounds, user-added sounds, user-added images, user-added video, user-added drawings, user-added text, and combinations of the above-listed elements.

### Claim 9 (depends on 1)

The method of claim 1 further comprising:
the computer display presenting a plurality of results from the search engine in response to the search inquiry;
the computer accepting commands for selecting a first result, from the plurality of results, as a new first data object; and,
the computer UI accepting commands for selecting a new first feature associated with the new first data object.

### Claim 10 (depends on 1)

The method of claim 1 wherein the computer UI accepting commands for selecting the first feature associated with the first data object includes accepting commands for a plurality of selected first features.

### Claim 11 (depends on 1)

The method of claim 1 wherein the computer constructing the search inquiry includes weighting the first data object, first feature, and second feature based upon factors selected from a group consisting of the surface being presented when the search inquiry is initiated, the object plane being presented when the search inquiry is initiated, and a spatial analysis of first and second feature locations with respect to the first data object location.

### Claim 12 (depends on 1)

The method of claim 1 wherein the computer UI accepting commands for selecting the first feature includes selecting the first feature from a group consisting of accentuating a preexisting aspect of the first data object, adding video, adding text, image-associated sounds, adding sounds, adding drawings, and editing a previously selected feature.

### Claim 13 (depends on 1)

The method of claim 1 further comprising:
the computer display presenting a results surface with a plurality of result objects received from the search engine in response to the search inquiry, where the relevance of a first result object is related to spatial location on the results surface and responsive to the spatial location of the first feature with respect to the first data object, and the relevance of a second result object is related to spatial location on the results surface and responsive to the spatial location of the second feature with respect to the first data object.

### Claim 14 (depends on 1)

The method of claim 1 wherein accepting commands for selecting the first data object includes accepting commands for selecting a plurality of first data objects.

### Claim 15

A search inquiry natural user interface (NUI) enabled by a computer executing a sequence of processor instructions stored in a non-transitory medium, the NUI comprising:
a front page menu presenting a first data object with a top plane as located within a geometrical boundary representation of a virtual object-bearing front surface, the front page menu providing user interface (UI) mechanisms for:
selecting the first data object;
selecting a first feature associated with the first data To object;
initiating a search inquiry;
accepting a flip command;
a flip page menu, selected from a group consisting of a flip object menu and a flip surface menu, presented in response to the flip command, the flip page menu presenting a first data object bottom plane, opposite to the first data object top plane, and providing UI mechanisms for:
selecting a second feature associated with the first data object;
initiating a search inquiry; and,
accepting an unflip command for returning to the front page menu.

### Claim 16 (depends on 15)

The NUI of claim 15 wherein the first data object bottom plane is presented as a reversed first data object top plane, representing a two-dimensional first data top plane image as reversed from left to right.

### Claim 17 (depends on 15)

The NUI of claim 15 wherein the front page menu flip command UI mechanism accepts a flip surface command;
wherein the flip surface menu presents a geometric boundary representation of a virtual object-bearing transparent hack surface, opposite of the front surface, through which the first data object bottom plane is visible, in response to the flip surface command; and,
wherein the flip surface menu UI mechanism accepts commands for associating the second feature with the first data object bottom plane.

### Claim 18 (depends on 15)

The NUI of claim 15 wherein the front page menu flip command UI mechanism accepts a flip object command;
wherein the flip object menu presents the first data object bottom plane overlying the front surface in response to the flip object command; and
wherein the flip object menu UI mechanism accepts commands for associating the second feature with the first data object bottom plane.

### Claim 19 (depends on 18)

The NUI of claim 18 wherein the front page menu presents the first data object top plane as located on the front surface in response to an unflip command accepted by the flip object menu;
wherein a flip surface menu presents a geometric boundary representation of a virtual object-bearing transparent back surface, opposite of the front surface, through which the first data object bottom plane is visible, in response to a flip surface command accepted by the front page menu; and,
wherein the flip surface menu further comprises a UI mechanism for selecting a third feature associated with the first data object bottom surface.

### Claim 20 (depends on 15)

The NUI of claim 15 wherein the front page menu further comprises a UI mechanism for accepting a shift command to relocate the first data object on the front surface and present a geometric boundary representation of a virtual object-bearing occluded surface underlying the first data object;
the NUI further comprising:
a shift object menu comprises a UI mechanism for accepting commands to select a third feature, associated with the first data object, to the occluded surface; and,
wherein a shift object menu search inquiry UI mechanism initiates a search inquiry in response to the first data object, first feature, second feature, and third feature.

### Claim 21 (depends on 15)

The NUI of claim 15 wherein the front page and flip page menu search inquiry UI mechanisms initiate search inquiries by differentially weighting the first data object, first feature, and second feature.

### Claim 22 (depends on 15)

The NUI of claim 15 wherein the front page menu first object selection UI mechanisms accept commands for selecting the first data object from a group consisting of preexisting still images, preexisting video, preexisting text documents, preexisting sounds, user-added sounds, user-added images, user-added video, user-added drawings, user-added text, and combinations of the above-listed elements.

### Claim 23 (depends on 15)

The NUI of claim 15 further comprising:
a results menu presenting a plurality of result objects received in response to the search inquiry, and providing UI mechanisms for:
selecting a first result object, from the plurality of result objects, as a new first data object; and,
selecting a new first feature associated with the new first data object.

### Claim 24 (depends on 15)

The NUI of claim 15 wherein the front page menu first feature selection UI mechanism accepts commands for selecting a plurality of first features associated with the first data object.

### Claim 25 (depends on 15)

The NUI of claim 15 wherein the front page and flip menu search inquiry UI mechanisms initiate search inquiries by weighting the first data object, first feature, and second feature based upon factors selected from a group consisting of the surface being presented when the search inquiry is initiated, the object plane being presented when the search inquiry is initiated, and a spatial analysis of first and second feature locations with respect to the first data object location.

### Claim 26 (depends on 15)

The NUI of claim 15 wherein the front page menu first feature selection UI mechanism accepts commands for selecting the first feature from a group consisting of accentuating a preexisting aspect of the first data object, adding video, adding text, image-associated sounds, adding sounds, adding drawings, and editing a previously selected feature.

### Claim 27 (depends on 15)

The NUI of claim 15 further comprising:
a results menu for presenting a results surface with a plurality of result objects received in response to the search inquiry, where the relevance of a first result object is related to spatial location on the results surface and responsive to the spatial location of the first feature with respect to the first data object, and the relevance of a second result object is related to spatial location on the results surface and responsive to the spatial location of the second feature with respect to the first data object.

### Claim 28 (depends on 15)

The NUI of claim 15 wherein the front page menu first object selection UI mechanism accepts commands for selecting a plurality of first data objects.

### Claim 29

A non-transitory computer-readable media for formulating a search inquiry comprising computer-readable instructions recorded thereon for:
accepting user interface (UI) commands for selecting a first data object;
presenting the first data object with a top plane as located within a geometric boundary representation of a virtual object-bearing front surface;
accepting UI commands for selecting a first feature associated with the first data object;
accepting a UI flip command selected from a group consisting of a flip object command and a flip surface command;
in response to the flip command, presenting a first data object bottom plane, opposite of the first data object top plane;
accepting UI commands for selecting a second feature associated with the first data object;
constructing a search inquiry in response to the first data object, the first feature, and the second feature; and,
sending the search inquiry to a search engine.

### Claim 30

An electronic device for formulating a search inquiry comprising:
a processor;
a search inquiry application enabled with a sequence of processor executable instructions stored in a non-transitory medium;
a user interface (UI) accepting UI commands;
a display;
wherein the search inquiry application accepts UI commands for selecting a first data object, and in response, sends commands to the display for presenting the first data object with a top plane as located within a geometric boundary representation of a virtual object-bearing front surface;
wherein the search inquiry application accepts UI commands for selecting a first feature associated with the first data object, and a UI flip command selected from a group consisting of a flip object command and a flip surface command, and in response, sends commands to the display for presenting a first data object bottom plane, opposite of the first data object top plane; and,
wherein the search inquiry application accepts UI commands for selecting a second feature associated with the first data object, constructs a search inquiry in response to the first data object, the first feature, and the second feature, and sends the search inquiry to a search engine.
