# US 8,873,857 B2 · Mobile image search and indexing system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,873,857 B2
- Document ID: US8873857B2
- Status: Granted
- Kind: B2
- Application number: 13/936,726
- Filed: 2013-07-08
- Published: 2013-11-07
- Granted: 2014-10-28
- Earliest claimed: 2008-12-22
- Assignees: Ipointer, Inc.
- Applicants: iPointer, Inc.
- Inventors: Christopher Edward Frank; David Caduff
- Web page: https://patentgrep.com/patent/US8873857B2

## CPC

- H04W 4/02: Services making use of location information
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 16/70: of video data
- H04W 4/026: {using orientation information, e.g. compass}
- H04W 4/185: {by embedding added-value information into content, e.g. geo-tagging}
- H04L 67/52: specially adapted for the location of the user terminal
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A computer-implemented system and method are described for image searching and image indexing that may be incorporated in a mobile device that is part of an object identification system. A computer-implemented system and method relating to a MISIS client and MISIS server that may be associated with mobile pointing and identification system for the searching and indexing of objects in in situ images in geographic space taken from the perspective of a system user located near the surface of the Earth including horizontal, oblique, and airborne perspectives.

## Claims

### Claim 1

A computer-implemented method for image searching and image indexing for identifying image content in at least two digital images representative of a real world visual scene, comprising the steps of
(A) determining for a first mobile pointing device a mobile device geographic location, pointing direction, and a viewing angle;
(B) determining for a second mobile pointing device a mobile device geographic location, pointing direction, and a viewing angle;
(C) retrieving from a first storage location a first digital image that relates to the real world visual scene that is associated with the first mobile pointing device geographic location;
(D) retrieving from the first storage location a second digital image that relates to the real world visual scene that is associated with the second mobile pointing device geographic location;
(E) generating a first viewing content cone that includes the first mobile pointing device pointing direction, with the first mobile pointing device geographic location being the index of the first viewing content cone, and overlaying the first digital image with the first viewing content cone;
(F) determining potential objects of the first digital image that are within limits of the first viewing content cone;
(G) generating a second viewing content cone that includes the second mobile pointing device pointing direction, with the second mobile pointing device geographic location being the index of the second viewing content cone, and overlaying the second digital image with the second viewing content cone;
(H) determining potential objects of the second digital image that are within limits of the second viewing content cone;
(I) retrieving object-based datasets from a second storage location, with the object-based datasets including at least information relating to potential objects within the first viewing content cone determined at Step (F);
(J) retrieving object-based datasets from a second storage location, with the object-based datasets including at least information relating to potential objects within the second viewing content cone determined at Step (H);
(K) determining each object within the limits of the first viewing content cone at Step (F) by linking object-based dataset information relating to that object retrieved at Step (I) with each such object by indexing with an indexing mechanism each object potentially contained within the first viewing content cone and assessing at least thematic information associated with each such object;
(L) determining each object within the limits of the second viewing content cone at Step (H) by linking object-based dataset information relating to that object retrieved at Step (J) with each such object by indexing with an indexing mechanism each object potentially contained within the second viewing content cone and assessing at least thematic information associated with each such object;
(M) electronically annotating the first digital image with an image position and directional orientation based on the determination at Step (A) relating to the first mobile pointing device, and with respect to at least each object determined to be within the first viewing content cone related annotating thematic information, and electronically annotating the second digital image with an image position and directional orientation based on the determination at Step (B), and with respect to at least each object determined to be within the second viewing content cone annotating thematic information; and
(N) identifying the objects that are separately within the first viewing content cone and the second viewing content cone, objects that are within both the first and second viewing content cones, and objects that are not within the first or the second viewing content cone.

### Claim 2 (depends on 1)

The method as recited in claim 1, wherein the method further includes the step of:
(O) updating the indexing mechanism at Step (K) with information about available images for objects determined to be within the first viewing content cone.

### Claim 3 (depends on 1)

The method recited in claim 1, wherein the method further includes the step of:
(P) updating the indexing mechanism at Step (L) with information about available images for objects determined to be within the second viewing content cone.

### Claim 4 (depends on 1)

The method as recited in claim 1, wherein the first mobile pointing device geographic location includes being determined by a global positioning system (GPS).

### Claim 5 (depends on 1)

The method as recited in claim 1, wherein the second mobile pointing device geographic location includes being determined by a global positioning system (GPS).

### Claim 6 (depends on 1)

The method as recited in claim 1, wherein the first mobile pointing device pointing direction includes being determined by a digital compass.

### Claim 7 (depends on 1)

The method as recited in claim 1, wherein the second mobile pointing device pointing direction includes being determined by a digital compass.

### Claim 8 (depends on 1)

The method as recited in claim 1, wherein the first mobile pointing device viewing angle includes being determined by an optical system associated with the mobile pointing device.

### Claim 9 (depends on 1)

The method as recited in claim 1, wherein the second mobile pointing device viewing angle includes being determined by an optical system associated with the mobile pointing device.

### Claim 10 (depends on 1)

The method as recited in claim 1, wherein indexing with an indexing mechanism includes indexing using a Binary Space Partitioning (BSP) Tree, Octree, R-Tree, or KD-Tree (k-dimensional).

### Claim 11 (depends on 1)

The method as recited in claim 1, wherein the method further includes determining for each object within the limits of the first or second viewing content cone a numeric value determination for the completeness of the object's representation within the first or second digital image.

### Claim 12 (depends on 11)

The method as recited in claim 11, wherein the numeric value determination includes nearness of the object to the index of the first or second viewing content cone, a location of the object with respect to the pointing direction determined at Step (A) or Step (B), or a percentage of the object that is within the limits of the first or second viewing content cone, respectively.