Back to results
US 11,386,151 B2Granted
Image search in walkthrough videos
AssigneeOpen Space Labs, Inc.
InventorsMichael Ben Fleischman, Gabriel Hein, Thomas Friel Allen, Philip DeCamp
Life of this patent
Estimating the term…
Abstract
A spatial indexing system receives a set of walkthrough videos of an environment taken over a period of time and receives an image search query that includes an image of an object. The spatial indexing system searches the set of walkthrough videos for instances of the object. The spatial indexing system presents search results in a user interface, displaying in a first portion a 2D map associated with one walkthrough video with marked locations of instances of the object and a second portion with a histogram of instances of the object over time in the set of walkthrough videos.
Claims20
A method comprising:
- receiving an image search query for a walkthrough video of an environment;
- determining a set of feature vectors corresponding to the image search query;
- searching the walkthrough video to identify portions of frames of the walkthrough video that correspond to the set of feature vectors;
- refining the set of feature vectors based on the identified portions of frames of the walkthrough video;
- receiving a request to search a set of walkthrough videos of the environment using the image search query, each walkthrough video associated with a timestamp;
- searching the set of walkthrough videos to identify additional portions of frames within the set of walkthrough videos corresponding to the set of feature vectors;
- and
- generating an interface for display comprising:
- a first interface portion including a representation of a floorplan of the environment with marked locations within the representation of the floorplan corresponding to the identified portions of frames of the walkthrough video, and
- a second interface portion including a histogram, the independent axis of the histogram comprising a timeline corresponding to the timestamps of the set of walkthrough videos, the histogram including bars indicating instances of the identified additional portions of frames within the set of walkthrough videos.
7 more elements
A system comprising:
- a hardware processor;
- and
- a non-transitory computer-readable storage medium storing executable instructions that, when executed by the hardware processor, cause the hardware processor to perform steps comprising:
- receiving an image search query for a walkthrough video of an environment;
- determining a set of feature vectors corresponding to the image search query;
- searching the walkthrough video to identify portions of frames of the walkthrough video that correspond to the set of feature vectors;
- refining the set of feature vectors based on the identified portions of frames of the walkthrough video;
- receiving a request to search a set of walkthrough videos of the environment using the image search query, each walkthrough video associated with a timestamp;
- searching the set of walkthrough videos to identify additional portions of frames within the set of walkthrough videos corresponding to the set of feature vectors;
- and
- generating an interface for display comprising:
- a first interface portion including a representation of a floorplan of the environment with marked locations within the representation of the floorplan corresponding to the identified portions of frames of the walkthrough video, and
- a second interface portion including a histogram, the independent axis of the histogram comprising a timeline corresponding to the timestamps of the set of walkthrough videos, the histogram including bars indicating instances of the identified additional portions of frames within the set of walkthrough videos.
10 more elements
A non-transitory computer-readable storage medium storing executable instructions that, when executed by a hardware processor, cause the hardware processor to perform steps comprising:
- receiving an image search query for a walkthrough video of an environment;
- determining a set of feature vectors corresponding to the image search query;
- searching the walkthrough video to identify portions of frames of the walkthrough video that correspond to the set of feature vectors;
- refining the set of feature vectors based on the identified portions of frames of the walkthrough video;
- receiving a request to search a set of walkthrough videos of the environment using the image search query, each walkthrough video associated with a timestamp;
- searching the set of walkthrough videos to identify additional portions of frames within the set of walkthrough videos corresponding to the set of feature vectors;
- and
- generating an interface for display comprising:
- a first interface portion including a representation of a floorplan of the environment with marked locations within the representation of the floorplan corresponding to the identified portions of frames of the walkthrough video, and
- a second interface portion including a histogram, the independent axis of the histogram comprising a timeline corresponding to the timestamps of the set of walkthrough videos, the histogram including bars indicating instances of the identified additional portions of frames within the set of walkthrough videos.
7 more elements
Dependencies are inferred from the claim text.