Back to results
US 12,675,524 B2Granted
Image search in walkthrough videos
AssigneeOpen Space Labs, Inc.
InventorsMichael Ben Fleischman, Gabriel Hein, Thomas Friel Allen, Philip DeCamp
Life of this patent
Estimating the term…
Abstract
A spatial indexing system receives a set of walkthrough videos of an environment taken over a period of time and receives an image search query that includes an image of an object. The spatial indexing system searches the set of walkthrough videos for instances of the object. The spatial indexing system presents search results in a user interface, displaying in a first portion a 2D map associated with one walkthrough video with marked locations of instances of the object and a second portion with a histogram of instances of the object over time in the set of walkthrough videos.
Claims15
A method comprising:
- generating a set of image sub-frames by 1) performing a cropping operation on each of a set of walkthrough videos of an environment to produce overlapping portions of frames of the walkthrough videos, and 2) performing a de-duplication operation to remove overlapping portions of frames that correspond to a same image search query, each of the set of image sub-frames corresponding to one of the walkthrough videos and associated with a timestamp;
- generating an interface for display including a representation of a floorplan of the environment corresponding to the set of walkthrough videos and a camera path corresponding to at least one of the set of walkthrough videos, and including a histogram, the independent axis of the histogram comprising a timeline corresponding to the timestamps associated with the generated set of image sub-frames, the histogram including bars indicating both 1) instances of the generated set of image sub-frames within the set of walkthrough videos, and 2) a number of search results corresponding to the image search query;
- and
- in response to a histogram bar being selected, populating a portion of the interface with walkthrough video search results corresponding to a time associated with the selected histogram bar.
1 more element
A system comprising:
- a hardware processor;
- and
- a non-transitory computer-readable storage medium storing executable instructions that, when executed by the hardware processor, cause the hardware processor to perform steps comprising:
- generating a set of image sub-frames by 1) performing a cropping operation on each of a set of walkthrough videos of an environment to produce overlapping portions of frames of the walkthrough videos, and 2) performing a de-duplication operation to remove overlapping portions of frames that correspond to a same image search query, each of the set of image sub-frames corresponding to one of the walkthrough videos and associated with a timestamp
- generating an interface for display including a representation of a floorplan of the environment corresponding to the set of walkthrough videos and a camera path corresponding to at least one of the set of walkthrough videos, the independent axis of the histogram comprising a timeline corresponding to the timestamps associated with the generated set of image sub-frames, the histogram including bars indicating both 1) instances of the generated set of image sub-frames within the set of walkthrough videos, and 2) a number of search results corresponding to the image search query;
- and
- in response to a histogram bar being selected, populating a portion of the interface with walkthrough video search results corresponding to a time associated with the selected histogram bar.
4 more elements
A non-transitory computer-readable storage medium storing executable instructions that, when executed by a hardware processor, cause the hardware processor to perform steps comprising:
- generating a set of image sub-frames by 1) performing a cropping operation on each of a set of walkthrough videos of an environment to produce overlapping portions of frames of the walkthrough videos, and 2) performing a de-duplication operation to remove overlapping portions of frames that correspond to a same image search query, each of the set of image sub-frames corresponding to one of the walkthrough videos and associated with a timestamp
- generating an interface for display including a representation of a floorplan of the environment corresponding to the set of walkthrough videos and a camera path corresponding to at least one of the set of walkthrough videos, and including a histogram, the independent axis of the histogram comprising a timeline corresponding to the timestamps associated with the generated set of image sub-frames, the histogram including bars indicating both 1) instances of the generated set of image sub-frames within the set of walkthrough videos, and 2) a number of search results corresponding to the image search query;
- and
- in response to a histogram bar being selected, populating a portion of the interface with walkthrough video search results corresponding to a time associated with the selected histogram bar.
1 more element
Dependencies are inferred from the claim text.