# US 2018/0322197 A1 · Video data creation and management system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322197 A1
- Document ID: US20180322197A1
- Status: Published application
- Kind: A1
- Application number: 15/970,532
- Filed: 2018-05-03
- Published: 2018-11-08
- Earliest claimed: 2017-05-03
- Applicants: Survae Inc.
- Inventors: Eric Scott HESTERMAN
- Web page: https://patentgrep.com/patent/US20180322197A1

## CPC

- G06F 16/78: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- B61L 15/0027: {Radio-based, e.g. using GSM-R}
- B61L 15/0094: {Recorders on the vehicle}
- B61L 23/041: {Obstacle detection}
- B61L 25/025: {Absolute localisation, e.g. providing geodetic coordinates}
- G06F 16/29: Geographical information databases
- G06F 16/70: of video data
- G06F 16/73: Querying
- G06F 16/7335: {Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate (end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728)}
- G06F 16/75: Clustering; Classification
- G06F 16/9537: Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
- H04Q 9/02: Automatically-operated arrangements
- B61L 2205/04: Satellite based navigation systems, e.g. global positioning system [GPS]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- B61L: GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC (brakes or auxiliary equipment B61H, B61K; point or crossing construction E01B)
- H04Q: SELECTING (switches, relays, selectors H01H; wireless communication networks H04W)

## Abstract

A method includes maintaining a representation of a spatial region, maintaining a plurality of trajectory records, each trajectory record comprising a sequence of time points and corresponding spatial coordinates, maintaining, for each trajectory record of the plurality of trajectory records, sensor data, the sensor data being synchronized to the sequence of time points and corresponding spatial coordinates, presenting a portion of the representation of the spatial region including presenting a representation of multiple trajectory records of the plurality of trajectory records, each trajectory record of the multiple trajectory records having at least some spatial coordinates located in the portion of the spatial region.

## Claims

### Claim 1

An method comprising:
maintaining a representation of a spatial region;
maintaining a plurality of trajectory records, each trajectory record comprising a sequence of time points and corresponding spatial coordinates;
maintaining, for each trajectory record of the plurality of trajectory records, sensor data, the sensor data being synchronized to the sequence of time points and corresponding spatial coordinates. presenting a portion of the representation of the spatial region including
presenting a representation of multiple trajectory records of the plurality of trajectory records, each trajectory record of the multiple trajectory records having at least some spatial coordinates located in the portion of the spatial region.

### Claim 2 (depends on 1)

The method of claim 1 further comprising determining the multiple trajectory records based on a query.

### Claim 3 (depends on 2)

The method of claim 2 wherein the query includes one or both of spatial query parameters and temporal query parameters.

### Claim 4 (depends on 2)

The method of claim 2 wherein the query includes sensor data query parameters.

### Claim 5 (depends on 2)

The method of claim 2 wherein the query includes a specification of a second sequence of time points and corresponding spatial coordinates, and the multiple trajectory records intersect with the second sequence of time points and corresponding spatial coordinates.

### Claim 6 (depends on 5)

The method of claim 5 wherein intersecting with the second sequence of time points and corresponding spatial coordinates includes collecting sensor data related to the second sequence of time points and corresponding spatial coordinates.

### Claim 7 (depends on 5)

The method of claim 5 further comprising receiving a selection of a part of a first line representing the specification of the second sequence of time points and corresponding spatial coordinates and, the part of the first line corresponding to a first time point in the second sequence of time points and corresponding spatial coordinates, the selection causing presentations representations of the sensor data corresponding to the first time point from the multiple trajectory records.

### Claim 8 (depends on 7)

The method of claim 7 further comprising receiving an input causing the selection of the part of the first line to move along the first line, the moving causing sequential presentation of the representations of the sensor data corresponding to a plurality of time points adjacent to the first time point in the sequence of time points of the multiple trajectory records.

### Claim 9 (depends on 2)

The method of claim 2 wherein the query constrains the multiple trajectory records to trajectory records of the plurality of trajectory records that include spatial coordinates that traverse a first portion of the spatial region.

### Claim 10 (depends on 9)

The method of claim 9 wherein the first portion of the spatial region is equivalent to the presented portion of the representation of the spatial region.

### Claim 11 (depends on 9)

The method of claim 9 wherein the first portion of the spatial region is a spatial region inside of the presented portion of the representation of the spatial region.

### Claim 12 (depends on 1)

The method of claim 1 wherein, for each of the multiple trajectory records, the representation of the trajectory record includes a line tracing a route defined by the trajectory record.

### Claim 13 (depends on 12)

The method of claim 12 where the line includes a polyline.

### Claim 14 (depends on 12)

The method of claim 12 further comprising receiving a selection of a part of a first line representing a first trajectory record of the multiple trajectory records, the part of the first line corresponding to a first time point in the sequence of time points and corresponding spatial coordinates of the first trajectory, the selection causing presentation of a representation of the sensor data corresponding to the first time point.

### Claim 15 (depends on 14)

The method of claim 14 wherein the selection further causes presentation of a representation of sensor data corresponding to the first time point of a second trajectory record of the multiple trajectory records.

### Claim 16 (depends on 14)

The method of claim 14 further comprising receiving an input causing the selection of the part of the first line to move along the first line, the moving causing sequential presentation of representations of the sensor data corresponding to a number of time points adjacent to the first time point in the sequence of time points and corresponding spatial coordinates of the first trajectory record.

### Claim 17 (depends on 16)

The method of claim 16 wherein the moving further causes sequential presentation of representations of sensor data corresponding to a number of time points adjacent to the first time point in a sequence of times points and corresponding spatial coordinates of a second trajectory record of the multiple trajectory records.

### Claim 18 (depends on 1)

The method of claim 1 wherein the spatial coordinates include geospatial coordinates.

### Claim 19 (depends on 1)

The method of claim 1 wherein the spatial coordinates include three-dimensional coordinates.

### Claim 20 (depends on 1)

The method of claim 1 wherein the sensor data includes camera data.

### Claim 21 (depends on 1)

The method of claim 1 wherein the sensor data includes telemetry data.

### Claim 22 (depends on 21)

The method of claim 21 wherein the telemetry data includes one or more of temperature data, pressure data, acoustic data, velocity data, acceleration data, fuel reserve data, battery reserve data, altitude data, heading data, orientation data, force data, acceleration data, sensor orientation data, field of view data, zoom data, sensor type data, exposure data, date and time data, electromagnetic data, chemical detection data, and signal strength data.

### Claim 23 (depends on 1)

The method of claim 1 further comprising receiving a selection of a representation of a first trajectory record of the multiple trajectory records and, based on the selection, presenting a trajectory record viewer including:
presenting a sensor data display region in the trajectory record viewer for viewing the sensor data according to the sequence of time points of the first trajectory record, the sensor display region including a first control for scrubbing through the sensor data according to the time points of the first trajectory record, and
presenting spatial trajectory display region in the trajectory record viewer for viewing the representation of the first trajectory record in a second portion of the representation of the spatial region, the spatial trajectory display region including a second control for scrubbing through the time points of the first trajectory record according to the spatial coordinates of the first trajectory record.

### Claim 24 (depends on 23)

The method of claim 23 wherein movement of the first control to a first time point and corresponding first sensor data point of the first trajectory record causes movement of the second control to the first time point and corresponding first spatial coordinate of the first trajectory record, and movement of the second control to a second spatial coordinate and corresponding second time point of the first trajectory record causes movement of the first control to the second time point and corresponding second sensor data point of the first trajectory record.

### Claim 25 (depends on 23)

The method of claim 23 wherein the sensor data of the first trajectory record includes video data and the sensor display region in the trajectory viewer includes a video player.

### Claim 26 (depends on 23)

The method of claim 23 further comprising presenting a second sensor data display region in the trajectory record viewer for viewing sensor data according to a sequence of time points of a second trajectory record, the second sensor display region including a third control for scrubbing through the sensor data according to the time points of the second trajectory record, wherein scrubbing the third control causes a corresponding scrubbing of the first control.

### Claim 27

A video capture and interface system comprising:
an interface system configured to receive information acquired from a moving vehicle, including receiving a plurality of parts of the information with corresponding different delays relative to the time of acquisition at the vehicle;
the interface system including a presentation subsystem for displaying or providing access to the parts of the information as they become available in conjunction with a display of a trajectory of the vehicle.