# US 2023/0266144 A1 · Method of predicting road attributer, data processing system and computer executable code

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0266144 A1
- Document ID: US20230266144A1
- Status: Published application
- Kind: A1
- Application number: 18/012,130
- Filed: 2021-08-03
- Published: 2023-08-24
- Earliest claimed: 2020-08-07
- Assignees: Grabtaxi Holdings Pte. Ltd.
- Applicants: Grabtaxi Holdings Pte. Ltd
- Inventors: Yifang YIN; Jagannadan VARADARAJAN; Roger ZIMMERMANN; Wenmiao HU; An TRAN; Guanfeng WANG
- Web page: https://patentgrep.com/patent/US20230266144A1

## CPC

- G01C 21/3852: {Data derived from aerial or satellite images}
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/803: {of input or preprocessed data}
- G06V 10/82: using neural networks
- G06V 20/13: Satellite images
- G06V 20/182: {Network patterns, e.g. roads or rivers}
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of predicting one or more road segment attributes corresponding to a road segment in a geographical area, the method including: providing trajectory data and satellite image of the geographical area; calculating one or more image channels based on the trajectory data; and using at least one processor, classifying the road segment based on the one or more image channels and the satellite image using a trained classifier into prediction probabilities of the road attributes A data processing system including one or more processors configured to carry out a the method of predicting road attributes. A computer executable code including instructions for predicting one or more road segment attributes according to the method.

## Claims

### Claim 1

A method of predicting one or more road attributes corresponding to a road segment in a geographical area, the method comprising:
providing trajectory data and satellite image of the geographical area;
calculating two or more image channels based on the trajectory data;
classifying the road segment based on the two or more image channels and the satellite image using a trained classifierinto prediction probabilities of the road attributes (20); and
concatenating (118) the two or more image channels into a concatenated trajectory image before classifying the road segment,
wherein the classifying the road segment comprises: using the trained classifier (64), fusing the satellite image and the concatenated trajectory image into a fused image and inputting the fused image in a neural network stream of the trained classifier (64).

### Claim 2 (depends on 1)

The method of claim 1, wherein calculating two or more image channels based on the trajectory data comprises at least two of:
i) calculating a trajectory image channel, as part of the two or more image channels, based on the trajectory data;
ii) calculating a bearing image channel, as part of the two or more image channels (32), based on the trajectory data (30);
iii) calculating a speed image channel, as part of the two or more image channels (32), based on the trajectory data (30).

### Claim 3 (depends on 2)

The method of claim 2, comprising i), ii), and iii).

### Claim 4 (depends on 3)

The method of claim 3, wherein calculating the trajectory image channel comprises assigning a count of a number of trajectory points of the trajectory data that are projected onto each pixel of a matrix of pixels onto that pixel.

### Claim 5 (depends on 3)

The method of claim 3, wherein calculating the bearing image channel comprises providing a multichannel bearing image comprising multichannel pixels, wherein a number of channels represents a number of bearing bins, and quantizing bearing values into the bearing bins forming a bearing histogram for each of the multichannel pixels.

### Claim 6 (depends on 3)

The method of claim 3, wherein calculating the speed image channel comprises providing a multichannel speed image comprising multichannel pixels, wherein a number of channels represents a number of speed bins, and quantizing speed values into the speed bins forming a speed histogram for each of the multichannel pixels.

### Claim 7 (depends on 3)

The method of claim 3, comprising concatenating the trajectory image channel, the bearing image channel, and the speed image channel into the concatenated trajectory image before classifying the road segment.

### Claim 8 (depends on 7)

The method of claim 7, further comprising applying a smoothing filter on the concatenated trajectory image before classifying the road segment.

### Claim 9 (depends on 7)

The method of claim 7, further comprising applying image rotation until the road segment in the concatenated trajectory image is aligned with the road segment in the satellite image before classifying the road segment.

### Claim 10 (depends on 7)

The method of claim 7, wherein the trained classifier comprises:
a trajectory neural network stream;
a satellite image neural network stream;
a fully connected layerfor receiving input from the trajectory neural network stream and the satellite image neural network stream and outputting a fused stream,
wherein the classifying comprises inputting the concatenated trajectory image into the trajectory neural network stream, and inputting the satellite image in the satellite image neural network stream.

### Claim 11 (depends on 10)

The method of claim 10, wherein the trajectory neural network stream is configured to process a multiple trajectory images of a same geographical area comprising different times, wherein the multiple trajectory images include the concatenated trajectory image (34).

### Claim 12 (depends on 1)

The method of claim 1, wherein the trained classifier comprises a Convolutional Neural Network.

### Claim 13 (depends on 12)

The method of claim 12, wherein the trained classifier comprises:
a first group comprising 2 first convolutional layers, followed by
a second group comprising 3 second convolutional layers, followed by
a max pooling layer, followed by
a third group comprising 2 fully connected layers, followed by
an output layer;
wherein each convolutional layer of the first group and the second group is followed by an activation unit, and wherein each fully connected layer of the third group is followed by a respective activation unit.

### Claim 14

A data processing system for predicting one or more road attributes corresponding to a road segment in a geographical area, the data processing system comprising:
a first memory configured to store trajectory data of the geographical area;
a second memory configured to store a satellite image of the geographical area;
a processor configured to calculate two or more image channels based on the trajectory data; and
a trained classifier comprising: a neural network configured to predict the road attributes based on the two or more image channels and the satellite image, and an output layer configured to provide prediction probabilities of the road attributes,
wherein the trained classifier is configured to classify the road segment based on the two or more image channels and the satellite image into the prediction probabilities of the road attributes,
wherein the system is configured to concatenate the two or more image channels into a concatenated trajectory image before classifying the road segment, and
wherein the trained classifier is configured to fuse the satellite image and the concatenated trajectory image into a fused image and input the fused image in the neural network of the trained classifier.

### Claim 15 (depends on 14)

The data processing system of claim 14, wherein the trained classifier comprises:
a trajectory neural network stream;
a satellite image neural network stream;
a fully connected layer for receiving input from the trajectory neural network stream and the satellite image neural network stream and outputting a fused stream,
wherein the trained classifier is configured to classify when the concatenated trajectory image is input into the trajectory neural network stream (S4), and the satellite image is input in the satellite image neural network stream.

### Claim 16 (depends on 15)

The data processing system of claim 15, wherein the trajectory neural network stream is configured to process multiple trajectory images of a same geographical area comprising different times, wherein the multiple trajectory images include the concatenated trajectory image (34).

### Claim 17 (depends on 14)

The data processing system of claim 14, wherein the trained classifier comprises a Convolutional Neural Network.

### Claim 18 (depends on 1)

A non-transitory computer-readable medium storing computer executable code comprising instructions for predicting one or more road attributes according to the method of claim 1.

### Claim 19 (depends on 1)

A computer executable code comprising instructions for predicting one or more road attributes according to the method of claim 1.
