# US 2022/0284533 A1 · Systems and methods for repositioning vehicles in a ride-hailing platform

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284533 A1
- Document ID: US20220284533A1
- Status: Published application
- Kind: A1
- Application number: 17/186,935
- Filed: 2021-02-26
- Published: 2022-09-08
- Earliest claimed: 2021-02-26
- Assignees: Beijing Didi Infinity Technology and Development Co., Ltd.
- Applicants: Beijing DiDi Infinity Technology and Development Co., Ltd.
- Inventors: Shuaiji LI; Yan JIAO; Zhiwei QIN
- Web page: https://patentgrep.com/patent/US20220284533A1

## CPC

- G06Q 30/0205: {based on location or geographical consideration}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/092: Reinforcement learning
- G06Q 30/0202: Market predictions or forecasting for commercial activities
- G06Q 50/40: Business processes related to the transportation industry (shipping G06Q10/083)
- G08G 1/202: {Dispatching vehicles on the basis of a location, e.g. taxi dispatching}
- G06N 3/045: Combinations of networks
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

This disclosure describes systems and methods for repositioning vehicles. An exemplary method includes obtaining a plurality of first signals corresponding to a vehicle and a plurality of second signals corresponding to supply-demand statuses in a plurality of neighboring areas of the vehicle; inputting the plurality of first and second signals into a trained neural network and obtaining, from the trained neural network, a plurality of action values for repositioning the vehicle to the plurality of neighboring areas respectively; determining, based on the plurality of action values, a plurality of probabilities for repositioning the vehicle to the plurality of neighboring areas respectively; determining, according to the plurality of probabilities, one of the plurality of neighboring areas for the vehicle to reposition to; and transmitting a signal to a computing device associated with the vehicle to reposition the vehicle to the one determined neighboring area.

## Claims

### Claim 1

A computer-implemented method, comprising:
obtaining, by one or more computing devices, a plurality of first signals corresponding to a vehicle and a plurality of second signals corresponding to supply-demand statuses in a plurality of neighboring areas of the vehicle;
inputting, by the one or more computing devices, the plurality of first and second signals into a trained neural network and obtaining, from the trained neural network, a plurality of action values for repositioning the vehicle to the plurality of neighboring areas respectively;
determining, by the one or more computing devices based on the plurality of action values, a plurality of probabilities for repositioning the vehicle to the plurality of neighboring areas respectively;
determining, by the one or more computing devices according to the plurality of probabilities, one of the plurality of neighboring areas for the vehicle to reposition to; and
transmitting, by the one or more computing devices, a signal to a computing device associated with the vehicle to reposition the vehicle to the one determined neighboring area.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
training a neural network using a state-action-reward-state-action (SARSA) framework based on a plurality of historical trajectories of one or more historical vehicles, historical supply-demand statuses in a plurality of neighboring areas of the one or more historical vehicles, and a plurality of actual action values based on historical data to obtain the trained neural network.

### Claim 3 (depends on 2)

The method of claim 2, wherein each of the plurality of historical trajectories corresponds to a historical vehicle, spans across a plurality of points in time, and comprises a set of states at each of the plurality of points in time, and
the set of states comprises a historical time, a historical location, one or more historical features of the historical vehicle, and a supply-demand status in a historical area in which the historical vehicle was located.

### Claim 4 (depends on 3)

The method of claim 3, wherein the training comprises:
for each of the plurality of historical trajectories, sequentially feeding the sets of states of the each historical trajectory and the corresponding historical supply-demand status in the plurality of neighboring areas of the historical vehicle to a neural network to obtain a predicted action value; and
training the neural network based on the predicted action value and one of the plurality of actual action values.

### Claim 5 (depends on 1)

The method of claim 1, wherein the determining the plurality of probabilities for repositioning the vehicle to the plurality of neighboring areas comprises:
inputting the plurality of action values into a softmax layer of the neural network to obtain a plurality of probabilities.

### Claim 6 (depends on 1)

The method of claim 1, wherein the plurality of first signals comprise:
a current time, a current location of the vehicle, features of the vehicle, and a supply-demand status at the current location of the vehicle.

### Claim 7 (depends on 1)

The method of claim 1, wherein the plurality of features of the vehicle comprise at least one of the following:
vehicle capacity, manufacturer, year, and model.

### Claim 8 (depends on 1)

The method of claim 1, wherein each of the plurality of action values respectively corresponds to a predicted reward for repositioning the vehicle to a corresponding neighboring area.

### Claim 9 (depends on 1)

The method of claim 1, wherein the neural network comprises an attention module, and the method further comprises:
for a corresponding neighboring area, determining, through the attention module, a score based on a first supply-demand vector representing the supply-demand status of the current location and a second supply-demand vector representing the supply-demand status in the corresponding neighboring area;
applying the score to the second supply-demand vector to obtain a weighted supply-demand vector; and
generating a weighted supply-demand context vector based on the plurality of weighted supply-demand vectors respectively corresponding to the plurality of neighboring areas.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
performing cerebellar embedding on one or more of the plurality of first signals to obtain one or more embedded first signals;
feeding the one or more embedded first signals to a first Multi-Layer Perceptron (MLP) to obtain a first output;
concatenating the first output with the weighted supply-demand context vector to obtain a second output; and
feeding the second output into a second MLP to obtain the plurality of action values.

### Claim 11 (depends on 1)

The method of claim 1, wherein the supply-demand status includes a ratio of the supply to the demand, the supply corresponds to a number of idle vehicles providing transportation services, and the demand corresponds to a number of pending orders for transportation.

### Claim 12 (depends on 1)

The method of claim 1, wherein the determining one of the plurality of neighboring areas for the vehicle to reposition to comprises:
performing unequal probability sampling from the plurality of neighboring areas based on the plurality of probabilities to obtain one sampled area.

### Claim 13

A system comprising one or more processors and one or more non-transitory computer-readable memories coupled to the one or more processors, the one or more non-transitory computer-readable memories storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
obtaining a plurality of first signals corresponding to a vehicle and a plurality of second signals corresponding to supply-demand statuses in a plurality of neighboring areas of the vehicle;
inputting the plurality of first and second signals into a trained neural network and obtaining, from the trained neural network, a plurality of action values for repositioning the vehicle to the plurality of neighboring areas respectively;
determining, based on the plurality of action values, a plurality of probabilities for repositioning the vehicle to the plurality of neighboring areas respectively;
determining, according to the plurality of probabilities, one of the plurality of neighboring areas for the vehicle to reposition to; and
transmitting a signal to a computing device associated with the vehicle to reposition the vehicle to the one determined neighboring area.

### Claim 14 (depends on 13)

The system of claim 13, wherein the operations further comprise:
training a neural network using a state-action-reward-state-action (SARSA) framework based on a plurality of historical trajectories of one or more historical vehicles, historical supply-demand statuses in a plurality of neighboring areas of the one or more historical vehicles, and a plurality of actual action values based on historical data to obtain the trained neural network.

### Claim 15 (depends on 14)

The system of claim 14, wherein each of the plurality of historical trajectories corresponds to a historical vehicle, spans across a plurality of points in time, and comprises a set of states at each of the plurality of points in time, and
the set of states comprises a historical time, a historical location, one or more historical features of the historical vehicle, and a supply-demand status in a historical area in which the historical vehicle was located.

### Claim 16 (depends on 15)

The system of claim 15, wherein the training comprises:
for each of the plurality of historical trajectories, sequentially feeding the sets of states of the each historical trajectory and the corresponding historical supply-demand status in the plurality of neighboring areas of the historical vehicle to a neural network to obtain a predicted action value; and
training the neural network based on the predicted action value and one of the plurality of actual action values.

### Claim 17

A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
obtaining a plurality of first signals corresponding to a vehicle and a plurality of second signals corresponding to supply-demand statuses in a plurality of neighboring areas of the vehicle;
inputting the plurality of first and second signals into a trained neural network and obtaining, from the trained neural network, a plurality of action values for repositioning the vehicle to the plurality of neighboring areas respectively;
determining, based on the plurality of action values, a plurality of probabilities for repositioning the vehicle to the plurality of neighboring areas respectively;
determining, according to the plurality of probabilities, one of the plurality of neighboring areas for the vehicle to reposition to; and
transmitting a signal to a computing device associated with the vehicle to reposition the vehicle to the one determined neighboring area.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the operations further comprise:
training a neural network using a state-action-reward-state-action (SARSA) framework based on a plurality of historical trajectories of one or more historical vehicles, historical supply-demand statuses in a plurality of neighboring areas of the one or more historical vehicles, and a plurality of actual action values based on historical data to obtain the trained neural network.

### Claim 19 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the neural network comprises an attention module, and the method further comprises:
for a corresponding neighboring area, determining, through the attention module, a score based on a first supply-demand vector representing the supply-demand status of the current location and a second supply-demand vector representing the supply-demand status in the corresponding neighboring area;
applying the score to the second supply-demand vector to obtain a weighted supply-demand vector; and
generating a weighted supply-demand context vector based on the plurality of weighted supply-demand vectors respectively corresponding to the plurality of neighboring areas.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the operations further comprise:
performing cerebellar embedding on one or more of the plurality of first signals to obtain one or more embedded first signals;
feeding the one or more embedded first signals to a first Multi-Layer Perceptron (MLP) to obtain a first output;
concatenating the first output with the weighted supply-demand context vector to obtain a second output; and
feeding the second output into a second MLP to obtain the plurality of action values.

### Claim at position 21

-40. (canceled)