# US 12,223,447 B2 · Drone taxi system based on multi-agent reinforcement learning and drone taxi operation using the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,447 B2 - Document ID: US12223447B2 - Status: Granted - Kind: B2 - Application number: 17/690,231 - Filed: 2022-03-09 - Published: 2022-09-22 - Granted: 2025-02-11 - Earliest claimed: 2021-03-17 - Assignees: Korea University Research and Business Foundation; Ajou University Industry-Academic Cooperation Foundation - Applicants: Korea University Research and Business Foundation; Ajou University Industry-Academic Cooperation Foundation - Inventors: Joongheon Kim; Won Joon Yun; Jae-Hyun Kim; Soyi Jung - Web page: https://patentgrep.com/patent/US12223447B2 ## CPC - G06Q 10/0631: Resource planning, allocation, distributing or scheduling for enterprises or organisations - G01C 21/20: Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence) - G05D 1/0027: {involving a plurality of vehicles, e.g. fleet or convoy travelling (fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00)} - G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)} - B64U 2101/61: for transporting passengers - B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] - B64U 2201/104: using satellite radio beacon positioning systems, e.g. GPS - G05D 1/101: {specially adapted for aircraft} - G05D 1/46: Control of position or course in three dimensions [3D] - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract Provided are a drone taxi system based on multi-agent reinforcement learning and a drone taxi operation method using the same. The drone taxi system includes a plurality of drone taxies configured to receive call information including departure point information and destination information from passenger terminals present within a certain range and a control server configured to receive call information of passengers from each drone taxi, select a candidate passenger depending on whether a passenger is present, generate travel route information of each drone taxi from drone state information of the plurality of drone taxies through multi-agent reinforcement learning, and transmit the travel route information to the drone taxi. ## Claims ### Claim 1 A processor-implemented drone taxi system comprising: a plurality of drone taxies configured to receive call information including a departure location point and a destination location point from passenger terminals present within a certain range; and a control server configured to receive call information of passengers from each drone taxi, select a candidate passenger depending on whether a passenger is present, generate travel route information of each drone taxi from drone state information of the plurality of drone taxies using an optimization model trained through multi-agent reinforcement learning, transmit the travel route information to each drone taxi, and control each drone taxi to travel, according to the generated travel route information and under a control of the control server, to a departure location point of the selected candidate passenger, the controlling further including controlling each drone taxi to update a route of the traveling to be set with the generated travel route information and controlling each drone taxi to transmit assignment information and boarding information to a terminal of the selected candidate passenger, wherein the control server comprises: a passenger selector configured to receive the call information of the passengers from each drone taxi and select the candidate passenger depending on whether a passenger is present; and a route optimizer configured to, every time the drone state information of each of the plurality of drone taxies is changed and updated, re-generate the travel route information of each drone taxi from the drone state information of the plurality of drone taxies using the optimization model trained through the multi-agent reinforcement learning and transmit the travel route information to each drone taxi, and the drone state information includes current location information, onboard passenger information, candidate passenger information, and vacant seat information, wherein the optimization model is trained using a two-stage attention mechanism trained through multi-agent reinforcement learning. ### Claim 2 (depends on 1) The drone taxi system of claim 1, wherein, when there is no passenger in the drone taxi, the passenger selector compares distances through the current location information of the drone taxi and the call information received from the drone taxi and determine the call information, which indicates a shortest distance from the current location of the drone taxi and allows a long-distance travel, to select the candidate passenger. ### Claim 3 (depends on 1) The drone taxi system of claim 1, wherein, when a passenger who allows a carpool is in the drone taxi, the passenger selector analyzes a cosine similarity between a travel direction based on the travel route information of the drone taxi and a travel direction based on the departure location point and the destination location point of the received call information and selects the call information which has a highest cosine similarity as the candidate passenger. ### Claim 4 (depends on 1) The drone taxi system of claim 1, wherein the route optimizer comprises: a graph definer configured to define relationships between the plurality of drone taxies as a graph structure using the drone state information of the drone taxies; an attention part configured to remove irrelevant edges by processing the graph structure and give weights; and a route generator configured to generate the travel route information of each drone taxi on the basis of the processed graph structure and transmit the travel route information to the drone taxi. ### Claim 5 (depends on 1) The drone taxi system of claim 1, wherein the control server further comprises a compensator configured to process payment of a fare according to an onboard time in the onboard passenger information when arrival information is received from the drone taxi. ### Claim 6 (depends on 5) The drone taxi system of claim 5, wherein the compensator compares the onboard time based on the onboard passenger information with the travel route information of the drone taxi to set a penalty according to a difference therebetween and processes the payment of the fare to which the set penalty is applied. ### Claim 7 A processor-implemented method using a drone taxi system based on multi-agent reinforcement learning, the drone taxi operation method comprising: a passenger search operation in which a drone taxi searches for a passenger present within a certain range and receives call information including a departure location point and a destination location point from passenger terminals; a passenger selection operation in which a control server receives the call information of passengers from the drone taxi and selects a candidate passenger depending on whether a passenger is present; and a route setting operation in which the control server generates travel route information of each drone taxi from drone state information of a plurality of drone taxies using an optimization model trained through the multi-agent reinforcement learning, transmits the travel route information to each drone taxi, and control each drone taxi to travel, according to the generated travel route information and under a control of the control server, to a departure location point of the selected candidate passenger, the controlling further including controlling each drone taxi to update a route of the traveling to be set with the generated travel route information and controlling each drone taxi to transmit assignment information and boarding information to a terminal of the selected candidate passenger, wherein the control server comprises: a passenger selector configured to receive the call information of the passengers from each drone taxi and select the candidate passenger depending on whether a passenger is present; and a route optimizer configured to, every time the drone state information of each of the plurality of drone taxies is changed and updated, re-generate the travel route information of each drone taxi from the drone state information of the plurality of drone taxies using the optimization model trained through the multi-agent reinforcement learning and transmit the travel route information to each drone taxi, wherein the drone state information includes current location information, onboard passenger information, candidate passenger information, and vacant seat information, and wherein the optimization model is trained using a two-stage attention mechanism trained through multi-agent reinforcement learning. ### Claim 8 (depends on 7) The drone taxi operation method of claim 7, wherein the passenger selection operation comprises, when there is no passenger in the drone taxi, comparing distances through the drone state information and the call information received from the corresponding drone taxi and determining the call information, which indicates a shortest distance from a current location of the drone taxi and allows a long-distance travel, to select the candidate passenger. ### Claim 9 (depends on 7) The drone taxi operation method of claim 7, wherein the passenger selection operation comprises, when a passenger who allows a carpool is in the drone taxi, analyzing a cosine similarity between a travel direction based on the travel route information of the drone taxi and a travel direction based on the departure location point and the destination location point of the received call information and selecting the call information having a highest cosine similarity as the candidate passenger.