Back to results
US 12,222,735 B2Granted
Systems, methods and programs for continuously directing an unmanned vehicle to an environment agnostic destination marked by a user
AssigneeXTEND Reality Expansion Ltd.
InventorsMatteo Shapira, Aviv Shapira, Adir Tubi, Rubi Liani, Erez Nehama
Life of this patent
Estimating the term…
Abstract
The disclosure relates to systems, methods and programs for maneuvering unmanned vehicles. More specifically, the disclosure relates to systems, methods and programs for controlling maneuverability of unmanned vehicles (ground, aerial and marine) by coupling vehicle controls with point of regard (PoR) in a 2D plane, translated to a continuously updating flight vector in a 3D space, based on 12 DOF head pose and/or hand gesture of a user.
Claims19
A system for directing an unmanned vehicle to a destination marked by a user, the system comprising:
- a) a display module;
- b) an unmanned aerial vehicle (UAV) comprising:
- i. an imaging module in communication with the display module;
- ii. an onboard flight controller, and
- iii. a sensor array configured to maintain at least one of a one-way or two-way communication with a central processing module (CPM);
- c) a human-machine interface (HMI) module in communication with the display module, wherein the HMI module is at least one of a wearable computing device or a remote controller, and
- d) the CPM is in communication with the display module, the UAV, the HMI module, and a processor in communication with a non-volatile memory comprising a processor-readable media having thereon a set of executable instructions that are configured, when executed, to cause the processor to:
- i. using a raster projection, receive, from the HMI on the display module, a two-dimensional (2D) coordinate value representing the destination marked by the user;
- ii. translate the 2D coordinate value to a three-dimensional (3D) coordinate value of the destination marked by the user in a 3D physical space;
- iii. using the UAV's imaging module, detect the UAV's current position in the 3D physical space coincident with the user marking of the destination on the display module;
- iv. compute a preferred approach angle in the 3D physical space from the UAV's current position to the 3D coordinate value of the destination marked by the user;
- v. based on the preferred approach angle and using the onboard flight controller, compute a 3D vector from the UAV's current position to the 3D coordinate value of the destination marked by the user;
- vi. use optical flow data, including a rate of motion of a texture in the raster projection, for continuous flight vector updating to correct the flight path to match the 3D vector and real speed of the UAV, and
- vii. maneuver the UAV to the 3D coordinate value of the destination marked by the user in the 3D physical space while simultaneously estimating the UAV's position in an environment.
11 more elements
A computer-readable medium storing a set of executable instructions configured, when executed by at least one processor, to cause the at least one processor to:
- a) using a human-machine interface (HMI) module in communication with a display module, an onboard flight controller, and a sensor array configured to maintain at least one of a one-way, or two-way communication with the at least one processor, detect a destination of an unmanned aerial vehicle (UAV) comprising an imaging module in communication with the display module, the destination marked by a user on the display module wherein, to detect the user-marked destination, the set of executable instructions is configured, when executed, to cause the at least one processor to:
- i. using a raster projection, receive a two-dimensional (2D) coordinate value from the HMI on the display module;
- and
- ii. translate the 2D coordinate value to a three-dimensional (3D) coordinate value in a 3D physical space;
- b) develop a flight plan for the UAV toward a predetermined destination in the 3D physical space, wherein to develop the flight plan, the set of executable instructions is configured, when executed, to cause the at least one processor to:
- i. using the UAV's imaging module, detect the UAV's current position in the 3D physical space coincident with the user marking of the destination on the display module;
- ii. compute a preferred approach trajectory in the 3D physical space from the UAV's current position to the 3D coordinate value in the 3D physical space;
- iii. based on the preferred approach trajectory and using the onboard flight controller, compute a 3D flight vector from the UAV's current position to the translated 3D coordinate value commensurate with the computed preferred approach trajectory;
- iv. use optical flow data, including a rate of motion of a texture of the raster projection, for continuous flight vector updating to correct the flight path to match the predetermined 3D flight vector and the real speed of the UAV commensurate with the preferred approach trajectory;
- and
- c) maneuver the UAV to the 3D coordinate value of the user-marked destination in the 3D physical space while simultaneously estimating the UAV's position in an environment.
8 more elements
Dependencies are inferred from the claim text.