# US 2023/0410345 A1 · Auto-locating and positioning relative to an aircraft

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410345 A1
- Document ID: US20230410345A1
- Status: Published application
- Kind: A1
- Application number: 18/339,184
- Filed: 2023-06-21
- Published: 2023-12-21
- Earliest claimed: 2022-06-21
- Assignees: Wilder Systems Inc.
- Applicants: Wilder Systems Inc.
- Inventors: Spencer VOISS; William WILDER
- Web page: https://patentgrep.com/patent/US20230410345A1

## CPC

- B64F 5/40: Maintaining or repairing aircraft
- B25J 5/00: Manipulators mounted on wheels or on carriages (B25J1/00 takes precedence; program-controlled manipulators B25J9/00)
- B25J 5/007: {mounted on wheels}
- B25J 9/1664: {characterised by motion, path, trajectory planning}
- B25J 9/1666: {Avoiding collision or forbidden zones}
- B25J 9/1694: {characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion}
- B25J 9/1697: {Vision controlled systems}
- G05B 19/4155: characterised by program execution, i.e. part program or machine function execution, e.g. selection of a program
- G05D 1/2435: {Extracting 3D information}
- G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras
- G06T 7/60: Analysis of geometric attributes
- G06T 7/74: {involving reference images or patches}
- G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)}
- G07C 5/085: {using electronic data carriers}
- G05B 2219/40269: Naturally compliant robot arm
- G05D 2111/10: Optical signals
- G06T 2200/04: involving 3D image data
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- B64F: GROUND OR AIRCRAFT-CARRIER-DECK INSTALLATIONS SPECIALLY ADAPTED FOR USE IN CONNECTION WITH AIRCRAFT; DESIGNING, MANUFACTURING, ASSEMBLING, CLEANING, MAINTAINING OR REPAIRING AIRCRAFT, NOT OTHERWISE PROVIDED FOR; HANDLING, TRANSPORTING, TESTING OR INSPECTING AIRCRAFT COMPONENTS, NOT OTHERWISE PROVIDED FOR
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE

## Abstract

Techniques for auto-locating and positioning relative to an aircraft are disclosed. An example method can include a robot receiving a multi-dimensional representation of an enclosure that includes a candidate target aircraft. The robot can extract a geometric feature from the multi-dimensional representation associated with the candidate target aircraft. The robot can compare the geometric feature of the candidate target aircraft with a second geometric feature from a reference model of a target aircraft. The robot can determine whether the candidate target aircraft is the target aircraft based on the comparison. The robot can calculate a path from a location of the robot to the target aircraft based on the determination. The robot can traverse the path from the location to the target aircraft based on the calculation.

## Claims

### Claim 1

A method performed by a robot, the method comprising:
receiving a multi-dimensional representation of an enclosure that comprises a candidate target aircraft upon which the robot is to perform an operation;
extracting a geometric feature from the multi-dimensional representation associated with the candidate target aircraft;
comparing a six degrees of freedom position of the geometric feature with a second six degrees of freedom position of the second geometric feature; and
determining whether the candidate target aircraft is the target aircraft based on the comparison of the six degrees of freedom position and the second six degrees of freedom position;
determining whether the candidate target aircraft is the target aircraft based on the comparison;
determining a location of the robot relative to the target aircraft based on the multi-dimensional representation;
determining a path from the location of the robot to the target aircraft; and
traversing the path from the location to the target aircraft based on the determined path.

### Claim 2 (depends on 1)

The method of claim 1, wherein the multi-dimensional representation further comprises a second candidate target aircraft, and wherein the method further comprises:
extracting a third geometric feature from the multi-dimensional representation, the third geometric feature being associated with the second candidate target aircraft;
comparing the third geometric feature with the second geometric feature; and
determining that the candidate target aircraft is the target aircraft based on the comparison of the third geometric feature with the second geometric feature and the comparison of the geometric feature with a second geometric feature.

### Claim 3

(canceled)

### Claim 4 (depends on 1)

The method of claim 1, wherein the method further comprises:
receiving a second multi-dimensional representation of the enclosure while traversing the path to the target aircraft;
extracting a fourth geometric feature from the multi-dimensional representation, the fourth geometric feature being associated with the target aircraft;
comparing the fourth geometric feature with the second geometric feature;
verifying whether the candidate target aircraft is the target aircraft based on the comparison;
determining a second path from a second location of the robot to the target aircraft based on the determination; and
traversing the path from the second location to the target aircraft based on the determined second path.

### Claim 5 (depends on 1)

The method of claim 1, wherein extracting the geometric feature comprises:
generating input data for a neural network based on the multi-dimensional representation;
inputting the input data into the neural network; and
receiving an output from the neural network, wherein the neural network is trained to extract the geometric feature from the multi-dimensional representation, wherein the output is an indication that the geometric feature is associated with the candidate target aircraft.

### Claim 6 (depends on 5)

The method of claim 5, wherein comparing the geometric feature with the second geometric feature comprises:
receiving the output from the neural network;
generating a second input data for a second neural network based on the output of the neural network;
inputting the second input data in the second neural network;
receiving a second output from the neural network, wherein the second neural network is trained to compare the geometric feature with the second geometric feature to determine whether the candidate target aircraft is the target aircraft.

### Claim 7 (depends on 1)

The method of claim 1, wherein the method further comprises performing an operation on the target aircraft.

### Claim 8

A robot, comprising:
a processor; and
a computer-readable medium including instructions that, when executed by the processor, cause the processor to perform operations comprising:
emitting a light-based signal toward an enclosure;
receiving a point cloud of the enclosure that includes a candidate target aircraft, wherein the point cloud is based on a signal reflected from a surface of the enclosure;
extracting a geometric feature from the point cloud associated with the candidate target aircraft;
comparing a six degrees of freedom position of the geometric feature with a second six degrees of freedom position of the second geometric feature; and
determining whether the candidate target aircraft is the target aircraft based on the comparison of the six degrees of freedom position and the second six degrees of freedom position;
determining whether the candidate target aircraft is the target aircraft based on the comparison;
determining a location of the robot relative to the target aircraft based on the point cloud;
determining a path from the location of the robot to the target aircraft based on the determined relative location of the robot; and
traversing the path from the location to the target aircraft based on the determined path.

### Claim 9 (depends on 8)

The robot of claim 8, wherein the light-based signal is emitted from a three-dimensional (3D) vision sensor unit coupled to the robot.

### Claim 10 (depends on 8)

The robot of claim 8, wherein the point cloud further comprises a second candidate target aircraft, and wherein the instructions, when executed by the processor, further cause the processor to perform operations comprising:
extracting a third geometric feature from the point cloud associated with the second candidate target aircraft;
comparing the third geometric feature with the second geometric feature; and
determining that the candidate target aircraft is the target aircraft based on the comparison.

### Claim 11 (depends on 8)

The robot of claim 8, wherein the point cloud further comprises a second candidate target aircraft, and wherein the instructions, when executed by the processor, further cause the processor to perform operations comprising:
extracting a third geometric feature from the point cloud, the third geometric feature being associated with the second candidate target aircraft;
comparing the third geometric feature with the geometric feature;
determining that the candidate target aircraft and the second candidate target aircraft are a same model type based on the comparison;
receiving image data of the candidate target aircraft based on the determination, wherein the image data includes an aircraft identifier;
receiving a second aircraft identifier of the target aircraft;
comparing the aircraft identifier and the second aircraft identifier; and
determining that the candidate target aircraft is the target aircraft based on comparison of the aircraft identifier and the second aircraft identifier.

### Claim 12

(canceled)

### Claim 13 (depends on 8)

The robot of claim 8, wherein the instructions, when executed by the processor, further cause the processor to perform operations comprising:
receiving a second point cloud of the enclosure while traversing the path to the target aircraft;
extracting a fourth geometric feature from the point cloud associated with the target aircraft;
comparing the fourth geometric feature of the candidate target aircraft with the second geometric feature from the reference model of the target aircraft;
verifying whether the candidate target aircraft is the target aircraft based on the comparison;
determining a second location based on an odometry system, wherein the second location is along the determined path;
determining a second path from the second location of the robot to the target aircraft based on the determination of the second path; and
traversing the second path from the second location to the target aircraft based on the determined second path.

### Claim 14 (depends on 13)

The robot of claim 13, wherein the robot comprises a robot arm, and wherein the instructions, when executed by the processor, further cause the processor to perform operations comprising:
determining a set of robot arm motions for the robot to perform while traversing the path from the location to the target aircraft;
detecting that the robot has determined the second location and the second path;
detecting an object along the second path based on the second point cloud; and
determining a second set of robot arm motions for the robot to perform to avoid colliding with the object while traversing the second path from the second location to the target aircraft, wherein the second set of robot motions for collision avoidance is derived using inverse kinematics.

### Claim 15 (depends on 8)

The robot of claim 8, wherein extracting the geometric feature from the point cloud comprises:
generating input data for a neural network based on the point cloud;
inputting the input data into the neural network;
receiving an output from the neural network, wherein the neural network is trained to detect the geometric feature from the point cloud, wherein the output is an indication that the geometric feature is associated with the candidate target aircraft.

### Claim 16 (depends on 15)

The robot of claim 15, wherein comparing the geometric feature with the second geometric feature comprises:
receiving the output from the neural network;
generating a second input data for a second neural network based on the output of the neural network;
inputting the second input data in the second neural network;
receiving a second output from the neural network, wherein the second neural network is trained to compare the geometric feature with the second geometric feature to determine whether the candidate target aircraft is the target aircraft.

### Claim 17 (depends on 8)

The robot of claim 8, wherein the instructions, when executed by the processor, further cause the processor to perform operations comprising, upon reaching the target aircraft, removing a fastener from the target aircraft.

### Claim 18

A non-transitory, computer-readable medium having stored thereon a sequence of instructions which, when executed, causes a processor to perform operations comprising:
receiving a multi-dimensional representation of an enclosure that includes a candidate target aircraft;
extracting a geometric feature from the multi-dimensional representation associated with the candidate target aircraft;
comparing a six degrees of freedom position of the geometric feature with a second six degrees of freedom position of the second geometric feature; and
determining whether the candidate target aircraft is the target aircraft based on the comparison of the six degrees of freedom position and the second six degrees of freedom position;
determining whether the candidate target aircraft is the target aircraft based on the comparison;
determining a path from a location of a robot to the target aircraft based on the determination; and
traversing the path from the location to the target aircraft based on the determined path.

### Claim 19 (depends on 18)

The non-transitory, computer-readable medium of claim 18, wherein the multi-dimensional representation further comprises a candidate second aircraft; and wherein the instructions which, when executed, further cause the processor to perform operations comprising:
extracting a third geometric feature from the multi-dimensional representation associated with the candidate second aircraft;
comparing the third geometric features of the candidate target aircraft with the second geometric feature from a reference model of the target aircraft; and
determining that the candidate target aircraft is the target aircraft based on the comparison.

### Claim 20

(canceled)

### Claim 21 (depends on 1)

The method of claim 1 wherein the method further comprises:
receiving a second multi-dimensional representation of the enclosure while traversing the path to the target aircraft;
extracting a fourth geometric feature from the multi-dimensional representation, the fourth geometric feature being associated with the target aircraft;
comparing the fourth geometric feature with the second geometric feature;
verifying whether the candidate target aircraft is the target aircraft based on the comparison;
determining a second path from a second location of the robot to the target aircraft based on the determination; and
traversing the path from the second location to the target aircraft based on the determined second path.

### Claim 22 (depends on 4)

The method of claim 4, wherein the method further comprises;
determining a set of robot arm motions for the robot to perform while traversing the path from the location to the target aircraft;
detecting that the robot has determined the second location and the second path;
detecting an object along the second path based on the second point cloud; and
determining a second set of robot arm motions for the robot to perform to avoid colliding with the object while traversing the second path from the second location to the target aircraft, wherein the second set of robot motions for collision avoidance is derived using inverse kinematics.

### Claim 23 (depends on 18)

The non-transitory, computer-readable medium of claim 18; and wherein the instructions which, when executed, further cause the processor to perform operations comprising:
receiving a second multi-dimensional representation of the enclosure while traversing the path to the target aircraft;
extracting a fourth geometric feature from the multi-dimensional representation, the fourth geometric feature being associated with the target aircraft;
comparing the fourth geometric feature with the second geometric feature;
verifying whether the candidate target aircraft is the target aircraft based on the comparison;
determining a second path from a second location of the robot to the target aircraft based on the determination; and
traversing the path from the second location to the target aircraft based on the determined second path.