# US 2024/0246179 A1 · Scan planning and scan operations for welding an object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0246179 A1
- Document ID: US20240246179A1
- Status: Published application
- Kind: A1
- Application number: 18/419,458
- Filed: 2024-01-22
- Published: 2024-07-25
- Earliest claimed: 2023-01-20
- Assignees: Path Robotics, Inc.
- Applicants: Path Robotics, Inc.
- Inventors: Madhavun CANDADAI VASU; Nima AJAM GARD; Sabhari NATARAJAN; Amrish BASKARAN; Gautham Narayan NARASIMHAN
- Web page: https://patentgrep.com/patent/US20240246179A1

## CPC

- B23K 9/0956: {using sensing means, e.g. optical}
- B25J 9/1684: {Tracking a line or surface by means of sensors}
- B23K 9/0953: {using computing means}
- B23K 26/032: {using optical means}
- B23K 37/0258: {Electric supply or control circuits therefor}
- B25J 9/1666: {Avoiding collision or forbidden zones}
- B25J 9/1671: {characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems}
- B25J 9/1679: {characterised by the tasks executed}
- B25J 9/1697: {Vision controlled systems}
- B25J 11/005: {Manipulators for mechanical processing tasks}
- G05B 19/4155: characterised by program execution, i.e. part program or machine function execution, e.g. selection of a program
- G06T 7/0004: {Industrial image inspection}
- G06T 7/75: {involving models}
- G05B 2219/40252: Robot on track, rail moves only back and forth
- G05B 2219/40314: Simulation of program locally before remote operation
- G05B 2219/45104: Lasrobot, welding robot
- G05B 2219/45135: Welding
- G05B 2219/45138: Laser welding
- G06T 2207/30152: Solder
- B23K: SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Disclosed are systems, methods, and apparatuses, including computer programs encoded on computer storage media, for operation of an assembly robotic system. In one aspect, the assembly robotic system performs at least one of a first or second scan operation. In the first scan operation, one or more scan poses is selected from among a plurality of generated candidate poses. For each scan pose of the one or more scan poses, the controller initiates a scan operation associated with a region identified to include a seam associated with a feature of the object. As part of the second scan operation, for each candidate scan pose, a scan operation is simulated. Based on the generated simulated scan data, multiple scan poses are selected and a scan trajectory is generated for a scan operation. Other aspects and features are also claimed and described.

## Claims

### Claim 1

An assembly robotic system configured to scan an object to be welded, the assembly robotic system comprising:
a controller that includes one or more processors and one or more memories coupled to the one or more processors, the controller configured to:
identify, based on an object pose of the object, a region associated with a seam of the object, the seam associated with a feature of one or more features of the object;
generate, based on the identified region, one or more candidate poses associated with a scan device, the scan device configured to perform a scan operation in association with the identified region;
select, based on the one or more candidate poses, one or more scan poses; and
for each scan pose of the one or more scan poses, initiate a scan operation associated with the identified region and performed by the scan device with the scan pose.

### Claim 2 (depends on 1)

The assembly robotic system of claim 1, wherein the controller is further configured to:
determine the object pose of the object based on a first scan performed on a segment of the object, the segment associated with the one or more features of the object, the one or more features identified based on a representation of the object.

### Claim 3 (depends on 2)

The assembly robotic system of claim 2, wherein the controller is further configured to:
determine a location of the feature, the seam, or both after the controller determines the object pose of the object;
wherein, to determine the location, the controller is configured to determine an approximate location of the feature, the seam, or both based on the first scan, the representation, or a combination thereof.

### Claim 4 (depends on 2)

The assembly robotic system of claim 2, wherein, to determine the object pose of the object, the controller is configured to:
identify, based on the one or more features, the segment of the object at which to initiate the first scan; and
initiate the first scan of the object at the identified segment, and
wherein the representation of the object includes a digital model of the object, the digital model including a computer aided design (CAD) model of the object.

### Claim 5 (depends on 2)

The assembly robotic system of claim 2, wherein the representation of the object includes:
a first digital representation of the object that includes first three dimensional (3D) point cloud data associated with a computer aided design (CAD) file associated with the object, and
a second digital representation of the object that includes second 3D point cloud data associated with the first scan.

### Claim 6 (depends on 5)

The assembly robotic system of claim 5, wherein the controller is further configured to:
determine a location of the feature, the seam, or a combination thereof, after the controller performs the first scan; and
wherein, to determine the location, the controller is configured to align a first coordinate system associated with the first digital representation and a second coordinate system associated with the second digital representation.

### Claim 7 (depends on 6)

The assembly robotic system of claim 6, wherein, to align the first coordinate system and the second coordinate system, the controller is configured to:
perform a coarse alignment; and
perform a fine alignment.

### Claim 8 (depends on 6)

The assembly robotic system of claim 6, wherein, to align the first coordinate system and the second coordinate system, the controller is configured to:
sample the first digital representation and the second digital representation such that first points associated with the first 3D point cloud data have an approximately equal density to second points associated with the second 3D point cloud data.

### Claim 9 (depends on 1)

The assembly robotic system of claim 1, wherein, to select the one or more scan poses, the controller is configured to select the one or more scan poses based on a size of an expected area of the region to be scanned by the scan device from the one or more scan poses based on a simulation of the scan operation performed from the one or more candidate poses.

### Claim 10 (depends on 1)

The assembly robotic system of claim 1, wherein the controller is configured to:
receive a representation of the object, wherein the representation includes a digital model of the object, and wherein the digital model includes a computer aided design (CAD) model of the object; and
identify the one or more features based on the representation of the object.

### Claim 11

A method, performed by a controller, for scanning an object to be welded, the method comprising:
identifying, based on an object pose of the object, a region associated with a seam of the object, the seam associated with a feature of one or more features of the object;
generating, based on the identified region, one or more candidate poses associated with a scan device, the scan device configured to perform a scan operation in association with the identified region;
selecting, based on the one or more candidate poses, one or more scan poses; and
for each scan pose of the one or more scan poses, initiating a scan operation associated with the identified region and performed by the scan device with the scan pose.

### Claim 12 (depends on 11)

The method of claim 11, wherein the one or more features include one or more disambiguating features, and wherein the one or more disambiguating features include one or more asymmetric features of the object.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
determining the object pose of the object based on a first scan performed on a segment of the object, the segment associated with the one or more features of the object, the one or more features identified based on a representation of the object.

### Claim 14 (depends on 13)

The method of claim 13, further comprising
after determining the object pose of the object, determining a location of the feature, the seam, or both,
wherein determining the location includes determining an approximate location of the feature, the seam, or both based on the first scan, the representation, or a combination thereof.

### Claim 15 (depends on 13)

The method of claim 13, wherein determining the object pose of the object includes:
identifying, based on the one or more features, the segment of the object at which to initiate the first scan; and
initiating the first scan of the object at the identified segment, and
wherein the representation of the object includes a digital model of the object, the digital model including a computer aided design (CAD) model of the object.

### Claim 16 (depends on 11)

The method of claim 11, further comprising:
identifying the one or more features, wherein identifying the one or more features includes processing three dimensional (3D) point cloud data associated with a representation of the object to identify one or more asymmetries of the object.

### Claim 17

A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a controller cause the controller to:
identify, based on an object pose of the object, a region associated with a seam of the object, the seam associated with a feature of one or more features of the object;
generate, based on the identified region, one or more candidate poses associated with a scan device, the scan device configured to perform a scan operation in association with the identified region;
select, based on the one or more candidate poses, one or more scan poses; and
for each scan pose of the one or more scan poses, initiate a scan operation associated with the identified region and performed by the scan device with the scan pose.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the instructions that, when executed by one or more processors of the controller, further cause the controller to:
receive a representation of the object, wherein the representation of the object includes:
a first digital representation of the object that includes first three dimensional (3D) point cloud data associated with a computer aided design (CAD) file associated with the object, and
a second digital representation of the object that includes second 3D point cloud data associated with the first scan.

### Claim 19 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the instructions that, when executed by the one or more processors of the controller, further cause the controller to:
receive, based on a first scan, a two dimensional (2D) image of an identified segment.

### Claim 20 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the instructions that, when executed by the one or more processors of the controller, further cause the controller to:
determine an axis of symmetry of the object, and
wherein the one or more features include at least one feature that is asymmetrical relative to the axis of symmetry of the object.

### Claim 21 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the instructions that, when executed by the one or more processors of the controller, further cause the controller to:
generate, after performance of the scan operation, a model of the object, the model based on:
a representation of the object, and
a first scan, the scan operation, or a combination thereof,
wherein, to generate the model, the instructions that, when executed by the one or more processors of the controller, cause the controller to combine:
mesh data associated with the representation, and
point cloud data associated with the first scan, the scan operation, or a combination thereof.
