# US 2024/0173866 A1 · Robotic system with multi-location placement control mechanism

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0173866 A1
- Document ID: US20240173866A1
- Status: Published application
- Kind: A1
- Application number: 18/518,278
- Filed: 2023-11-22
- Published: 2024-05-30
- Earliest claimed: 2022-11-27
- Assignees: MUJIN, Inc.
- Applicants: MUJIN, Inc.
- Inventors: Denys Kanunikov
- Web page: https://patentgrep.com/patent/US20240173866A1

## CPC

- B25J 9/1687: {Assembly, peg and hole, palletising, straight line, weaving pattern movement}
- B25J 9/1664: {characterised by motion, path, trajectory planning}
- B25J 9/1697: {Vision controlled systems}
- B65G 61/00: Use of pick-up or transfer devices or of manipulators for stacking or de-stacking articles not otherwise provided for
- B65G 67/02: Loading or unloading land vehicles
- G05B 2219/40006: Placing, palletize, un palletize, paper roll placing, box stacking
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D)
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A system and method for operating a robotic system to place objects on multiple placement locations on a given surface. The robotic system may use a sequence for such placement locations to control the placement of objects onto the placement locations.

## Claims

### Claim 1

A method for operating a robotic system, the method comprising:
identifying a package set representing available packages for placement on a platform;
obtaining a set of object properties of one or more packages in the package set, wherein the object properties represent physical dimensions, shapes, or a combination thereof of the one or more packages in the package set;
determining placement locations on the platform using the set of object properties, wherein the placement locations represent stacking locations for the package set or a portion thereof;
computing a set of distances for the placement locations, wherein the distance is between a corresponding one of the placement locations and a robotic unit that is configured to transfer the package set to the platform;
generating a sequence identifier for each of the placement locations based on the corresponding distance;
determining for a target package a target placement location from the placement locations, wherein the target placement location is determined by:
computing current stack heights of the placement locations; and
selecting the target placement location using the sequence identifier and the set of current stack heights, wherein a resulting height of the selected target placement location is within a stacking height limit and maintains heights of stacks nearer to the base robot lower or equal to heights of stacks farther from the base; and
implementing a motion plan for placing the target package on the platform at the target placement location.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
computing the current stack heights includes:
identifying a set of stacked packages currently placed at each of the placement locations;
calculating a total height for each of the placement locations, wherein the total height represents a vertical distance from the platform to a top surface of packages placed at the corresponding one of the placement locations; and
determining the target placement location includes:
using the sequence identifier, identifying candidate locations that, when the target object is placed, (1) maintains the corresponding resulting height within the stacking height limit and (2) maintains the nearer stack heights lower or equal to the farther stack heights;
for each candidate location, calculating a cumulative stack height by adding the corresponding resulting height to heights of remaining stacks; and
determining one of the candidate locations that has a highest cumulative stack height as the target placement location for the target package.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
the generated sequence identifier is higher for the placement locations that are closer to the robotic unit and lower for the placement locations that are farther from the robotic unit; and
the target placement location is determined based on a height-based placement rule of hi≤hj for i>j,
wherein i and j represent sequence identifiers and hi and hj represent heights at the corresponding placement locations, and
wherein the height-based placement rule is configured to maintain the heights of nearer stacks lower or equal to the heights of farther stacks.

### Claim 4 (depends on 3)

The method of claim 3, wherein:
the target placement location is determined based on an exception to the height-based placement rule,
wherein a first candidate location has a combined stack height greater than the stacking height limit,
wherein a second candidate location has a combined stack height less than the stacking height limit and greater than a current stacking height of the first candidate location,
wherein a first sequence identifier corresponding to the first candidate location is before a second sequence identifier corresponding to the second candidate location, and
wherein the second candidate location is selected as the target placement location.

### Claim 5 (depends on 4)

The method of claim 4, wherein the target placement location is determined based on the exception to the height-based placement rule when a height difference between a current stacking height of the second candidate placement location and the stacking height limit is within a height threshold.

### Claim 6 (depends on 3)

The method of claim 3, wherein:
determining the placement locations includes obtaining a packing plan that describes pre-planned locations for the package set over the platform; and
the target placement location is determined based on the height-based placement rule when the target package is received out of sequence.

### Claim 7 (depends on 3)

The method of claim 3, further comprising:
identifying multiple available objects for placement, wherein the multiple available objects include the target package;
determining a corresponding placement location for the available objects other than the target package; and
selecting the target package for transfer when the target placement location is farther than the placement locations corresponding to the other available objects.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
calculating a remaining height at each of the placement locations, wherein the remaining height corresponds to a difference between the current stack height at the corresponding one of the placement locations and the stacking height limit; and
wherein the target placement location is determined based on the exception to the height-based placement rule when the remaining height is less than the physical dimensions of the package set.

### Claim 9 (depends on 1)

The method of claim 1, wherein determining the target placement location for the target package includes:
identifying abnormal object properties associated with the target package,
wherein the abnormal object properties correspond to physical deviations of the target package, including deformation, bending, misalignment, and/or partial closing, and
wherein the abnormal object properties prevent additional stacking on top of the target package; and
adjusting the stacking height limit using the identified abnormal object properties.

### Claim 10 (depends on 1)

The method of claim 1, wherein a subset of properties in the set of object properties includes one or more lateral dimensions remaining fixed across the one or more packages in the package set.

### Claim 11

A tangible, non-transient computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to perform a method, the method comprising:
identifying a package set representing available packages for placement on a platform;
obtaining a set of object properties of one or more packages in the package set, wherein the object properties represent physical dimensions, shapes, or a combination thereof of the one or more packages in the package set;
determining placement locations on the platform using the set of object properties, wherein the placement locations represent stacking locations for the package set or a portion thereof;
computing a set of distances for the placement locations, wherein the distance is between a corresponding one of the placement locations and a robotic unit that is configured to transfer the package set to the platform;
generating a sequence identifier for each of the placement locations based on the corresponding distance;
determining for a target package a target placement location from the placement locations, wherein the target placement location is determined by:
computing current stack heights of the placement locations; and
selecting the target placement location using the sequence identifier and the set of current stack heights, wherein a resulting height of the selected target placement location is within a stacking height limit and maintains heights of stacks nearer to the base robot lower or equal to heights of stacks farther from the base; and
implementing a motion plan for placing the target package on the platform at the target placement location.

### Claim 12 (depends on 11)

The tangible, non-transient computer-readable medium of claim 11, wherein:
the generated sequence identifier is higher for the placement locations that are closer to the robotic unit and lower for the placement locations that are farther from the robotic unit; and
the target placement location is determined based on a height-based placement rule of hi≤hj for i>j,
wherein i and j represent sequence identifiers and hi and hj represent heights at the corresponding placement locations, and
wherein the height-based placement rule is configured to maintain the heights of nearer stacks lower or equal to the heights of farther stacks.

### Claim 13 (depends on 12)

The tangible, non-transient computer-readable medium of claim 12, wherein:
the target placement location is determined based on an exception to the height-based placement rule,
wherein a first candidate location has a combined stack height greater than the stacking height limit,
wherein a second candidate location has a combined stack height less than the stacking height limit and greater than a current stacking height of the first candidate location,
wherein a first sequence identifier corresponding to the first candidate location is before a second sequence identifier corresponding to the second candidate location, and
wherein the second candidate location is selected as the target placement location.

### Claim 14 (depends on 12)

The tangible, non-transient computer-readable medium of claim 12, wherein:
determining the placement locations includes obtaining a packing plan that describes pre-planned locations for the package set over the platform; and
the target placement location is determined based on the height-based placement rule when the target package is received out of sequence.

### Claim 15 (depends on 12)

The tangible, non-transient computer-readable medium of claim 12, wherein the method further comprising:
identifying multiple available objects for placement, wherein the multiple available objects include the target package;
determining a corresponding placement location for the available objects other than the target package; and
selecting the target package for transfer when the target placement location is farther than the placement locations corresponding to the other available objects.

### Claim 16

A robotic system comprising:
at least one processor; and
at least one memory device coupled to the at least one processor, the at least one memory having instructions stored thereon, that when executed by the processor, causes the processor to:
identify a package set representing available packages for placement;
obtain a set of object properties of one or more packages in the package set;
determine placement locations on a platform using the set of object properties, wherein the placement locations represent stacking locations for the package set or a portion thereof;
compute a set of distances for the placement locations, wherein the distance is between a corresponding one of the placement locations and a robotic unit that is configured to transfer the package set to the platform;
generate a sequence identifier for each of the placement locations based on the corresponding distance;
determine for a target package a target placement location from the placement locations; and
implement a motion plan for placing the target package on the platform at the target placement location.

### Claim 17 (depends on 16)

The robotic system of claim 16, wherein:
the generated sequence identifier is higher for the placement locations that are closer to the robotic unit and lower for the placement locations that are farther from the robotic unit; and
the target placement location is determined based on a height-based placement rule of hi≤hj for i>j,
wherein i and j represent sequence identifiers and hi and hj represent heights at the corresponding placement locations, and
wherein the height-based placement rule is configured to maintain the heights of nearer stacks lower or equal to the heights of farther stacks.

### Claim 18 (depends on 17)

The robotic system of claim 17, wherein:
the target placement location is determined based on an exception to the height-based placement rule,
wherein a first candidate location has a combined stack height greater than the stacking height limit,
wherein a second candidate location has a combined stack height less than the stacking height limit and greater than a current stacking height of the first candidate location,
wherein a first sequence identifier corresponding to the first candidate location is before a second sequence identifier corresponding to the second candidate location, and
wherein the second candidate location is selected as the target placement location.

### Claim 19 (depends on 17)

The robotic system of claim 17, wherein:
determining the placement locations includes obtaining a packing plan that describes pre-planned locations for the package set over the platform; and
the target placement location is determined based on the height-based placement rule when the target package is received out of sequence.

### Claim 20 (depends on 17)

The robotic system of claim 17, wherein the at least one memory includes instructions executable by the processor to:
identify multiple available objects for placement, wherein the multiple available objects include the target package;
determine a corresponding placement location for the available objects other than the target package; and
select the target package for transfer when the target placement location is farther than the placement locations corresponding to the other available objects.
