# US 2021/0241217 A1 · Picking workstation with mobile robots & machine vision verification of each transfers performed by human operators

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241217 A1
- Document ID: US20210241217A1
- Status: Published application
- Kind: A1
- Application number: 17/235,038
- Filed: 2021-04-20
- Published: 2021-08-05
- Earliest claimed: 2017-04-18
- Assignees: Alert Innovation Inc.
- Applicants: Alert Innovation Inc.
- Inventors: John G. Lert, JR.; William J. Fosnight; Devin Lert; Stephen Akiki
- Web page: https://patentgrep.com/patent/US20210241217A1

## CPC

- B65G 1/1378: {the orders being assembled on fixed commissioning areas remote from the storage areas}
- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- G06Q 10/08741: {by picking of items from inventory for fulfillment}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- B65G 1/0492: {with cars adapted to travel in storage aisles}
- B65G 29/02: for inclined or vertical transit
- B65G 2201/0258: Trays, totes or bins
- B65G 2203/041: Camera
- B65G 2203/044: Optical
- G05D 1/0287: {involving a plurality of land vehicles, e.g. fleet or convoy travelling (traffic control systems for road vehicles G08G1/00, particularly anticollision systems G08G1/16)}
- B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES

## Abstract

A workstation is disclosed for transferring objects between containers such as product and order totes in an order fulfillment system. The workstation may include an object pick system, for example including a measuring array or a machine vision system, which directs and monitors the transfer of objects between the containers.

## Claims

### Claim 1

A workstation for transferring objects from a first container to a second container, at least the second container comprising one or more segregated sections, the workstation comprising:
a support structure configured to receive mobile robots carrying the first and second containers; and
a machine vision system provided at one or more locations above at least the second container, the machine vision system configured to confirm placement of an object into the second container; and
a controller receiving feedback from the machine vision system to determine whether the object was placed in a predetermined section of the one or more segregated sections in the second container.

### Claim 2 (depends on 1)

A workstation as recited in claim 1, wherein the machine vision system comprises one or more cameras for detecting placement of at least one of a hand and the object being placed in the second container.

### Claim 3 (depends on 1)

A workstation as recited in claim 1, wherein the machine vision system is configured to determine: i) if a human hand is breaching a surface of the second container, ii) if the human hand is physically transferring an object into the second container and, iii) if the human is transferring the correct object.

### Claim 4 (depends on 1)

A workstation as recited in claim 1, wherein the machine vision system is further provided at one or more locations above the first container, and wherein the machine vision system configured to confirm removal of an object from a predetermined section of one or more segregated sections in the first container, the controller receiving feedback from the machine vision system to determine whether the object was removed from the predetermined section of the first container.

### Claim 5

A workstation for transferring objects from a first container to a second container, the workstation comprising:
a support structure configured to receive mobile robots carrying the first and second containers, the containers containing sub-totes in different configurations; and
an object pick system comprising indicators configured to:
direct picking of an object from a sub-totes of the configuration of sub-totes of the first container, and
direct placing of the object into a second predefined section of the second container, and
the object pick system further comprising sensors in a fixed position on the support structure, the sensors comprising a first group of sensors oriented along an X-axis and a second group of sensors oriented along a Y-axis orthogonal to the X-axis, the pick system receiving the configuration of sub-totes of the first container, and the pick system mapping the sensors along the X-axis and Y-axis to correspond to boundaries of the sub-totes in the configuration of sub-totes, the sensors configured to detect obstruction of a plane above the first container, the X-axis sensors configured to detect an X-axis position of the obstruction of the plane, and the Y-axis sensors configured to detect a Y-axis position of the obstruction of the plane, the object pick system receiving the X-axis and Y-axis positions to determine the sub-tote in the first container that was accessed from the mapping of the sensors to the sub-totes.

### Claim 6 (depends on 5)

A workstation as recited in claim 5, wherein the sensors of the object pick system are configured to sense entry and exit of an operator's hand into or from the first sub-tote.

### Claim 7 (depends on 5)

A workstation as recited in claim 5, wherein the sensors of the object pick system are configured to sense removal of the object from the first sub-tote.

### Claim 8 (depends on 5)

A workstation as recited in claim 5, wherein the sensors of the object pick system are configured to sense placement of the object into the second predefined section.

### Claim 9 (depends on 5)

A workstation as recited in claim 5, wherein the second predefined section comprises one of the entire second container, a sub-tote within the second container or an order bag.

### Claim 10 (depends on 5)

A workstation as recited in claim 5, wherein the indicators comprise one or more lights configured to illuminate an area of the first sub-tote from which the object is to be removed from the first container for transfer to the second container.

### Claim 11 (depends on 10)

A workstation as recited in claim 10, wherein the sensors are configured to sense whether the object is removed from the first sub-tote in conformity with the area illuminated by the one or more lights.

### Claim 12 (depends on 11)

A workstation as recited in claim 11, wherein the object pick system is further configured to provide a visual indicator of whether or not the object is removed from the first sub-tote in conformity with the area illuminated by the one or more lights.

### Claim 13 (depends on 12)

A workstation as recited in claim 12, wherein the visual indicator comprises a first colored light when the object is removed in conformity with the illuminated area, and a second colored light when the object is not removed in conformity with the illuminated area.

### Claim 14 (depends on 5)

A workstation as recited in claim 5, wherein the indicators comprise one or more lights configured to illuminate an area of the second predefined section into which the object is to be placed.

### Claim 15 (depends on 14)

A workstation as recited in claim 14, wherein the sensors are configured to sense whether the object is removed from the first predefined section in conformity with the area illuminated by the one or more lights.

### Claim 16 (depends on 15)

A workstation as recited in claim 15, wherein the object pick system is further configured to provide a visual indicator of whether or not the object is placed into the second predefined section in conformity with the area illuminated by the one or more lights.

### Claim 17 (depends on 16)

A workstation as recited in claim 16, wherein the visual indicator comprises a first colored light when the object is placed in conformity with the illuminated area, and a second colored light when the object is not placed in conformity with the illuminated area.

### Claim 18 (depends on 5)

A workstation as recited in claim 5, wherein sensors of the object pick system comprise one or more optical sensors.

### Claim 19 (depends on 18)

A workstation as recited in claim 18, wherein the optical sensors comprise a series of light emitters and receivers.

### Claim 20 (depends on 18)

A workstation as recited in claim 18, wherein the optical sensors comprise one or more cameras.

### Claim 21 (depends on 20)

A workstation as recited in claim 20, wherein the one or more cameras are configured to sense removal of the object from the first predefined section.

### Claim 22 (depends on 20)

A workstation as recited in claim 20, wherein the one or more cameras are configured to sense placement of the object into the second predefined section.

### Claim 23

A workstation for transferring objects from a first container to a second container, the second container comprising a configuration of sub-totes from a plurality of different configurations of sub-totes, the workstation comprising:
a support structure configured to receive mobile robots carrying the first and second containers; and
a measuring array comprising a two-dimensional array of sensors provided in a fixed position in a plane over at least the second container, the measuring array configured to emit a two-dimensional array of light beams, the measuring array further configured to determine a two-dimensional position where the array of light beams is obstructed; and
a controller configured to:
receive the configuration of sub-totes of the second container,
map the sensors along the X-axis and Y-axis to correspond to boundaries of the sub-totes in the configuration of sub-totes in the second container,
determine an X-Y position of obstruction of a plane above the second container,
determine the sub-tote in the second container that was accessed from the mapping of the sensors to the sub-totes.

### Claim 24 (depends on 23)

A workstation as recited in claim 23, wherein the measuring array is configured to emit light beams in the two-dimensional array of light beams that are orthogonal to each other.

### Claim 25 (depends on 23)

A workstation as recited in claim 23, wherein the measuring array is configured to detect a hand of an operator placing an object into the sub-tote.

### Claim 26 (depends on 23)

A workstation as recited in claim 23, wherein the measuring array is configured to detect an obstruction above a rim of the second container.

### Claim 27 (depends on 23)

A workstation as recited in claim 23, wherein the measuring array comprises a series of paired emitters and receivers configured to emit and receive the two-dimensional array of light beams.

### Claim 28 (depends on 23)

A workstation as recited in claim 23, wherein the controller receives feedback from the receivers to determine which receivers receive the light beam emitted from the emitters.

### Claim 29 (depends on 23)

A workstation as recited in claim 23, wherein the measuring array provided over the second container comprises a first measuring array, the workstation further comprising:
a second measuring array provided in a plane over the first container, the second measuring array configured to emit a two-dimensional array of light beams, the second measuring array further configured to determine a two-dimensional position over the first container where the array of light beams is obstructed; and
the controller receiving feedback from the second measuring array to whether the two-dimensional position over the first container corresponds to a position over a predetermined section of the one or more sections in the first container.

### Claim 30

A workstation for transferring objects from a first container to a second container, comprising:
a plurality of supports configured to support robots transferring objects;
a drive system for driving the plurality of supports in a closed loop path, the drive system receiving a first robot at first workstation position in the closed loop path, presenting a second robot for object insertion/removal at second workstation position in closed loop path, and delivering a third robot for exit at a third workstation position in the closed loop path.

### Claim 31 (depends on 30)

A workstation as recited in claim 30, wherein the closed loop path is a circular path.

### Claim 32 (depends on 30)

A workstation as recited in claim 30, wherein the wheel is defined by a plurality of spokes.

### Claim 33 (depends on 30)

A workstation as recited in claim 30, wherein the wheel is defined by a plurality of chains.

### Claim 34 (depends on 30)

A workstation as recited in claim 30, wherein the drive system further constrains the supports to remain in a horizontal position during the closed loop path.

### Claim 35 (depends on 30)

A workstation as recited in claim 30, wherein the drive system further constrains the supports to move between a horizontal position and tilted to a predefined angle other than horizontal during the closed loop path.

### Claim 36 (depends on 35)

A workstation as recited in claim 35, wherein the supports are positioned at the predefined angle other than horizontal at the second position presenting the second robot for object insertion/removal.

### Claim 37 (depends on 30)

A workstation as recited in claim 30, wherein the first workstation position of the closed loop path is at a first level, and wherein the third workstation position of the closed loop path is at a second level different than the first level, robots travelling to and from the workstation at the first and second levels.

### Claim 38

A workstation for transferring objects from a first container to a second container, comprising:
at least one circular lift;
a plurality of supports configured to support robots transferring objects, the plurality of supports affixed to and circulating on the at least one circular lift;
a drive system for driving the at least one circular lift and the plurality of supports around a circular path, the drive system receiving a first robot at first workstation position in the circular path, presenting a second robot for object insertion/removal at second workstation position in circular path, and delivering a third robot for exit at a third workstation position in the circular path.

### Claim 39 (depends on 38)

A workstation as recited in claim 38, wherein the drive system further constrains the supports to remain in a horizontal position at least at the first workstation position and at the third workstation position during the circular path.

### Claim 40 (depends on 38)

A workstation as recited in claim 38, wherein the drive system further constrains the supports to move between a horizontal position and tilted to a predefined angle other than horizontal at the second workstation position presenting the second robot for object insertion/removal.

### Claim 41 (depends on 38)

A workstation as recited in claim 38, wherein the first workstation position of the circular path is at a first level, and wherein the third workstation position of the circular path is at a second level different than the first level, robots travelling to and from the workstation at the first and second levels.
