# US 10,766,141 B1 · Robotic system with a coordinated transfer mechanism > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,766,141 B1 - Document ID: US10766141B1 - Status: Granted - Kind: B1 - Application number: 16/824,673 - Filed: 2020-03-19 - Granted: 2020-09-08 - Earliest claimed: 2019-05-09 - Assignees: MUJIN, Inc. - Applicants: MUJIN, Inc. - Inventors: Rosen Nikolaev Diankov; Shintaro Matsuoka; Yoshiki Kanemoto; Hironori Mizoguchi - Web page: https://patentgrep.com/patent/US10766141B1 ## CPC - B25J 9/1682: {Dual arm manipulator; Coordination of several manipulators} - B25J 9/1664: {characterised by motion, path, trajectory planning} - B25J 9/1687: {Assembly, peg and hole, palletising, straight line, weaving pattern movement} - B25J 9/1697: {Vision controlled systems} - B25J 15/0616: {with vacuum} - B25J 19/023: {including video camera means} - B65G 15/00: Conveyors having endless load-conveying surfaces, i.e. belts and like continuous members, to which tractive effort is transmitted by means other than endless driving elements of similar configuration (having load-conveying surfaces formed by interconnected longitudinal links B65G17/06) - B65G 47/76: Fixed or adjustable ploughs or transverse scrapers - B65G 47/90: Devices for picking-up and depositing articles or materials - B65G 67/00: Loading or unloading vehicles (by means incorporated in the vehicles B60-B64, e.g., B60P1/00, B61D9/00, B63B27/00, B64D9/00; ground or aircraft-carrier-deck installations for aircraft B64F1/32) - B65G 2203/041: Camera - B65G 2205/00: Stopping elements used in conveyors to stop articles or arrays of articles - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D) ## Abstract The present disclosure relates to operation of a robotic system to transfer an object from a source to a destination. The robotic system may implement one or more motion plans or portions thereof to operate a picking robot to grip and lift the object and place a transfer tray under the lifted object. The robotic system may further implement the one or more motion plans or portions thereof to place the object on the transfer tray and laterally displace the transfer tray and the object thereon toward the destination. The robotic system may operate a placement mechanism to transfer the object from the transfer tray to the destination. ## Claims ### Claim 1 A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause one or more computing systems to perform a method for operating a robotic system to transfer an object from a source to a destination, the method comprising: accessing image data representative of the source; deriving a motion plan based on the image data, wherein the motion plan includes one or more portions for operating (1) a picking robot configured to grip and manipulate the object and (2) a lateral transfer mechanism including a transfer tray located adjacent to the source and the destination and configured to move laterally between the source and the destination; implementing a picking portion of the motion plan for operating the picking robot to grip and lift the object from the source; detecting a clearing event representative of the object reaching a predetermined height that is above a height of the transfer tray; implementing a source transfer portion of the motion plan for operating the lateral transfer mechanism to move the transfer tray toward the source; implementing a tray placement portion of the motion plan based on the clearing event and/or a tray location for placing the object on the transfer tray; and implementing a destination transfer portion of the motion plan for operating the lateral transfer mechanism to laterally move the transfer tray toward the destination with the object thereon and to move the object from the transfer tray and onto the destination. ### Claim 2 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein the method further comprises implementing a stopper placement portion of the motion plan for positioning a stopper such that when the transfer tray moves toward the destination, the stopper contacts the object thereon and causes the object to slide off the transfer tray and drop onto the destination. ### Claim 3 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein the method further comprises implementing a stopper placement portion of the motion plan for: when the transfer tray is over the source, disengaging a stopper to allow the object to pass by the stopper while the transfer tray moves toward the destination; and when the transfer tray is over the destination, engaging the stopper to contact the object and cause the object to slide off the transfer tray and drop onto the destination. ### Claim 4 (depends on 3) The non-transitory computer-readable storage medium of claim 3, wherein the method further comprises implementing a stopper alignment portion of the motion plan for moving the stopper along a horizontal direction and aligning an edge of the stopper over a drop location. ### Claim 5 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein the image data representative of the source is first image data, and wherein the method further comprises: when the transfer tray is over the source, accessing second image data representative of the destination; and when the transfer tray is over the destination: accessing third image data representative of the object on the transfer tray; and deriving a destination placement portion of the motion plan based on the second and third image data, wherein the destination placement portion is for operating a packing robot. ### Claim 6 (depends on 5) The non-transitory computer-readable storage medium of claim 5, wherein the method further comprises implementing the destination placement portion for operating the packing robot to: pick the object from the transfer tray when the transfer tray is adjacent to or above the destination; and place the object onto the destination. ### Claim 7 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein the method further comprises implementing a source reloading portion of the motion plan for replacing a container at the source after placing the object on the transfer tray and/or while the transfer tray moves toward the destination. ### Claim 8 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein the detection of the clearing event is based on a triggering event from one or more area sensors. ### Claim 9 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein the method further comprises: accessing next image data representative of the source while the transfer tray is adjacent to the destination; deriving a next motion plan for operating the picking robot and the lateral transfer mechanism based on the next image data while the transfer tray is over the destination; and implementing a portion of the next motion plan for operating the picking robot to grip and lift a next object from the source while the transfer tray is over the destination. ### Claim 10 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein a timing for moving the transfer tray is based on additional image data from a source sensor and/or a tracked height of an end-effector of the picking robot. ### Claim 11 A method of operating a robotic system to transfer an object from a source to a destination, the method comprising: accessing image data representative of the source; deriving a motion plan based on the image data, wherein the motion plan includes one or more portions for operating (1) a picking robot configured to grip and manipulate the object and (2) a lateral transfer mechanism including a transfer tray configured to move laterally between the source and the destination; implementing a picking portion of the motion plan for operating the picking robot to grip and lift the object from the source; implementing a source transfer portion of the motion plan for operating the lateral transfer mechanism to move the transfer tray toward the source; implementing a tray placement portion of the motion plan for placing the object on the transfer tray; and implementing a destination transfer portion of the motion plan for operating the lateral transfer mechanism to laterally move the transfer tray toward the destination with the object thereon. ### Claim 12 (depends on 11) The method of claim 11, wherein deriving the motion plan includes deriving the picking portion of the motion plan for operating the picking robot to vertically transfer the gripped object without laterally displacing the gripped object. ### Claim 13 (depends on 11) The method of claim 11, wherein: the motion plan includes one or more portions for operating a stopper configured to engage the object on the transfer tray during lateral movement thereof; and the method further comprises implementing a stopper placement portion of the motion plan for positioning the stopper to engage the object such that when the transfer tray moves toward or away from the destination, the stopper contacts the object thereon and causes the object to slide off the transfer tray and drop onto the destination. ### Claim 14 (depends on 11) The method of claim 11, wherein the method further comprises: obtaining second image data representative of the destination; obtaining third image data representative of the object on the transfer tray; deriving a destination placement portion of the motion plan based on the second and third image data, wherein the destination placement portion is for operating a packing robot; and implementing the destination placement portion for operating the packing robot to— pick the object from the transfer tray when the transfer tray is adjacent to or above the destination; and place the object onto the destination. ### Claim 15 An object transfer system comprising: a picking robot configured to grip and lift an object from a source; a lateral transfer mechanism configured to laterally transfer a transfer tray and the object thereon from the source to a destination; and a placement mechanism configured to engage the object and place the object on the destination. ### Claim 16 (depends on 15) The object transfer system of claim 15, wherein: the transfer tray is above the source and the destination; and the placement mechanism includes a stopper located above the transfer tray and configured to engage the object on the transfer tray during lateral transfer of the object. ### Claim 17 (depends on 16) The object transfer system of claim 16, wherein the transfer tray includes slots extending below a top surface of the transfer tray and along a direction parallel to lateral movement of the transfer tray. ### Claim 18 (depends on 16) The object transfer system of claim 16, wherein the stopper is a fork-style stopper including extensions that extend below the top surface of the transfer tray and into the slots. ### Claim 19 (depends on 16) The object transfer system of claim 16, wherein: the picking robot includes an end-effector having a set of suction cups configured to contact a top surface of the object, the end-effector configured to create a vacuum condition between the set of suction cups and the top surface of the object for gripping the object; and the transfer tray is a perforated-surface transfer tray having depressions on a top surface thereof configured to prevent vacuum conditions when a subset in the set of suction cups contact the top surface of the transfer tray. ### Claim 20 (depends on 15) The object transfer system of claim 15, wherein the placement mechanism is a packing robot configured to grip and lift the object from the transfer tray and place the object on the destination.