US 9,649,767 B2 · Methods and systems for distributing remote assistance to facilitate robotic object manipulation

Abstract

Methods and systems for distributing remote assistance to facilitate robotic object manipulation are provided herein. Regions of a model of objects in an environment of a robotic manipulator may be determined, where each region corresponds to a different subset of objects with which the robotic manipulator is configured to perform a respective task. Certain tasks may be identified, and a priority queue of requests for remote assistance associated with the identified tasks may be determined based on expected times at which the robotic manipulator will perform the identified tasks. At least one remote assistor device may then be requested, according to the priority queue, to provide remote assistance with the identified tasks. The robotic manipulator may then be caused to perform the identified tasks based on responses to the requesting, received from the at least one remote assistor device, that indicate how to perform the identified tasks.

Claims20

Dependent claims

A system, comprising:

a remote assistor device;
and
a control system for a robotic device, wherein the control system is configured to:
identify a first task and a second task, wherein the first task involves a first object and the second task involves a second object;
determine a first expected time at which the robotic device is expected to perform the first task;
determine a second expected time at which the robotic device is expected to perform the second task;
based on the first expected time and the second expected time, determine an order in which to transmit a first request for remote assistance with the first task and a second request for remote assistance with the second task;
transmit the first request and the second request to the remote assistor device in accordance with the order;
receive, from the remote assistor device, a first response including data relating to performance of the first task;
receive, from the remote assistor device, a second response including data relating to performance of the second task;
and
cause the robotic device to perform both the first task and the second task based on the first response and the second response.

9 more elements

A method performed by a control system of a robotic device, the method comprising:

identifying a first task and a second task, wherein the first task involves a first object and the second task involves a second object;
determining a first expected time at which the robotic device is expected to perform the first task;
determining a second expected time at which the robotic device is expected to perform the second task;
based on the first expected time and the second expected time, determining an order in which to transmit a first request for remote assistance with the first task and a second request for remote assistance with the second task;
transmitting the first request and the second request to a remote assistor device in accordance with the order;
receiving, from the remote assistor device, a first response including data relating to performance of the first task;
receiving, from the remote assistor device, a second response including data relating to performance of the second task;
and
causing the robotic device to perform both the first task and the second task based on the first response and the second response.

6 more elements

A non-transitory computer readable medium having stored thereon instructions that, upon execution by a control system for a robotic device, cause the control system to perform operations comprising:

identifying a first task and a second task, wherein the first task involves a first object and the second task involves a second object;
determining a first expected time at which the robotic device is expected to perform the first task;
determining a second expected time at which the robotic device is expected to perform the second task;
based on the first expected time and the second expected time, determining an order in which to transmit a first request for remote assistance with the first task and a second request for remote assistance with the second task;
transmitting the first request and the second request to a remote assistor device in accordance with the order;
receiving, from the remote assistor device, a first response including data relating to performance of the first task;
receiving, from the remote assistor device, a second response including data relating to performance of the second task;
and
causing the robotic device to perform both the first task and the second task based on the first response and the second response.

6 more elements

Dependencies are inferred from the claim text.

Methods and systems for distributing remote assistance to facilitate robotic object manipulation

Life of this patent

Methods and systems for distributing remote assistance to facilitate robotic object manipulation

Life of this patent

Abstract

Claims20