Multimodal object identification
AssigneeGoogle Inc.
InventorsMichael Joseph Quinlan, Gabriel A. Cohen
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus for receiving a command for controlling a robot, the command referencing an object, receiving sensor data for a portion of an environment of the robot, identifying, from the sensor data, a gesture of a human that indicates a spatial region located outside of the portion of the environment described by the sensor data, searching map data for the object, determining, based at least on searching the map data for the object referenced in the command, that the object referenced in the command is present in the spatial region, and in response to determining that the object referenced in the command is present in the spatial region, controlling the robot to perform an action with respect to the object referenced in the command.
Claims20
A computer-implemented method comprising:
- storing information indicative of objects in an environment of a robot;
- detecting, based on sensor data that is captured by the robot and that describes a first region, a user in the environment;
- determining, based on a location of the detected user, a second region outside the first region described by the sensor data;
- accessing a portion of the stored information indicating locations of one or more objects in the second region;
- identifying, based at least on the accessed portion of the stored information, an object that that is located in the second region;
- and
- causing the robot to perform an action associated with the identified object that is located in the second region.
A system comprising:
- one or more sensors;
- one or more processors;
- and
- one or more machine-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
- storing information indicative of objects in an environment of a robot;
- detecting, based on sensor data that is captured by a robot and that describes a first region, a user in the environment;
- determining, based on a location of the detected user, a second region outside the first region described by the sensor data;
- accessing a portion of the stored information indicating locations of one or more objects in the second region;
- identifying, based at least on the accessed portion of the stored information, an object that that is located in the second region;
- and
- causing the robot to perform an action associated with the identified object that is located in the second region.
One or more non-transitory machine-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
- storing information indicative of objects in an environment of a robot;
- detecting, based on sensor data that is captured by a robot and that describes a first region, a user in the environment;
- determining, based on a location of the detected user, a second region outside the first region described by the sensor data;
- accessing a portion of the stored information indicating locations of one or more objects in the second region;
- identifying, based at least on the accessed portion of the stored information, an object that that is located in the second region;
- and
- causing the robot to perform an action associated with the identified object that is located in the second region.
Dependencies are inferred from the claim text.