# US 2022/0088776 A1 · Robot to Human Feedback

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0088776 A1
- Document ID: US20220088776A1
- Status: Published application
- Kind: A1
- Application number: 17/457,831
- Filed: 2021-12-06
- Published: 2022-03-24
- Earliest claimed: 2014-08-25
- Assignees: X Development LLC
- Applicants: X Development LLC
- Inventors: James Joseph Kuffner
- Web page: https://patentgrep.com/patent/US20220088776A1

## CPC

- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- B25J 9/1676: {Avoiding collision or forbidden zones}
- B25J 9/1674: {characterised by safety, monitoring, diagnostic}
- B25J 11/0005: {Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means}
- B25J 19/06: Safety devices
- H04W 4/30: Services specially adapted for particular environments, situations or purposes
- H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
- B25J 19/061: {with audible signals (audio controls B25J13/003)}
- G05B 2219/39: Robotics, robotics to robotics hand
- G05B 2219/40: Robotics, robotics mapping to robotics vision
- G05B 2219/40202: Human robot coexistence
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

Example implementations may relate to a robotic system configured to provide feedback. In particular, the robotic system may determine a model of an environment in which the robotic system is operating. Based on this model, the robotic system may then determine one or more of a state or intended operation of the robotic system. Then, based one or more of the state or the intended operation, the robotic system may select one of one or more of the following to represent one or more of the state or the intended operation: visual feedback, auditory feedback, and one or more movements. Based on the selection, the robotic system may then engage in one or more of the visual feedback, the auditory feedback, and the one or more movements.

## Claims

### Claim 1

A method comprising:
determining, by a robotic system, a motion path of a user in an environment of the robotic system;
based on the motion path of the user in the environment, determining, by the robotic system, a performance metric corresponding to a position of the user along the motion path;
based on the determined performance metric, selecting, by the robotic system, a first feedback mode; and
engaging, by the robotic system, in the first feedback mode.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining a change in the performance metric based on a change in the position of the user;
based on the determined change in the performance metric, selecting, by the robotic system, a second feedback mode; and
engaging, by the robotic system, in the second feedback mode.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
determining a change in the performance metric based on a change in the position of the user; and
based on the determined change in the performance metric, stopping the robot from engaging in the first feedback mode.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first feedback mode includes operation of a visual indicator.

### Claim 5 (depends on 4)

The method of claim 4, wherein the visual indicator is a light generated by the robotic system.

### Claim 6 (depends on 4)

The method of claim 4, wherein the visual indicator is a gesture performed by the robotic system.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first feedback mode includes operation of an auditory indicator.

### Claim 8 (depends on 7)

The method of claim 7, wherein the auditory indicator comprises a siren.

### Claim 9 (depends on 1)

The method of claim 1, wherein the first feedback mode further comprises a notification sent by the robotic system to a computing device.

### Claim 10 (depends on 1)

The method of claim 1, wherein the robotic system comprises one or more appendages, and wherein engaging in the first feedback mode comprises carrying out one or more gestures using the one or more appendages.

### Claim 11 (depends on 1)

The method of claim 1, wherein engaging in the first feedback mode comprises repositioning of the robotic system from a first location in the environment to a second location in the environment.

### Claim 12 (depends on 1)

The method of claim 1, wherein engaging in the first feedback mode comprises the robotic system repositioning an object in the environment from a first location in the environment to a second location in the environment.

### Claim 13 (depends on 12)

The method of claim 12, wherein the first location is in a path of the user, and wherein the second location is away from the path of the user.

### Claim 14 (depends on 1)

The method of claim 1, wherein engaging in the first feedback mode comprises projecting light from one or more light sources of the robotic system towards an area of interest.

### Claim 15

A robotic system comprising:
one or more processors;
a non-transitory computer readable medium; and
program instructions stored on the non-transitory computer readable medium and executable by the one or more processors to:
determine an intended path of a user in an environment of the robotic system;
based on the intended path of the user in the environment, determine a performance metric corresponding to a position of the user along the intended path;
based on the determined performance metric, select a first feedback mode; and
engage in the first feedback mode.

### Claim 16 (depends on 15)

The robotic system of claim 15, wherein the program instructions are further executable by the one or more processors to:
determine a change in the performance metric based on a change in the position of the user;
based on the determined change in the performance metric, select a second feedback mode; and
engage in the second feedback mode.

### Claim 17 (depends on 16)

The robotic system of claim 16, wherein the first and second feedback modes comprise different types of visual indicators.

### Claim 18 (depends on 16)

The robotic system of claim 16, wherein the first feedback modes comprises a visual indicator and the second feedback modes comprises an auditory indicator.

### Claim 19 (depends on 16)

The robotic system of claim 16, wherein the second feedback modes comprises a visual indicator and an auditory indicator.

### Claim 20

A non-transitory computer readable medium having stored therein instructions executable by one or more processors to cause a robotic system to perform functions comprising:
determining, by the robotic system, a motion path of a user in an environment of the robotic system;
based on the motion path of the user in the environment, determining, by the robotic system, a performance metric corresponding to a position of the user along the motion path;
based on the determined performance metric, selecting, by the robotic system, a first feedback mode; and
engaging, by the robotic system, in the first feedback mode.
