# US 10,369,459 B2 · Tactile, interactive neuromorphic robots

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,369,459 B2
- Document ID: US10369459B2
- Status: Granted
- Kind: B2
- Application number: 15/984,880
- Filed: 2018-05-21
- Published: 2018-11-01
- Granted: 2019-08-06
- Earliest claimed: 2013-11-04
- Assignees: The Regents of the University of California
- Applicants: The Regents of the University of California
- Inventors: Liam David Bucci; Ting-Shuo Chou; Jeffrey Lawrence Krichmar
- Web page: https://patentgrep.com/patent/US10369459B2

## CPC

- A63F 9/24: Games using electronic circuits not otherwise provided for
- A63F 9/0604: {based on the use of colours}
- A63H 11/00: Self-movable toy figures
- A63H 33/005: {Motorised rolling toys}
- G06F 3/03547: {Touch pads, in which fingers can move on a surface}
- G06F 3/03549: {Trackballs (G06F3/03541 takes precedence)}
- G06F 3/0383: {Signal control means within the pointing device}
- G06F 3/041: Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06N 3/008: based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/063: using electronic means
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- A63F 2009/2408: {Touch-sensitive buttons}
- A63F 2009/2447: {Motion detector}
- A63H 2200/00: Computerized interactive toys, e.g. dolls
- G05B 2219/40625: Tactile sensor
- G05B 2219/40627: Tactile image sensor, matrix, array of tactile elements, tixels
- G06F 2203/04103: Manufacturing, i.e. details related to manufacturing processes specially suited for touch sensitive devices
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63H: TOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

In one embodiment, a neuromorphic robot includes a curved outer housing that forms a continuous curved outer surface, a plurality of trackball touch sensors provided on and extending across the continuous curved outer surface in an array, each trackball sensor being configured to detect a direction and velocity a sweeping stroke of a user, and a plurality of lights, one light being collocated with each trackball touch sensor and being configured to illuminate when its collocated trackball touch sensor is stroked by the user, wherein the robot is configured to interpret the sweeping stroke of the user sensed with the plurality of trackball touch sensors and to provide immediate visual feedback to the user at the locations of the touched trackball touch sensors.

## Claims

### Claim 1

A neuromorphic robot comprising:
a curved outer housing that forms a continuous curved outer surface;
a plurality of trackball touch sensors provided on and extending across the continuous curved outer surface in an array, each trackball sensor being configured to (i) detect a direction and velocity of a sweeping stroke of a user touching the outer surface, and (ii) encode signals, in response to the sweeping stroke, indicative of the direction and velocity of the sweeping stroke;
a plurality of lights, one light being collocated with each trackball touch sensor and being configured to illuminate when its collocated trackball touch sensor is touched by the sweeping stroke of the user; and
a primary controller configured to interpret, based on the encoded signals, the sweeping stroke of the user sensed with the plurality of trackball touch sensors to provide immediate visual feedback to the user at the locations of the touched trackball touch sensors.

### Claim 2 (depends on 1)

The robot of claim 1, wherein the outer housing is convex.

### Claim 3 (depends on 1)

The robot of claim 1, wherein the outer housing is hemispherical.

### Claim 4 (depends on 1)

The robot of claim 1, further comprising a bump sensor configured to detect when the robot is bumped by the user.

### Claim 5 (depends on 1)

The robot of claim 1, further comprising a pan and tilt camera configured to capture images of the user.

### Claim 6 (depends on 1)

The robot of claim 1, further comprising a wireless communication component configured to communicate with another device.

### Claim 7 (depends on 1)

The robot of claim 1, further comprising a microphone configured to capture sounds made by the user.

### Claim 8 (depends on 1)

The robot of claim 1, further comprising a speaker configured to emit sounds that relate to the interpretation of the touch.

### Claim 9 (depends on 1)

The robot of claim 1, further comprising a drive system configured to move the robot in a desired direction.

### Claim 10 (depends on 9)

The robot of claim 9, wherein the drive system comprises a holonomic drive system including multiple omni-wheels having rollers that can be rotated to drive the robot in a direction that is perpendicular to a drive direction of the wheel.

### Claim 11 (depends on 1)

The robot of claim 1, further comprising a controller that executes one or more spiking neural network (SNN) algorithms that control the robot's behavior.

### Claim 12 (depends on 11)

The robot of claim 11, wherein the touch sensors are configured to provide spike data representing the user's stroke to the controller, which converts the spike data into SNN inputs.

### Claim 13 (depends on 11)

The robot of claim 11, wherein the robot's behavior depends upon the nature of the user's stroke in terms of one or more of its velocity and direction.

### Claim 14

A method for interacting with a human user, the method comprising:
providing a neuromorphic robot having a continuous curved outer surface, a plurality of trackball touch sensors provided on and extending across the continuous curved outer surface in an array, and a plurality of lights, one light being collocated with each trackball touch sensor and being configured to illuminate when its collocated trackball touch sensor is stroked by a sweeping stroke of the user;
the robot detecting a sweeping stroke of the user's hand across the outer surface with the trackball touch sensors;
the robot encoding signals from the trackball touch sensors with spatiotemporal information that enables decoding for a direction and velocity of the sweeping stroke;
the robot interpreting the spatiotemporal information; and
the robot reacting to the sweeping stroke with feedback for the user.

### Claim 15 (depends on 14)

The method of claim 14, wherein the robot encoding signals comprises the robot encoding the signals as neural spikes that retain the spatiotemporal information and wherein the robot interpreting the spatiotemporal information comprises the robot interpreting the spatiotemporal information using a spiking neural network (SNN) that is capable of learning temporal patterns and therefore discriminate user inputs.

### Claim 16 (depends on 14)

The method of claim 14, wherein the robot reacting comprises one or more of the robot emitting a particular sound, emitting a particular color, and moving in a particular direction.

### Claim 17 (depends on 14)

The method of claim 14, wherein detecting a sweeping stroke of the user's hand comprises detecting the sweeping stroke while the user is playing a game with the robot.

### Claim 18 (depends on 17)

The method of claim 17, wherein the game involves emitting a correct color for the user when the sensors detect the same velocity and direction from the sweeping stroke.

### Claim 19 (depends on 17)

The method of claim 17, wherein the game involves the user moving the robot in a desired direction by controlling the direction of the sweeping stroke.

### Claim 20 (depends on 17)

The method of claim 17, wherein the game involves training the robot to learn a color that predicts a particular sweeping stroke.