# US 10,766,144 B2 · Map related acoustic filtering by a mobile robot

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,766,144 B2
- Document ID: US10766144B2
- Status: Granted
- Kind: B2
- Application number: 15/924,145
- Filed: 2018-03-16
- Published: 2019-07-11
- Granted: 2020-09-08
- Earliest claimed: 2018-01-08
- Assignees: Digital Dream Labs, LLC
- Applicants: Digital Dream Labs, LLC
- Inventors: Daniel Thomas Casner; Lee Crippen; Hanns W. Tappeiner; Anthony Armenta; Kevin Yoon
- Web page: https://patentgrep.com/patent/US10766144B2

## CPC

- B25J 13/003: {by means of an audio-responsive input (audible safety signals B25J19/061)}
- B25J 5/005: {mounted on endless tracks or belts}
- B25J 9/162: {Mobile manipulator, movable base with manipulator arm mounted on it}
- B25J 9/1694: {characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion}
- B25J 13/081: {Touching devices, e.g. pressure-sensitive}
- B25J 13/089: {Determining the position of the robot with reference to its environment}
- G01S 7/52004: {Means for monitoring or calibrating (short-range imaging G01S7/5205)}
- G01S 15/89: for mapping or imaging
- G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00)
- G05D 1/0231: {using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16)}
- G05D 1/0255: {using acoustic signals, e.g. ultra-sonic singals (sonar systems designed for anti-collision purposes G01S15/93)}
- G05D 1/027: {comprising intertial navigation means, e.g. azimuth detector (inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165)}
- G05D 1/0291: {Fleet control (monitoring fleets in traffic control systems for road vehicles G08G1/127, G08G1/127)}
- G10L 21/02: Speech enhancement, e.g. noise reduction or echo cancellation (reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08)
- H04R 1/028: {associated with devices performing functions other than acoustics, e.g. electric candles}
- H04R 1/406: {microphones}
- H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)}
- H04R 3/04: for correcting frequency response
- H04S 7/30: {Control circuits for electronic adaptation of the sound field}
- G01S 3/80: using ultrasonic, sonic or infrasonic waves
- G10L 2021/02166: {Microphone arrays; Beamforming}
- H04R 29/00: Monitoring arrangements; Testing arrangements {(for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308)}
- H04S 2400/15: Aspects of sound capture and related signal processing for recording or reproduction
- Y10S 901/01: Mobile robot
- Y10S 901/46: Sensing device
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- H04S: STEREOPHONIC SYSTEMS
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

This specification relates to robots and audio processing in robots. One aspect of the subject matter includes: a body and one or more physically moveable components; a plurality of microphones; one or more processors; and one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations. The operations can include: obtaining map data of an environment of the robot; selecting a test location from the map data; navigating to the selected test location; receiving a sound wave propagating through the environment of the robot; computing one or more acoustic transfer functions for the test signal, wherein each acoustic transfer function represents how the test signal was transformed by the environment of the robot during its propogation through the environment of the robot; and storing each of the transfer functions in association with the test location.

## Claims

### Claim 1

A robot comprising:
a body and one or more physically moveable components;
a plurality of microphones;
one or more processors; and
one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations comprising:
obtaining map data of an environment of the robot;
selecting a test location from the map data;
navigating to the selected test location;
receiving a test signal of a sound wave propagating through the environment of the robot;
computing one or more acoustic transfer functions for the test signal, wherein each acoustic transfer function represents how the test signal was transformed by the environment of the robot while the sound wave propagated through the environment of the robot; and
storing each of the transfer functions in association with the test location.

### Claim 2 (depends on 1)

The robot of claim 1, wherein the operations further comprise:
receiving, by the robot, an original audio signal at a particular location in the environment of the robot;
obtaining one of the previously computed acoustic transfer functions associated with the particular location; and
computing a recovered audio signal by applying the previously computed acoustic transfer function for the particular location to the received original audio signal.

### Claim 3 (depends on 2)

The robot of claim 2, wherein obtaining one of the previously computed acoustic transfer functions associated with the particular location comprises determining a previous test location that is closest in the environment of the robot to the test location.

### Claim 4 (depends on 1)

The robot of claim 1, wherein the operations further comprise:
repeatedly and automatically selecting new test locations in the environment of the robot;
computing one or more additional acoustic transfer functions for each new test location; and
storing each additional acoustic transfer function in association with each respective new test location.

### Claim 5 (depends on 4)

The robot of claim 4, wherein selecting a new test location comprises:
computing, for each of a plurality of areas in the environment of the robot, a respective measure of density of previously tested locations; and
selecting a particular location in an area of the environment having a density that satisfies a threshold.

### Claim 6 (depends on 1)

The robot of claim 1, wherein the test signal is emitted by the robot.

### Claim 7 (depends on 1)

The robot of claim 1, wherein the test signal is emitted by a different second robot or a stationary sound emitter.

### Claim 8

A robot comprising:
a body and one or more physically moveable components;
a plurality of microphones;
one or more processors; and
one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations comprising:
determining a current location of the robot in an environment of the robot;
determining that the current location is associated with one or more location-specific audio processing techniques;
obtaining, by the plurality of microphones, respective audio signals of sound waves propagating in the environment of the robot; and
processing the audio signals according to the one or more location-specific audio processing techniques.

### Claim 9 (depends on 8)

The robot of claim 8, wherein the one or more location-specific audio processing techniques comprise a location-specific acoustic filter.

### Claim 10 (depends on 9)

The robot of claim 9, wherein determining that the current location of the robot is associated with one or more location-specific audio processing techniques comprises determining that the current location of the robot is within a threshold distance of a wall in the environment of the robot, and
wherein the location-specific acoustic filter amplifies audio signals of sound waves reflecting off the wall.

### Claim 11 (depends on 9)

The robot of claim 9, wherein determining that the current location of the robot is associated with one or more location-specific audio processing techniques comprises determining that the current location of the robot is in a hallway of the environment of the robot, and
wherein the location-specific acoustic filter amplifies audio signals of sound waves traveling along the hallway.

### Claim 12 (depends on 8)

The robot of claim 8, where the one or more location-specific audio processing techniques comprise a location-specific acoustic transfer function.

### Claim 13 (depends on 12)

The robot of claim 12, wherein determining that the current location of the robot is associated with one or more location-specific audio processing techniques comprises determining that the current location of the robot is in a dynamic area having obstacles that are likely to change orientation, and
wherein the location-specific acoustic transfer function is an object-specific transfer function that represents how sound waves are transformed by a particular object.

### Claim 14 (depends on 13)

The robot of claim 13, wherein the object-specific transfer function represents how sound waves are transformed by a closed door or an open door.

### Claim 15 (depends on 13)

The robot of claim 13, wherein the object-specific transfer function represents how sound waves are transformed by furniture in the environment of the robot.

### Claim 16

A robot comprising:
a body and one or more physically moveable components;
a plurality of microphones;
one or more processors; and
one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations comprising:
receiving, by the robot, one or more audio signals at a particular location in an environment of the robot;
selecting, based on the particular location of the robot, a plurality of precomputed acoustic transfer functions from a database of acoustic transfer functions that associates each acoustic transfer function with a respective location in the environment of the robot;
computing a single composite acoustic transfer function from the plurality of selected acoustic transfer functions; and
processing the received one or more audio signals using the composite acoustic transfer function.

### Claim 17 (depends on 16)

The robot of claim 16, wherein selecting the plurality of precomputed acoustic transfer functions comprises determining which acoustic transfer functions are associated with locations in the environment that are closest to the location of the robot.

### Claim 18 (depends on 16)

The robot of claim 16, wherein computing a single composite acoustic transfer function from the plurality of selected acoustic transfer functions comprises assigning a respective weight to each acoustic transfer function based on a respective distance between a location associated with the acoustic transfer function and the particular location of the robot.

### Claim 19 (depends on 16)

The robot of claim 16, wherein the acoustic transfer functions in the database of acoustic transfer functions are precomputed by the robot moving to multiple locations within the environment of the robot.

### Claim 20 (depends on 19)

The robot of claim 19, wherein the operations further comprise:
generating the database of acoustic transfer functions, including repeatedly performing operations comprising:
selecting a test location from map data of the environment of the robot,
navigating to the selected test location,
receiving a test signal of a sound wave propagating through the environment of the robot,
computing one or more acoustic transfer functions for the test signal, wherein each acoustic transfer function represents how the test signal was transformed by the environment of the robot while the sound wave propagated through the environment of the robot, and
storing each of the transfer functions in association with the test location in the database.

### Claim 21 (depends on 16)

The robot of claim 16, wherein the test signal is emitted by the robot.

### Claim 22 (depends on 16)

The robot of claim 16, wherein the test signal is emitted by a different second robot or a stationary sound emitter.

### Claim 23

A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving data representing a plurality of transfer functions that each represents how a sound wave was transformed by the environment of a robot while the sound wave propagated from a respective sound origin in the environment of the robot to a respective sound destination in the environment of the robot;
selecting a plurality of sound origins associated with one of the plurality of transfer functions;
computing a plurality of candidate locations in the environment of the robot;
evaluating each candidate location of the plurality of candidate locations, including:
computing a respective origin score for each selected sound origin of the plurality of selected sound origins, each respective score representing a decrease in sound quality of a sound wave propagating from the selected sound origin to the candidate location according to a particular transfer function having the candidate location as a sound destination; and
computing an aggregate score for the candidate location based on the origin scores computed between the selected sound origins and the candidate location;
determining a candidate location of the plurality of candidate locations having a best aggregate score; and
providing, to the robot, data that causes the robot to navigate within the environment of the robot to the candidate location having the best aggregate score.

### Claim 24 (depends on 23)

The system of claim 23, wherein each origin score represents a measure of volume change or a measure of distortion according to a particular transfer function.

### Claim 25 (depends on 23)

The system of claim 23, wherein providing data that causes the robot to navigate within the environment of the robot to the candidate location having the best aggregate score comprises providing data identifying the candidate location.

### Claim 26 (depends on 23)

The system of claim 23, wherein providing the data causes the robot to navigate to the candidate location upon receiving a downtime trigger.

### Claim 27 (depends on 23)

The system of claim 23, wherein the plurality of transfer functions were computed automatically by the robot navigating within the environment of the robot.

### Claim 28 (depends on 23)

The system of claim 23, wherein the one or more computers and one or more storage devices are installed on the robot, and wherein the robot comprises a body and one or more physically moveable components.

### Claim 29 (depends on 23)

The system of claim 23, wherein selecting a plurality of sound origins associated with one of the plurality of transfer functions comprises selecting peripheral locations within the environment of the robot.

### Claim 30 (depends on 23)

The system of claim 23, wherein selecting a plurality of sound origins associated with one of the plurality of transfer functions comprises selecting locations associated with entryways, doors, or windows.

### Claim 31 (depends on 23)

The system of claim 23, the operations further comprising:
determining at least one of a pose and orientation in order to improve its ability to process audio of interest.