# US 11,122,380 B2 · Personal robot enabled surround sound > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,122,380 B2 - Document ID: US11122380B2 - Status: Granted - Kind: B2 - Application number: 15/901,848 - Filed: 2018-02-21 - Published: 2019-03-14 - Granted: 2021-09-14 - Earliest claimed: 2017-09-08 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Sony Interactive Entertainment Inc. - Inventors: Erik Beran - Web page: https://patentgrep.com/patent/US11122380B2 ## CPC - H04S 7/303: {Tracking of listener position or orientation} - G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)} - G05D 1/0282: {generated in a local control room} - H04R 1/323: {for loudspeakers (H04R1/34 and H04R1/40 take precedence)} - H04R 5/02: Spatial or constructional arrangements of loudspeakers - H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} - H04S 7/30: {Control circuits for electronic adaptation of the sound field} - B25J 9/162: {Mobile manipulator, movable base with manipulator arm mounted on it} - B25J 9/1697: {Vision controlled systems} - G05D 1/0231: {using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16)} - H04R 27/00: Public address systems (circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; {monitoring or testing arrangements for public address systems H04R29/007}; amplifiers H03F) - H04R 2201/025: Transducer mountings or cabinet supports enabling variable orientation of transducer of cabinet - H04R 2205/024: Positioning of loudspeaker enclosures for spatial sound reproduction - H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)} - H04S 7/301: {Automatic calibration of stereophonic sound system, e.g. with test microphone} - H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved - Y10S 901/01: Mobile robot - H04S: STEREOPHONIC SYSTEMS - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract Methods and systems are provided for providing one or more sound component to a viewer within a media presentation space by a robot having one or more speakers. A method includes an operation for establishing a connection to a media device that is presenting media content to the viewer, and an operation for capturing input from a plurality of sensors for obtaining a map of the media presentation space. The method additionally provides for identifying a position of the viewer and a speaker configuration in the media presentation space. The method allows the robot to move to a location based on the map of the media presentation space, the position of the viewer and the speaker configuration. Moreover, the method provides an operation for receiving one or more sound components for presentation to the user. ## Claims ### Claim 1 A method for providing one or more sound components to a viewer within a media presentation space by a robot having one or more speakers, the method comprising: establishing a connection with a media device, the media device for presenting media content to a display screen disposed at a viewing location within the media presentation space, the viewing location provides for a separation space between the viewer and the display screen; capturing input from a plurality of sensors for obtaining a map of the media presentation space; identifying, based on the map, a position of the viewer in the media presentation space, a speaker configuration in the media presentation space, and an obstacle in the media presentation space that is in addition to the display screen, the viewer and speakers of the speaker configuration; determining a location the robot is to move to achieve a surround sound configuration within the media presentation space via the one or more speakers of the robot, the location to which the robot moves is one that approximates a location of a sound component visually depicted in the media content relative to a position of the viewer in the media presentation space viewing the media content; receiving, from the media device, one or more sound components associated with the media content; moving the robot to the location, while avoiding the obstacle, so that a sound output by the one or more speakers of the robot, together with the speaker configuration in the media presentation space, appears to correspond to said sound component visually depicted in the media content relative to the position of the viewer in the media presentation space viewing the media content; and outputting the sound by the one or more speakers of the robot when the robot moves to the location; wherein the robot is configured to move to one or more additional locations within the media presentation space responsive to changes of the location of said sound components visually depicted on the display screen for the sound component as corresponding changes occur in the media content, and one of said locations is one that enables outputting the sound in the separation space between the viewer and the display screen. ### Claim 2 (depends on 1) The method of claim 1, wherein the location is further selected such that the one or more sound components during said presenting are unobstructed by the obstacle. ### Claim 3 (depends on 1) The method of claim 1, wherein the one or more sound components are one or more surround sound components, the one or more surround sound components are different from sound components of the speaker configuration. ### Claim 4 (depends on 1) The method of claim 1, wherein the speaker configuration is associated with a 1-channel, 2-channel, or 3-channel non-surround sound configuration, and wherein the one or more sound components that is output by the one or more speakers of the robot are associated with a 5-channel, 6-channel, or 7-channel to achieve the surround sound configuration. ### Claim 5 (depends on 1) The method of claim 1, further comprising: determining that the speaker configuration is of a pre-existing surround sound configuration from said identifying the speaker configuration in the media presentation space; wherein the one or more sound components are one or more additional surround sound components. ### Claim 6 (depends on 1) The method of claim 1, wherein the speaker configuration is associated with 5-channel surround sound configuration, and wherein the one or more sound components that is output by the one or more speakers of the robot are associated with a 6-channel or a 7-channel to achieve the surround sound configuration. ### Claim 7 (depends on 1) The method of claim 1, wherein said presenting the one or more sound components associated with the media content is in synchrony with said presenting the media content to the viewer. ### Claim 8 (depends on 1) The method of claim 1, wherein the location the robot moves to is behind the viewer when the viewer is viewing the display. ### Claim 9 (depends on 1) The method of claim 1, wherein the media presentation space includes a surface that is reflective of sound, and wherein at least one of the one or more sound components are delivered to the viewer by being reflected off of the surface. ### Claim 10 (depends on 1) The method of claim 1, wherein the one or more sound components include a first surround sound component and a second surround sound component. ### Claim 11 A robot for providing one or more sound components to a viewer within a media presentation space, the robot comprising: a communications link configured for establishing a connection with a media device, the media device configured for presenting media content to a display screen disposed at a viewing location within the media presentation space, the viewing location provides for a separation space between the viewer and the display screen; a plurality of sensors for obtaining data associated with the media presentation space, the plurality of sensors including one or more cameras, the data associated with the media presentation space usable for constructing a map of the media presentation space, for identifying a position of the viewer within the media presentation space, for identifying a speaker configuration present in the media presentation space, and for identifying an obstacle within the media presentation space that is in addition to the display screen; a processor configured for processing the data associated with the media presentation space for determining a location within the media presentation space that the robot is to move, the location to which the robot moves is one that approximates a location of a sound component visually depicted in the media content relative to a position of the viewer in the media presentation space viewing the media content; a locomotion unit for moving the robot to the location within the media presentation space, while avoiding the obstacle, so that a sound output by one or more speakers of the robot, together with the speaker configuration in the media presentation space, appears to correspond to said sound component visually depicted in the media content relative to the position of the viewer in the media presentation space viewing the media content; wherein the robot is configured to move to one or more additional locations within the media presentation space responsive to changes of the location of said sound components visually depicted on the display screen for the sound component as corresponding changes occur in the media content, and one of said locations is one that enables outputting the sound in the separation space between the viewer and the display screen. ### Claim 12 (depends on 11) The robot of claim 11, wherein the robot is configured to receive the one or more sound components associated with the media content from media device via the communications link, wherein the robot is configured to present the one or more sound components to the viewer from the location, and wherein the one or more sound components are one or more surround sound components, the one or more surround sound components are different from sound components of the speaker configuration. ### Claim 13 (depends on 11) The robot of claim 11, wherein the location is further determined such that the one or more sound components are unobstructed by the obstacle during said providing the one or more sound components to the viewer. ### Claim 14 (depends on 11) The robot of claim 11, wherein the one or more sound components are one or more additional surround sound components. ### Claim 15 (depends on 11) The robot of claim 11, wherein said providing the one or more components to the viewer occurs in synchrony with said presenting the media content to the viewer. ### Claim 16 (depends on 11) The robot of claim 11, wherein the processor is further configured to determine respective orientations for the one or more speakers based on the data associated with the media presentation space, and wherein the one or more speakers are configured to move to the respective orientations for said providing the one or more sound components. ### Claim 17 A method for providing one or more sound components to a viewer within a media presentation space by a robot having one or more speakers, comprising: establishing a connection with a media device, the media device for presenting media content to a display screen disposed at a viewing location within the media presentation space and being in connection with one or more existing speakers for presenting a first audio component to the viewer, the viewing location provides for a separation space between the viewer and the display screen; capturing input from a plurality of sensors for identifying a position of the viewer, the one or more existing speakers and an obstacle in the media presentation space that is in addition to the display screen, the viewer and the one or more speakers; determining a location within the media presentation space the robot is to move to provide a second audio component to the viewer, that augments the first audio component to provide a three-dimensional sound environment, the location to which the robot moves is one that approximates a location of a sound component visually depicted in the media content relative to a position of the viewer in the media presentation space viewing the media content, receiving, from the media device, the second audio component; moving the robot to the location; and outputting at least part of the second audio component by the one or more speakers of the robot when the robot has moved to the location; wherein the robot is configured to move to one or more additional locations within the media presentation space responsive to changes of the location of said sound components visually depicted on the display screen for the sound component as corresponding changes occur in the media content, and one of said locations is one that enables outputting the sound in the separation space between the viewer and the display screen. ### Claim 18 (depends on 17) The method of claim 17, wherein the media presentation space includes a wall that is reflective of sound, and wherein the second audio component is delivered to the viewer by being reflected off of the wall. ### Claim 19 (depends on 17) The method of claim 17, wherein the location is further determined such that the second audio component is unobstructed by the obstacle during said presenting.