Back to results
US 11,641,543 B2Granted
Sound source localization for robot
AssigneeLG Electronics Inc.
InventorKilsu Jung
Life of this patent
Estimating the term…
Abstract
Disclosed are a robot which localizes a sound source based on sound spot information of at least one sound spot, and a sound source localization method. The robot can communicate with other electronic devices and a server in a 5G communication environment.
Claims11
A robot, comprising:
- a memory configured to store a sound spot information associated with at least one sound spot and to store an object recognition model based on an artificial neural network, wherein the sound spot information comprises distribution information, sound spot score, sound spot attribute and an image of the at least one sound spot;
- and
- a processor configured to:
- determine, based at least in part on the distribution information of the at least one sound spot, a search area to search for a sound source that has emitted an input sound, wherein the sound spot information corresponds to a potential spatial location capable of emitting the input sound;
- search for the at least one sound spot located in the determined search area according to a search sequence, wherein the search sequence is determined based at least in part on the stored sound spot score of the at least one sound spot;
- determine a target sound spot among the at least one sound spot based at least in part on a result of the searching, wherein the target sound spot corresponds to a location of the sound source;
- and
- update the stored sound spot score of the determined target sound spot, wherein the processor is further configured to:
- obtain an image of the determined search area;
- extract at least one image area corresponding to each of the at least one sound spot from the obtained image based at least in part on the stored image of the at least one sound spot and
- determine the target sound spot among the extracted at least one image area using the object recognition model based at least in part on the stored image of the at least one sound spot.
8 more elements
A method for localizing a sound source, comprising:
- storing, in a memory, sound spot information associated with at least one sound spot and an object recognition model based on an artificial neural network, wherein the sound spot information comprises distribution information, sound spot score, sound spot attribute and an image of the at least one sound spot;
- determining, based at least in part on the distribution information of the at least one sound spot, a search area to search for the sound source that has emitted an input sound, wherein the sound spot information corresponds to a potential spatial location capable of emitting the input sound, wherein the sound spot information is associated with at least one sound spot;
- searching for the at least one sound spot located in the determined search area according to a search sequence, wherein the search sequence is determined based at least in part on the stored sound spot score of the at least one sound spot, wherein the searching for the at least one sound spot comprises:
- causing a camera to orient toward the determined search area,
- scanning, by the camera, the determined search area to obtain an image of the determined search area, and
- extracting at least one image area corresponding to each of the at least one sound spot from the obtained image of the determined search area based at least in part on the stored image of the at least one sound spot;
- determining a target sound spot among the at least one sound spot based at least in part on a result of the searching, wherein the target sound spot is determined by determining the target sound spot among the extracted at least one image area using the object recognition model based at least in part on the stored image of the at least one sound spot, wherein the target sound spot corresponds to a location of the sound source;
- and
- updating the sound spot score of the determined target sound spot.
6 more elements
Dependencies are inferred from the claim text.