# US 11,298,824 B2 · Robot system and control method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,298,824 B2 - Document ID: US11298824B2 - Status: Granted - Kind: B2 - Application number: 16/577,814 - Filed: 2019-09-20 - Published: 2020-12-31 - Granted: 2022-04-12 - Earliest claimed: 2019-06-28 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Ji Yoon Park - Web page: https://patentgrep.com/patent/US11298824B2 ## CPC - G06Q 10/0832: Special goods or special handling procedures, e.g. handling of hazardous or fragile goods - B25J 9/0084: {comprising a plurality of manipulators} - B25J 9/1602: {characterised by the control system, structure, architecture} - B25J 9/1661: {characterised by task planning, object-oriented languages} - B25J 9/1679: {characterised by the tasks executed} - B25J 9/1697: {Vision controlled systems} - B25J 11/008: {Manipulators for service tasks} - B25J 19/026: {Acoustical sensing devices} - G05D 1/021: {specially adapted to land vehicles} - G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching} - G06V 10/82: using neural networks - G06V 20/20: in augmented reality scenes - G06V 40/16: Human faces, e.g. facial parts, sketches or expressions - G06V 40/172: {Classification, e.g. identification} - G10L 15/08: Speech classification or search - G10L 17/00: Speaker identification or verification techniques - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Disclosed are a robot system and a control method for the same. The robot includes a first server, a first robot registered on the first server and configured to deliver goods to a client according to information received from the first server, a second robot configured to receive the delivery goods from the first robot, and a second server having the second robot registered thereon and being configured to operate the second robot. The control method for the robot system may include: confirming, by the first robot, whether a person is present in a surrounding area of delivery destination; when someone is present, acquiring, by the first robot, voice information and image information of the present person and transferring the same to the second robot; transferring, by the second robot, the voice information and the image information of the person to the second server; recognizing, by the second server, a person from at least one of the voice information and the image information of the person transferred from the second robot, and transferring a recognition result to the second robot; confirming, by the second robot, whether the present person is a family member of the client, based on the recognition result of the person transferred from the second server; and when the present person is a family member, transferring, by the first robot, the delivery goods to the present person. ## Claims ### Claim 1 A control method for a robot delivery system comprising: determining, via a first robot, whether a person is present at a surrounding area of a delivery destination, wherein the first robot is tasked with delivering a package to a client at the delivery destination; acquiring, via the first robot, at least voice information or image information of a present person based on a determination that the present person is present at the surrounding area of the delivery destination, transmitting the at least voice information or image information from the first robot to a second robot operating at the delivery destination; transmitting to a server, by the second robot, the at least voice information or image information; performing recognition, by the server, based on the at least voice information or image information received from the second robot; transmitting a recognition result from the server to the second robot; and confirming, by the second robot, whether the present person is a family member or an acquaintance of the client based on the recognition result received from the server. ### Claim 2 (depends on 1) The control method of claim 1, further comprising delivering, by the first robot, the package to the present person when the present person is confirmed as a family member of the client by the second robot. ### Claim 3 (depends on 1) The control method of claim 1, wherein the server is configured to utilize an artificial intelligence module, wherein the artificial intelligence module is configured to determine a similarity between voice information received from the second robot with stored voice data and to output information of a corresponding person when a degree of similarity between the voice information and one of the stored voice data is greater than or equal to a set value. ### Claim 4 (depends on 1) The control method of claim 1, wherein the server is configured to utilize an artificial intelligence module, wherein the artificial intelligence module is configured to determine a similarity between image information received from the second robot with stored image data, and to output information of a corresponding person when a degree of similarity between the image information and one of the stored image data is greater than or equal to a set value. ### Claim 5 (depends on 1) The control method of claim 1, wherein when it is determined that no person is present at the surrounding area of the delivery destination, the method further comprises: transmitting, by the first robot: a request to the second robot to open a door lock of the delivery destination; and product information and arrival information of the package. ### Claim 6 (depends on 5) The control method of claim 5, further comprising: opening the door lock by the second robot in response to the request to open the door lock; transmitting, by the second robot, a notification to the client of the opening of the door lock; and delivering, by the first robot, the package to the client. ### Claim 7 (depends on 1) The control method of claim 1, wherein when it is confirmed that the present person is a family member or acquaintance of the client, the method further comprises transmitting, by the second robot, personal information of the present person to the client. ### Claim 8 (depends on 1) The control method of claim 1, wherein when the second robot confirms that the present person is not a family member or an acquaintance of the client, the method further comprises: transmitting, by the second robot, image information of the present person to the client; receiving, by the second robot, a response as to whether the present person is an acquaintance of the client; and receiving, by the second robot, information regarding a delivery option based on the response. ### Claim 9 (depends on 8) The control method of claim 8, wherein when the second robot confirms that the present person is an acquaintance of the client based on the received response, the method further comprises storing, by the second robot, information of the present person and transmitting a request to the server to store the information of the present person. ### Claim 10 (depends on 8) The control method of claim 8, wherein when the received information regarding the delivery option comprises a command to open a door lock of the delivery destination, the method further comprises: opening the door lock by the second robot; transmitting, by the second robot, a notification to the client of the opening of the door lock; and delivering, by the first robot, the package to the client. ### Claim 11 (depends on 5) The control method of claim 5, further comprising: opening the door lock, by the second robot, in response to the request to open the door lock; transmitting, by the second robot, a notification to a user terminal of the opening of the door lock, wherein the second robot is registered with the user terminal; and delivering, by the first robot, the package to the second robot or unloading the package at the delivery destination. ### Claim 12 (depends on 1) The control method of claim 1, wherein when the present person is confirmed as not being a family member of the client, the method further comprises determining, by the second robot, whether the client has requested in advance to be notified when a non-family person is present at the surrounding area of the delivery destination. ### Claim 13 (depends on 12) The control method of claim 12, wherein when it is determined that the client has not requested in advance to be notified, the method further comprises: transmitting, by the second robot, a request to the first robot to search the surrounding area of the delivery destination to determine again whether a person is still present at the surrounding area; and transmitting, by the first robot, additional voice information or additional image information of the surrounding area of the delivery destination to the second robot. ### Claim 14 (depends on 13) The control method of claim 13, further comprising: confirming, by the second robot, that no one is present at the surrounding area of the delivery destination based on the additional voice information or additional image information transmitted by the first robot; transmitting, by the first robot, a request to the second robot to open a door lock of the delivery destination and product information and arrival information of the package; delivering, by the first robot, the package to the second robot or unloading the package at the delivery destination; and transmitting, by the second robot, a notification to a user terminal of opening of the door lock and delivery of the package, wherein the second robot is registered with the user terminal. ### Claim 15 (depends on 13) The control method of claim 13, further comprising: confirming, by the second robot, that a person is still present at the surrounding area of the delivery destination based on the additional voice information or the additional image information transmitted by the first robot; transmitting, by the second robot, a request to the first robot to unload the package at another location; and transmit, by the second robot, a notification to a user terminal of delivery of the package to the another location, wherein the second robot is registered with the user terminal. ### Claim 16 (depends on 12) The control method of claim 12, wherein when it is determined that the client requested in advance to be notified, the method further comprises: transmitting, by the second robot, the at least image information of the present person to a user terminal, wherein the second robot is registered with the user terminal; determining, at the user terminal, whether the present person is an acquaintance of the client; and transmitting, by the user terminal, information to the second robot regarding a delivery option based on the determination of whether the present person is an acquaintance. ### Claim 17 (depends on 16) The control method of claim 16, further comprising transmitting, by the user terminal, requests to the second robot and the server to store information of the present person when it is determined that the present person is an acquaintance of the client.