Concierge robot system, concierge service method, and concierge robot

A concierge robot comprising:

• a main body having a robot shape;
• a camera that is provided in the main body to capture an external image;
• a microphone that is provided in the main body to receive an external voice;
• a speaker that is provided in the main body to output a voice;
• a display device that is provided in the main body to output data on a screen;
• an input/output port that is provided in the main body and connected with a communication network and an external device;
• a point of sales (POS) device that is provided in the main body and processes an order of a product or a service according to a result of a voice recognition, recognizes a payment means in a contact or non-contact manner, performs a payment process with a payment server that is connected with the communication network, and outputs details of the order and payment;
• a light emission unit that is provided in the main body and includes at least one light emitting diode (LEDs) that is driven according to a control signal;
• an actuator driver that is provided in the main body and generates a driving signal;
• an operation member that is provided in the main body to rotate within a range of a predetermined rotation angle or move in a predetermined direction by being driven according to the driving signal received from the actuator driver;
• a short-distance communication device that is provided in the main body and performs short-distance wireless communication with a user terminal;
• a memory that is provided in the main body to store a program that controls operation of the concierge robot based on learning data generated by using a neural network model;
• and
• a processor that executes the program, wherein the program includes instructions for:
• recognizing a face image from the external image input from the camera, recognizing an emotion that corresponds to the face image using a convolutional neural network (CNN) model,
• generating emotion expression data including an emoticon or graphic data and outputting the generated emotion expression data to the display device,
• outputting a signal that controls at least one of light emission time, a color of light emission, and a light emission amount according to the generated emotion expression data to the light emission unit,
• generating a first conversation sentence that corresponds to a natural language of web data externally collected through web scraping based on the learning data and outputting the first conversation sentence by the voice,
• recognizing the external voice received from the microphone, generating a second conversation sentence corresponding to an answer according to a recognition result of the external voice and outputting the second conversation sentence by the voice,
• requesting the POS device to process the order of the product or and service according to the recognition result of the external voice and collecting the details of the order and payment from the POS device and storing the details as user personal data,
• generating a relationship between the user personal data, emotion learning data and customer relationship management (CRM) data as a neural network and generating a user recommendation data including menu recommendation data or promotion information by backward propagating the neural network,
• outputting the user recommendation data to the display device,
• processing the user recommendation data into the natural language and outputting a corresponding third conversation sentence by the voice,
• processing the generated emotion expression data and the user recommendation data, learning a fourth conversation sentence that corresponds to the natural language by using the neural network model, and generating the fourth conversation sentence that corresponds to each the emotion expression data and the user recommendation data by using a result of the learning data generated by using the neural network model, broadcasting a concierge service providing URL for providing marketing information that includes at least one of service recommendation information, promotion information and advertisement information derived by matching the user personal data and the CRM data through a short-distance communication device,
• processing a concierge service data into the natural language and outputting a corresponding fifth conversation sentence by the voice,
• wherein the concierge service data is defined by a model, view, and controller (MVC) and comprises at least one of membership guide information, a personal information usage agreement request information, service shop information, service menu information, a waiting number guide information, a product information or service order information, and an online payment service information,
• the concierge service providing URL comprising at least one of an access address of a concierge service server, an access address of a web space and an access address of a storage device, and the user terminal that selects the concierge service providing URL receives the concierge service data from the access address.