Method for controlling cook based on artificial intelligent and intelligent device

A method for controlling cooking at a kitchen appliance based on artificial intelligence, the method comprising:

• acquiring, before the cooking, first food ingredient image information through a plurality of cameras located inside a cooking chamber of the kitchen appliance where food ingredients are placed,
• wherein the plurality of cameras comprise:
• a first camera located to acquire a first image of a top surface of the food ingredients;
• a second camera located to acquire a second image of a left surface of the food ingredients;
• and
• a third camera located to acquire a third image of a right surface of the food ingredients;
• transmitting the first food ingredient image information to an external server;
• receiving reference information including image information of completed dishes from the external server,
• wherein the reference information including the image information of completed dishes is based at least on a fourth image comprising stereoscopic image information about the food ingredients, the fourth image generated by combining overlapping portions of the first image, the second image and the third image and uncaptured portions generated by predicting pixel information based on information of the first image, the second image and the third image;
• acquiring, during the cooking, second food ingredient image information through the plurality of cameras;
• transmitting the second food ingredient image information to the external server;
• determining a cooked state by comparing the second food ingredient image information with the image information of the completed dishes;
• transmitting a message including information indicating the determined cooked state to a mobile terminal;
• generating a message recommending to change control information depending on the determined cooked state;
• and
• transmitting the message recommending to change the control information to the mobile terminal,
• wherein the message recommending to change the control information comprises information indicating a current cooked state, information indicating a cooking result expected to be yielded if no change is made, and recommended control information.

An artificial intelligence device comprising:

• an antenna configured to receive, from a network, first food ingredient image information which is acquired before cooking through a plurality of cameras located inside a cooking chamber of a kitchen appliance,
• wherein the plurality of cameras comprise:
• a first camera located to acquire a first image of a top surface of food ingredients;
• a second camera located to acquire a second image of a left surface of the food ingredients;
• and
• a third camera located to acquire a third image of a right surface of the food ingredients;
• a memory;
• and
• a processor,
• wherein the processor is configured to:
• combine the first image, the second image and the third image of the first food ingredient image information to produce a fourth image comprising stereoscopic image information about the food ingredients, by combining overlapping portions of the first image, the second image and the third image and uncaptured portions generated by predicting pixel information based on information of the first image, the second image and the third image;
• and
• generate reference information including image information of completed dishes by applying the first food ingredient image information to a pre-trained artificial neural network model and applying the stereoscopic image information of the fourth image to the pre-trained artificial neural network model,
• wherein the antenna is further configured to receive, from the network, second food ingredient information which is acquired through the plurality of cameras,
• wherein the processor is further configured to determine a cooked state by comparing the second food ingredient image information with the image information of the completed dishes,
• wherein the processor is further configured to:
• transmit the reference information or a message including the determined cooked state to the kitchen appliance or a mobile terminal;
• generate a message recommending to change control information depending on the determined cooked state;
• and
• transmit the message recommending to change the control information to the kitchen appliance or the mobile terminal, and
• wherein the message recommending to change the control information comprises information indicating a current cooked state, information indicating a cooking result expected to be yielded if no change is made, and recommended control information.