Real-time automated cooking cycles using computer vision and deep learning
AssigneeWhirlpool Corporation
InventorsMichael B. DALY, Blake W. EHRENBECK, Mohammad HAGHIGHAT, Seth HERNDON, Bhargav M. JAIN, Jason Y. LEE, Saqib Nizam SHAMSI, Sunil Suresh SHASTRI, Bruce M. WIATRAK
Life of this patent
Estimating the term…
Abstract
A food class of a food item is recognized. A target doneness score is defined for the food item based on the food class and a desired doneness level. A recurrent model is utilized to determine a current doneness score of the food item according to sensor data captured of the food item. The current doneness score and the target doneness score are utilized to control a heating system to cook the food item.
Claims21
A method for controlling a heating process, comprising:
- identifying, by a processor of a smart oven, a food class of a food item;
- selecting, by the processor, a set of sensor modalities from a plurality of available sensor modalities of the smart oven based on the food class;
- capturing sensor data of the food item over time during a cooking cycle using the selected set of sensor modalities;
- applying, by the processor, the sensor data as input to a recurrent model configured to process the sensor data over time to determine a current doneness score of the food item during the cooking cycle;
- and
- controlling, by the processor, a heating system of the smart oven based on the current doneness score and a target doneness score associated with a desired doneness level for the food item.
A smart oven system for controlling a heating process, comprising:
- a plurality of sensors including sensors of different modality types;
- a heating system;
- and
- a processor programmed to:
- identify a food class of a food item,
- select a set of sensor modalities from the plurality of sensors based on the food class,
- capture sensor data of the food item over time during a cooking cycle using the selected set of sensor modalities,
- apply the sensor data as input to a recurrent model configured to process the sensor data over time to determine a current doneness score of the food item during the cooking cycle, and
- control the heating system based on the current doneness score and a target doneness score associated with a desired doneness level for the food item.
A non-transitory computer readable medium comprising instructions for controlling a heating process of a smart oven having a plurality of sensors of different modality types, a heating system, and a processor that, when executed by the processor, cause the smart oven to perform operations including to:
- identify a food class of a food item;
- select a set of sensor modalities from the plurality of sensors based on the food class;
- capture sensor data of the food item over time during a cooking cycle using the selected set of sensor modalities;
- apply the sensor data as input to a recurrent model configured to process the sensor data over time to determine a current doneness score of the food item during the cooking cycle;
- and
- control the heating system based on the current doneness score and a target doneness score associated with a desired doneness level for the food item.
Dependencies are inferred from the claim text.