# US 10,504,384 B1 · Augmented reality user engagement system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,504,384 B1 - Document ID: US10504384B1 - Status: Granted - Kind: B1 - Application number: 16/158,372 - Filed: 2018-10-12 - Granted: 2019-12-10 - Earliest claimed: 2018-10-12 - Assignees: Haier US Appliance Solutions, Inc. - Applicants: Haier US Appliance Solutions, Inc. - Inventors: Jeff Donald Drake - Web page: https://patentgrep.com/patent/US10504384B1 ## CPC - G09B 19/0092: {Nutrition} - F24C 7/082: {on ranges, e.g. control panels, illumination} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G09B 19/003: {Repetitive work cycles; Sequence of movements} - G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - F24C: DOMESTIC STOVES OR RANGES (exclusively for solid fuels F24B); DETAILS OF DOMESTIC STOVES OR RANGES, OF GENERAL APPLICATION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A user engagement system and methods for use thereof in an augmented reality context are provided herein. The user engagement system may include a camera assembly, an image monitor, and a controller. The camera assembly may be directed at a cooking zone. The image monitor may be directed at an area forward from the cooking zone. The controller may be in operative communication with the camera assembly and the image monitor. The controller may be configured to initiate a directed cooking operation. The directed cooking operation may include receiving a video signal of the cooking zone from the camera assembly, presenting a real-time feed of the cooking zone at the image monitor, overlaying a dynamic figure onto the real-time feed of the cooking zone at the image monitor, and animating the dynamic figure over the real-time feed based on a selected recipe. ## Claims ### Claim 1 A user engagement system comprising: a camera assembly directed at a cooking zone; an image monitor directed at an area forward from the cooking zone; and a controller in operative communication with the camera assembly and the image monitor, wherein the controller is configured to initiate a directed cooking operation, the directed cooking operation comprising: receiving a video signal of the cooking zone from the camera assembly, presenting a real-time feed of the cooking zone at the image monitor according to the received video signal, overlaying a dynamic figure onto the real-time feed of the cooking zone at the image monitor, and animating the dynamic figure over the real-time feed based on a selected recipe, receiving a recipe information signal from a remote server based on the selected recipe prior to animating the dynamic figure, wherein the recipe information signal includes animation instructions for animating the dynamic figure. ### Claim 2 (depends on 1) The user engagement system of claim 1, wherein the cooking zone is a cooktop of a cooking appliance, and wherein the cooktop comprises a heating element for receiving a utensil thereon. ### Claim 3 (depends on 2) The user engagement system of claim 2, further comprising detecting a heat setting at the heating element, wherein animating the dynamic figure includes animating the dynamic figure as illustrating manipulation of the heating element based on the detected heat setting. ### Claim 4 (depends on 2) The user engagement system of claim 2, wherein animating comprises displaying the dynamic figure over a portion of the real-time feed at which the heating element is located. ### Claim 5 (depends on 2) The user engagement system of claim 2, wherein animating comprises displaying the dynamic figure over a portion of the real-time feed at which the utensil is located, and wherein animating the dynamic figure comprises simulating engagement between the dynamic figure and the utensil. ### Claim 6 (depends on 5) The user engagement system of claim 5, wherein animating further comprises animating the dynamic figure as illustrating a user action of the selected recipe. ### Claim 7 (depends on 1) The user engagement system of claim 1, wherein the camera assembly is positioned directly above the cooking zone. ### Claim 8 (depends on 1) The user engagement system of claim 1, wherein the image monitor is positioned directly above the cooking zone. ### Claim 9 A method of operating a user engagement system adjacent to a cooking zone, the user engagement system comprising a camera assembly and an image monitor, the method comprising: receiving a video signal of the cooking zone from the camera assembly; presenting a real-time feed of the cooking zone at the image monitor according to the received video signal; overlaying a dynamic figure onto the real-time feed of the cooking zone at the image monitor; animating the dynamic figure over the real-time feed based on a selected recipe; and receiving a recipe information signal from a remote server based on the selected recipe prior to animating the dynamic figure, wherein the recipe information signal includes animation instructions for animating the dynamic figure. ### Claim 10 (depends on 9) The method of claim 9, wherein the cooking zone is a cooktop of a cooking appliance, and wherein the cooktop comprises a heating element for receiving a utensil thereon. ### Claim 11 (depends on 10) The method of claim 10, further comprising detecting a heat setting at the heating element, wherein animating the dynamic figure includes animating the dynamic figure as illustrating manipulation of the heating element based on the detected heat setting. ### Claim 12 (depends on 10) The method of claim 10, wherein animating comprises displaying the dynamic figure over a portion of the real-time feed at which the heating element is located. ### Claim 13 (depends on 10) The method of claim 10, wherein animating comprises displaying the dynamic figure over a portion of the real-time feed at which the utensil is located, and wherein animating the dynamic figure comprises simulating engagement between the dynamic figure and the utensil. ### Claim 14 (depends on 13) The method of claim 13, wherein animating further comprises animating the dynamic figure as illustrating a user action of the selected recipe. ### Claim 15 (depends on 9) The method of claim 9, wherein the camera assembly is positioned directly above the cooking zone. ### Claim 16 (depends on 9) The method of claim 9, wherein the image monitor is positioned directly above the cooking zone.