# US 2014/0192182 A1 · Method for viewing virtual objects within an appliance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0192182 A1 - Document ID: US20140192182A1 - Status: Published application - Kind: A1 - Application number: 13/738,206 - Filed: 2013-01-10 - Published: 2014-07-10 - Earliest claimed: 2013-01-10 - Assignees: General Electric Company - Applicants: General Electric Company - Inventors: Keith Wesley Wait - Web page: https://patentgrep.com/patent/US20140192182A1 ## CPC - G06Q 30/00: Commerce - H04M 1/72403: with means for local support of applications that increase the functionality - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract A method for viewing virtual objects within or on an appliance is provided. The method includes recording an image of the appliance and a positional reference disposed on a surface of the appliance. The method also includes generating a modification of the image. The modification illustrates the appliance and a virtual object disposed adjacent the positional reference. ## Claims ### Claim 1 A method for viewing virtual food items within a refrigerator appliance, the refrigerator appliance having a housing that defines a chilled chamber for receipt of food items for storage and a positional reference disposed within the chilled chamber of the refrigerator appliance, the method comprising: receiving an image of the chilled chamber of the refrigerator appliance and the positional reference disposed within the chilled chamber from a camera; determining a position and an orientation of the positional reference within the image; generating a modification of the image, the modification illustrating the chilled chamber of the refrigerator appliance and a virtual food item disposed adjacent the positional reference within the chilled chamber; and presenting the modification of the image on a display device. ### Claim 2 (depends on 1) The method of claim 1, wherein said step of receiving comprises directing a camera of a mobile device towards the chilled chamber of the refrigerator appliance and the positional reference disposed within the chilled chamber and utilizing the camera to record the image. ### Claim 3 (depends on 2) The method of claim 2, wherein said step of presenting comprises displaying the modification of the image on a display of the mobile device. ### Claim 4 (depends on 1) The method of claim 1, wherein the positional reference comprises a two-dimensional barcode. ### Claim 5 (depends on 1) The method of claim 1, wherein the positional reference comprises a plurality of positional references disposed at various locations within the chilled chamber of the refrigerator appliance. ### Claim 6 (depends on 1) The method of claim 1, wherein the modification of the image illustrates a virtual food item at each positional reference of the plurality of positional references at said step of generating. ### Claim 7 (depends on 1) The method of claim 1, wherein said steps of receiving, generating, and presenting are performed as an app on a mobile device. ### Claim 8 A system for viewing virtual food items within a refrigerator appliance, the refrigerator appliance having a housing that defines a chilled chamber for receipt of food items for storage and a positional reference disposed within the chilled chamber of the refrigerator appliance, the system comprising: a mobile device, comprising: a camera for producing images; a display for viewing images produced by said camera; one or more processors; and at least one memory coupled to the one or more processors, the at least one memory comprising computer-readable instructions for execution by the one or more processors to cause said one or more processors to perform operations, comprising: recording an image of the chilled chamber of the refrigerator appliance and the positional reference disposed within the chilled chamber with said camera; determining a position and orientation of the positional reference within the image; modifying the image by placing a virtual food item adjacent the positional reference within the image; and displaying the modified image with the virtual food item on said display. ### Claim 9 (depends on 8) The system of claim 8, wherein said mobile device comprises a smartphone or a tablet computer. ### Claim 10 (depends on 8) The system of claim 8, wherein the positional reference comprises a two-dimensional barcode. ### Claim 11 (depends on 8) The system of claim 8, wherein the positional reference comprises a plurality of positional references disposed at various locations within the chilled chamber of the refrigerator appliance. ### Claim 12 (depends on 11) The system of claim 11, wherein the modified image illustrates a virtual food item at each positional reference of the plurality of positional references at said step of modifying. ### Claim 13 (depends on 11) The system of claim 11, wherein the refrigerator appliance includes shelves, racks, and bins positioned within the chilled chamber of the refrigerator appliance for storing food items, each positional reference of the plurality of positional references being positioned at a respective one of the shelves, racks, and bins. ### Claim 14 (depends on 8) The system of claim 8, wherein the virtual food item comprises one of a turkey, a milk container, and a case of soda. ### Claim 15 A computer-readable medium having computer-readable instruction for execution by a processor, causing the processor to perform the operations of: receiving an image an appliance and a positional reference disposed on a surface of the appliance from a camera; determining a position and an orientation of the positional reference within the image; generating a modification of the image, the modification illustrating the appliance and a virtual object disposed adjacent the positional reference; and presenting the modification of the image on a display device. ### Claim 16 (depends on 15) The computer-readable medium of claim 15, wherein the camera is coupled to a mobile device. ### Claim 17 (depends on 16) The computer-readable medium of claim 16, wherein the display device is coupled to the mobile device. ### Claim 18 (depends on 15) The computer-readable medium of claim 15, wherein the positional reference comprises a two-dimensional barcode. ### Claim 19 (depends on 15) The computer-readable medium of claim 15, wherein the positional reference comprises a plurality of positional references disposed at various locations within the chamber of the appliance. ### Claim 20 (depends on 19) The computer-readable medium of claim 19, wherein said step of generating comprises modifying the image such that the modification illustrates a virtual object at each positional reference of the plurality of positional references.