# US 9,883,110 B2 · Toggling between augmented reality view and rendered view modes to provide an enriched user experience

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,883,110 B2
- Document ID: US9883110B2
- Status: Granted
- Kind: B2
- Application number: 15/148,002
- Filed: 2016-05-06
- Published: 2016-11-10
- Granted: 2018-01-30
- Earliest claimed: 2015-05-09
- Assignees: CNZ, Inc.
- Applicants: Margaret A. Martin; Alex S. Hill; Harrison D. Leach
- Inventors: Margaret A. Martin; Alex S. Hill; Harrison D. Leach
- Web page: https://patentgrep.com/patent/US9883110B2

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06Q 30/0601: Electronic shopping [e-shopping]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

The present invention relates to a camera enabled mobile device and method of toggling between an augmented reality image viewing mode and a rendered image viewing mode based, in part, on viewability of an augmented reality target (visible or invisible—markerless) proximate the camera enabled mobile device. The method comprising steps of positioning at least one of an augmented reality target in a physical location, aiming the camera at the augmented reality target, initiating, once, an immersive user software experience, determining which of the augmented reality appliance images and the rendered appliance images are associated with the augmented reality target, then toggling between the augmented reality image viewing mode when the augmented reality target is viewable by the camera else entering the rendered image viewing mode.

## Claims

### Claim 1

A method of toggling between an augmented reality image viewing mode and a rendered image viewing mode based, in part, on viewability of an augmented reality target proximate a camera enabled mobile device, the method comprising the steps of:
positioning at least one of an augmented reality target in a physical location, each of the augmented reality targets is uniquely associated with at least one of an augmented reality appliance image representation of an appliance and at least one of a rendered appliance image representation of the appliance, the camera enabled mobile device further comprising a microprocessor, a camera, and a display, the camera and the display are operationally related to the microprocessor;
aiming the camera at the augmented reality target;
initiating, once, an immersive user software experience, by way of the camera detection of the augmented reality target within the physical location, wherein the immersive user software experience becomes operable by a user;
determining which of the augmented reality appliance images and the rendered appliance images are associated with the augmented reality target;
toggling between the augmented reality image viewing mode and the rendered image viewing mode, as follows:
presenting, in the augmented reality image viewing mode, the augmented reality appliance image overlaid with the camera captured image of the physical location, viewable by the user, on the display, when the augmented reality target is viewable by the camera; or
presenting, in the rendered image viewing mode, the rendered appliance image viewable by the user, on the display, when the augmented reality target is not viewable by the camera;
engendering the user to interact with the immersive user software experience and selectively adjust at least one of an experience parameter; and
returning to the step of toggling.

### Claim 2 (depends on 1)

The method in accordance with claim 1, wherein in the step of presenting, in the augmented reality image viewing mode, the augmented reality appliance image is RATIO SCALED graphically and displayed dimensionally accurate with respect to the appliance's actual dimensions with respect to objects in the physical location captured by the camera view, providing visually accurate correlation of dimension between the augmented reality appliance image displayed within the physical location.

### Claim 3 (depends on 1)

The method in accordance with claim 1, further comprising:
receiving by the user or customer the augmented reality target by way of email; and
printing the augmented reality target.

### Claim 4 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
allowing the user to slidably position the augmented reality appliance image on the display, in a manner, as to allow the user to graphically reposition the augmented reality appliance image within the physical location.

### Claim 5 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
allowing the user to change the experience parameter associated with the augmented reality appliance image or the rendered appliance image graphical representation of the appliance color, wherein the user can envision how color of the appliance would appear in the physical location.

### Claim 6 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
allowing a floating label to be positioned and affixed on the augmented reality appliance image; and
affixing, automatically, without intervention by the user, the floating label in same location on the rendered appliance image, wherein the floating label is viewable in both the augmented reality image viewing mode and the rendered image viewing mode.

### Claim 7 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
initiating a remote interaction with a remote attendant;
communicating audible, text, data, or visual data with the remote attendant;
allowing the remote attendant to view images on the display remotely at the remote attendant's location;
receiving from the remote attendant, by way of data communication, for viewability on the display, different at least one the augmented reality appliance images or at least one of the rendered appliance images; and
allowing the remote attendant to remotely control the immersive user software experience for the user.

### Claim 8 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
prompting the user to select certain of the experience parameters to customize graphically displayed images of the appliance; and
placing an order, initiated by the user, for the appliance with customized experience parameters.

### Claim 9 (depends on 7)

The method in accordance with claim 7, the step of engendering further comprising:
allowing the user to place a voice order with the remote attendant or with a global data processing resource.

### Claim 10 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising at least one of the following steps:
allowing the user to take a picture of the current image on the display;
allowing the user to print the picture;
allowing the user to data communicate the picture to a recipient, a website, or a server.

### Claim 11 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
toggling, manually, between the augmented reality image viewing mode and the rendered image viewing mode, by way of, the user pressing a button or a touch screen, the camera enabled mobile device further comprising the button and the touch screen each operationally related to the microprocessor.

### Claim 12 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
toggling, manually, between the languages used by the immersive user software experience, by way of, the user pressing a button or a touch screen, the camera enabled mobile device further comprising the button and the touch screen each operationally related to the microprocessor.

### Claim 13 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
tracking, a plurality of user metrics, associated with the user's engagement with the immersive user software experience;
communicating, the plurality of user metrics, to a remote data processing resource.

### Claim 14 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
toggling, manually ‘On’ and ‘Off’, a ruler overlay to effectuate the user's ability to measure distance, on the display, of the physical location in context with the augmented reality image, by way of, the user pressing a button or a touch screen, the camera enabled mobile device further comprising the button and the touch screen each operationally related to the microprocessor.

### Claim 15 (depends on 1)

The method in accordance with claim 1, wherein in the step of presenting, in the augmented reality image viewing mode, a ruler overlay is scaled graphically and displayed dimensionally accurate with respect to the augmented reality appliance image and objects, in the physical location, as captured by the camera view, providing visually accurate correlation of the ruler overly dimensions and image displayed, on the display, within the physical location.

### Claim 16 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising:
providing access, by way of the immersive user software experience, to a global data processing resource based source of a plurality of electronic documents, wherein the plurality of electronic documents consist of sales collateral, the appliance manuals, the augmented reality appliance images, or the rendered appliance images.

### Claim 17 (depends on 1)

The method in accordance with claim 1, the step of engendering further comprising at least one of the following steps:
allowing the user to exploit certain of the experience parameters as to dissemble through imagery the augmented reality appliance images or the rendered appliance images;
allowing the user to exploit certain of the experience parameters as to open doors or panels through imagery of the augmented reality appliance images;
allowing the user to exploit certain of the experience parameters as to place consumer products through imagery into the augmented reality appliance image, wherein the user can visualize on the display what the appliance would look like, in the physical location, stocked with the consumer products; or
allowing the user to exploit certain of the experience parameters as to entice a customer to effectuate a transaction which allows the appliance to be placed in the physical location by visualizing, on the display, through the use of the augmented reality appliance images what the appliance would look like in the physical location.

### Claim 18 (depends on 1)

The method in accordance with claim 1, the step of initiating further comprising:
selecting a language for use with the immersive user software experience by way of determining a geo-location of the camera enabled mobile device and associating the geo-location with the language.

### Claim 19 (depends on 1)

The method in accordance with claim 1, the step of initiating further comprising:
enabling, from a portfolio of the augmented reality appliance images and the rendered appliance images, a subset of the augmented reality appliance images and the rendered appliance image based, in part, on a geo-location of the camera enabled mobile device.

### Claim 20 (depends on 1)

The method in accordance with claim 1, wherein the immersive user software experience resides on the camera enabled mobile device.

### Claim 21 (depends on 1)

The method in accordance with claim 1, wherein the immersive user software experience resides on a remote data processing resource that is accessible and operable by the camera enabled mobile device.

### Claim 22

A camera enabled mobile device toggles between an augmented reality image viewing mode and a rendered image viewing mode based, in part, on viewability of an augmented reality target proximate the camera enabled mobile device, the camera enabled mobile device comprising:
a microprocessor;
a display operationally related with the microprocessor;
a camera operationally related with the microprocessor; and
a microprocessor readable memory, the microprocessor readable memory is encoded with instructions that when executed perform the steps of:
initiating, once, an immersive user software experience, by way of the camera detection of the augmented reality target within a physical location, wherein the immersive user software experience becomes operable by a user, each of the augmented reality targets is uniquely associated with at least one of an augmented reality appliance image representation of an appliance and at least one of a rendered appliance image representation of the appliance;
determining which of the augmented reality appliance images and the rendered appliance images are associated with the augmented reality target;
toggling between the augmented reality image viewing mode and the rendered image viewing mode, as follows:
presenting, in the augmented reality image viewing mode, the augmented reality appliance image overlaid with the camera captured image of the physical location, viewable by the user, on the display, when the augmented reality target is viewable by the camera; or
presenting, in the rendered image viewing mode, the rendered appliance image viewable by the user, on the display, when the augmented reality target is not viewable by the camera;
engendering the user to interact with the immersive user software experience and selectively adjust at least one of an experience parameter; and
returning to the step of toggling.

### Claim 23

A method of toggling between an augmented reality image viewing mode and a rendered image viewing mode based, in part, on viewability of an augmented reality target proximate a camera enabled mobile device, the method comprising the steps of:
positioning at least one of an augmented reality target in a physical location, each of the augmented reality targets is uniquely associated with at least one of an augmented reality appliance image representation of an appliance and at least one of a rendered appliance image representation of the appliance, the camera equipped mobile device further comprising a microprocessor, a camera, and a display, the camera and the display are operationally related to the microprocessor;
aiming the camera at the augmented reality target;
initiating, once, an immersive user software experience, by way of the camera detection of the augmented reality target within the physical location, wherein the immersive user software experience becomes operable by a user;
determining which of the augmented reality appliance images and the rendered appliance images are associated with the augmented reality target;
toggling between the augmented reality image viewing mode and the rendered image viewing mode, as follows:
presenting, in the augmented reality image viewing mode, the augmented reality appliance image overlaid with the camera captured image of the physical location, viewable by the user, on the display, when the augmented reality target is viewable by the camera; or
presenting, in the rendered image viewing mode, the rendered appliance image viewable by the user, on the display, when the augmented reality target is not viewable by the camera;
engendering the user to interact with the immersive user software experience and selectively adjust at least one of an experience parameter, the step of engendering further comprising:
allowing a floating label to be positioned and affixed on the rendered appliance image; and
affixing, automatically, without intervention by the user, the floating label in same location on the augmented reality appliance image, wherein the floating label is viewable in both the augmented reality image viewing mode and the rendered image viewing mode;
allowing the user to exploit certain of the experience parameters as to entice a customer to effectuate a transaction which allows the appliance to be placed in the physical location by visualizing, on the display, through the use of the augmented reality appliance images what the appliance would look like in the physical location; and
returning to the step of toggling.