# US 10,558,325 B2 · Systems and methods for controlling operation of an on-board component

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,558,325 B2
- Document ID: US10558325B2
- Status: Granted
- Kind: B2
- Application number: 14/556,593
- Filed: 2014-12-01
- Published: 2016-12-01
- Granted: 2020-02-11
- Earliest claimed: 2013-12-01
- Assignees: Upskill, Inc.
- Applicants: James Leighton Athey; Todd Richard Reily; Michael Anthony Sciscenti
- Inventors: James Leighton Athey; Todd Richard Reily; Michael Anthony Sciscenti
- Web page: https://patentgrep.com/patent/US10558325B2

## CPC

- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- A45C 11/00: Receptacles for purposes not provided for in groups A45C1/00-A45C9/00 (specially adapted for toiletry or cosmetic equipment A45D29/20, A45D44/18; travelling sewing kits A45F3/48)
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G06F 1/163: {Wearable computers, e.g. on a belt}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04812: Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0485: Scrolling or panning
- G06F 3/0487: using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 21/71: to assure secure computing or processing of information
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04L 12/1822: {Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission}
- H04L 67/52: specially adapted for the location of the user terminal
- H04M 1/72454: according to context-related or environment-related conditions
- H04N 23/51: Housings
- H04N 23/61: based on recognised objects
- H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters}
- H04N 23/635: {Region indicators; Field of view indicators}
- H04W 4/08: User group management
- H04W 12/084: using delegated authorisation, e.g. open authorisation [OAuth] protocol
- H04W 76/10: Connection setup
- A45C 11/002: {for storing portable handheld communication devices, e.g. pagers or smart phones}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0141: {characterised by the informative content of the display}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
- G06F 3/0486: Drag-and-drop
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A45C: PURSES; LUGGAGE; HAND CARRIED BAGS
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Systems and methods allow a user to interact with an augmented reality device. In one implementation, a wearable device for controlling operation of an on-board component includes a display; at least one sensor configured to provide an output indicative of a viewing direction of a user; and at least one processing device. The at least one processing device is configured to cause at least one graphical icon associated with a control of at least one aspect of the on-board component to be shown on the display; determine, based on the output of the at least one sensor, whether the user is looking in a direction of the at least one graphical icon; and initiate the control of the at least one aspect of the on-board component when the user is determined to be looking in the direction of the at least one graphical icon.

## Claims

### Claim 1

A wearable device for controlling operation of an on-board component, the wearable device comprising:
a display;
at least one sensor configured to provide an output indicative of a viewing direction of a user; and
at least one processing device configured to:
cause at least one graphical icon associated with a control of at least one aspect of the on-board component to be shown on the display;
cause a reticle to be displayed on the display;
move, based on an output of the at least one sensor indicating a change in the viewing direction of the user, the reticle relative to the display in a first direction corresponding to the change in the viewing direction of the user;
move the at least one graphical icon in a second direction toward the reticle as the reticle moves relative to the display in the first direction, such that the reticle intercepts a portion of the at least one graphical icon;
change, based on the reticle having intercepted the portion of the at least one graphical icon, at least one aspect of the reticle to provide a visual indication to the user that the at least one graphical icon can be selected by looking in the direction of the at least one graphical icon for a predetermined period of time;
select the at least one graphical icon when it is determined that the user has been looking in the direction of the at least one graphical icon for the predetermined period of time; and
cause the on-board component to capture an image of a field of view of the user when the at least one graphical icon is selected.

### Claim 2 (depends on 1)

The wearable device of claim 1, wherein the on-board component includes a camera.

### Claim 3 (depends on 2)

The wearable device of claim 2, wherein the at least one graphical icon is associated with a power-on function of the on-board camera.

### Claim 4 (depends on 2)

The wearable device of claim 2, wherein the at least one graphical icon is associated with an image capture function of the on-board camera.

### Claim 5 (depends on 2)

The wearable device of claim 2, wherein the at least one processing device is configured to show a menu of camera control icons on the display.

### Claim 6 (depends on 5)

The wearable device of claim 5, wherein the menu of camera control icons comprises icons associated with one or more of shutter speed, aperture, ISO, and flash.

### Claim 7 (depends on 1)

The wearable device of claim 1, wherein the at least one processing device is configured to update a position at which the at least one graphical icon is shown on the display based on the output of the at least one sensor.

### Claim 8 (depends on 1)

The wearable device of claim 1, wherein the at least one sensor is configured to provide an output indicative of the viewing direction of the user by tracking a pattern of movement of an orientation of a head of the user.

### Claim 9 (depends on 1)

The wearable device of claim 1, wherein the at least one sensor is configured to provide an output indicative of the viewing direction of the user by tracking a gaze of the user's eyes.

### Claim 10

A method for controlling operation of an on-board component of a wearable device, the method comprising:
causing at least one graphical icon associated with a control of at least one aspect of the on-board component to be shown on a display of the wearable device;
causing a reticle to be displayed on the display;
moving, based on an output of at least one sensor of the wearable device indicating a change in a viewing direction of the user of the wearable device, the reticle relative to the display in a first direction corresponding to the change in the viewing direction of the user;
moving the at least one graphical icon in a second direction toward the reticle as the reticle moves relative to the display in the first direction, such that the reticle intercepts a portion of the at least one graphical icon;
changing, based on the reticle having intercepted the portion of the at least one graphical icon, at least one aspect of the reticle to provide a visual indication to the user that the at least one graphical icon can be selected by looking in the direction of the at least one graphical icon for a predetermined period of time;
selecting the at least one graphical icon when it is determined that the user has been looking in the direction of the at least one graphical icon for the predetermined period of time; and
causing the on-board component to capture an image of a field of view of the user when at least one graphical icon is selected.

### Claim 11 (depends on 10)

The method of claim 10, wherein the on-board component includes a camera.

### Claim 12 (depends on 11)

The method of claim 11, wherein the at least one graphical icon is associated with a power on function of the on-board camera.

### Claim 13 (depends on 11)

The method of claim 11, wherein the at least one graphical icon is associated with an image capture function of the on-board camera.

### Claim 14 (depends on 11)

The method of claim 11, wherein a menu of camera control icons is shown on the display.

### Claim 15 (depends on 14)

The method of claim 14, wherein the menu of camera control icons comprises icons associated with one or more of shutter speed, aperture, ISO, and flash.

### Claim 16 (depends on 10)

The method of claim 10, further comprising updating a position at which the at least one graphical icon is shown on the display based on the output of the at least one sensor.

### Claim 17 (depends on 10)

The method of claim 10, further comprising monitoring a pattern of movement of an orientation of a head of the user.

### Claim 18 (depends on 10)

The method of claim 10, further comprising monitoring a gaze of the user's eyes.

### Claim 19

A non-transitory computer-readable storage medium storing program instructions executable by at least one processor to perform a method for controlling operation of an on-board component of a wearable device, the method comprising:
causing at least one graphical icon associated with a control of at least one aspect of the on-board component to be shown on a display of the wearable device;
causing a reticle to be displayed on the display;
moving, based on an output of at least one sensor of the wearable device indicating a change in a viewing direction of the user of the wearable device, the reticle relative to the display in a first direction corresponding to the change in the viewing direction of the user;
moving the at least one graphical icon in a second direction toward the reticle as the reticle moves relative to the display in the first direction, such that the reticle intercepts a portion of the at least one graphical icon;
changing, based on the reticle having intercepted the portion of the at least one graphical icon, at least one aspect of the reticle to provide a visual indication to the user that the at least one graphical icon can be selected by looking in the direction of the at least one graphical icon for a predetermined period of time;
selecting the at least one graphical icon when it is determined that the user has been looking in the direction of the at least one graphical icon for the predetermined period of time; and
causing the on-board component to capture an image of a field of view of the user when the at least one graphical icon is selected.