# US 2023/0393674 A1 · External user interface for head worn computing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0393674 A1
- Document ID: US20230393674A1
- Status: Published application
- Kind: A1
- Application number: 18/454,648
- Filed: 2023-08-23
- Published: 2023-12-07
- Earliest claimed: 2014-01-17
- Assignees: Mentor Acquisition One, LLC
- Applicants: Mentor Acquisition One, LLC
- Inventors: Ralph F. OSTERHOUT
- Web page: https://patentgrep.com/patent/US20230393674A1

## CPC

- G06F 3/03545: {Pens or stylus}
- B43K 29/00: Combinations of writing implements with other articles
- B43K 29/004: {with more than one object}
- B43K 29/08: with measuring, computing or indicating devices
- G02B 27/017: {Head mounted}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0317: {in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface}
- G06F 3/042: by opto-electronic means
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- B43K: IMPLEMENTS FOR WRITING OR DRAWING (containers, casings or accessories for cosmetic substances, e.g. shaving soap, lipstick, make-up A45D34/00, A45D40/00)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

Aspects of the present invention relate to external user interfaces used in connection with head worn computers (HWC).

## Claims

### Claim 1

A method, comprising:
determining a virtual line in a mixed reality environment, the mixed reality environment visible to a user of a wearable device via a see-through display of the wearable device;
tracking a motion of a finger of the user as the finger moves relative to the virtual line, wherein said tracking comprises:
receiving, via a sensor, a signal indicative of a movement of the finger in the mixed reality environment relative to the virtual line; and
obtaining images of the movement of the finger;
determining, based on the signal, a first relative motion between the finger and the virtual line in the mixed reality environment;
determining, based on the images, a second relative motion between the finger and the virtual line in the mixed reality environment; and
determining an input sequence corresponding to one or more input strokes of the user, wherein the input sequence is determined based on a comparison of the first relative motion and the second relative motion.

### Claim 2 (depends on 1)

The method of claim 1, further comprising presenting the virtual line to the user via the see-through display.

### Claim 3 (depends on 1)

The method of claim 1, further comprising presenting the virtual line to the user via the see-through display as a portion of a message.

### Claim 4 (depends on 1)

The method of claim 1, further comprising presenting the virtual line to the user via the see-through display as a portion of an email.

### Claim 5 (depends on 1)

The method of claim 1, further comprising presenting the virtual line to the user via the see-through display as a portion of a writing platform.

### Claim 6 (depends on 1)

The method of claim 1, further comprising presenting the virtual line to the user via the see-through display as a portion of a painting platform.

### Claim 7 (depends on 1)

The method of claim 1, further comprising presenting the virtual line to the user via the see-through display as a portion of a drawing platform.

### Claim 8 (depends on 1)

The method of claim 1, further comprising presenting the input sequence via the see-through display relative to the virtual line.

### Claim 9 (depends on 8)

The method of claim 8, further comprising presenting the input sequence in real time.

### Claim 10 (depends on 1)

The method of claim 1, wherein said determining the input sequence comprises determining a position of the tip of the finger relative to the virtual line.

### Claim 11 (depends on 1)

The method of claim 1, wherein said motion comprises motion on a surface in the mixed reality environment and wherein said images comprise images of the surface.

### Claim 12 (depends on 1)

The method of claim 1, further comprising applying the input sequence to the mixed reality environment.

### Claim 13

A non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform a method comprising:
determining a virtual line in a mixed reality environment, the mixed reality environment visible to a user of a wearable device via a see-through display of the wearable device;
tracking a motion of a finger of the user as the finger moves relative to the virtual line, wherein said tracking comprises:
receiving, via a sensor, a signal indicative of a movement of the finger in the mixed reality environment relative to the virtual line; and
obtaining images of the movement of the finger;
determining, based on the signal, a first relative motion between the finger and the virtual line in the mixed reality environment;
determining, based on the images, a second relative motion between the finger and the virtual line in the mixed reality environment; and
determining an input sequence corresponding to one or more input strokes of the user, wherein the input sequence is determined based on a comparison of the first relative motion and the second relative motion.

### Claim 14 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the method further comprises presenting the virtual line to the user via the see-through display.

### Claim 15 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the method further comprises applying the input sequence to the mixed reality environment.

### Claim 16 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the method further comprises presenting the input sequence via the see-through display relative to the virtual line.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein the method further comprises presenting the input sequence in real time.

### Claim 18 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein said determining the input sequence comprises determining a position of the tip of the finger relative to the virtual line.

### Claim 19 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein said motion comprises motion on a surface in the mixed reality environment and wherein said images comprise images of the surface.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the method further comprises presenting the virtual line on the surface in the mixed reality environment via the see-through display.
