# US 2024/0004483 A1 · Input device with optical sensors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004483 A1
- Document ID: US20240004483A1
- Status: Published application
- Kind: A1
- Application number: 17/809,848
- Filed: 2022-06-29
- Published: 2024-01-04
- Earliest claimed: 2022-06-29
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: William WHITNEY; Yuhao PAN; Ahmet Fatih CIHAN; Tong CHEN; Stephen E. DEY
- Web page: https://patentgrep.com/patent/US20240004483A1

## CPC

- G06F 3/03542: {Light pens for emitting or receiving light}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0312: {for tracking the rotation of a spherical or circular member, e.g. optical rotary encoders used in mice or trackballs using a tracking ball or in mouse scroll wheels (tracking relative movement in co-operation with a regularly or irregularly patterned surface, e.g. as in optical mice G06F3/0317; constructional details of scroll or thumb-wheels G06F3/0362; optical rotary encoders G01D5/3473)}
- G06F 3/0317: {in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/03545: {Pens or stylus}
- G06F 3/03546: {using a rotatable ball at the tip as position detecting member}
- G06F 3/03549: {Trackballs (G06F3/03541 takes precedence)}
- G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G06F 2203/0384: Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Optical sensors can be used for content generation using an input device without a touch sensitive surface. In some examples, the optical sensors can be used to detect characteristics of the input device including position, orientation, and/or motion of the input device. In some examples, some or all of the characteristics of the input device can be used in processing to generate content, including textual character input and three-dimensional objects. In some examples, the generation of content can use information from one or more additional sensors of the input device and/or from additional devices in combination with the characteristics of the input device based on the optical sensors.

## Claims

### Claim 1

A stylus comprising:
an optical sensor configured to track displacement of the stylus relative to one or more surfaces based on detecting light reflected from the one or more surfaces in response to light emitted from the stylus, wherein the optical sensor comprises an optical flow sensor or a laser speckle flow sensor; and
wireless communication circuitry coupled to the optical sensor, wherein the wireless communication circuitry is configured to transmit information from the stylus to an external device configured to use the information from the stylus as input, wherein the information includes displacement data from the optical sensor, other information derived from the displacement data, or both.

### Claim 2 (depends on 1)

The stylus of claim 1, wherein the optical sensor comprises a light emitter configured to emit light and a light detector configured to detect the light reflected from the one or more surfaces.

### Claim 3 (depends on 1)

The stylus of claim 1, wherein the optical sensor is configured to detect displacement of the stylus without a lens.

### Claim 4 (depends on 1)

The stylus of claim 1, comprising an additional sensor coupled to the wireless communication circuitry and configured to measure a tilt, an orientation, an axial rotation, or any combination thereof of the stylus to compensate for gain error in the displacement data.

### Claim 5 (depends on 4)

The stylus of claim 4, wherein the additional sensor comprises a second optical sensor, and inertial measurement unit (IMU) sensor, a magnetic sensor, a force sensor, or any combination thereof.

### Claim 6 (depends on 1)

The stylus of claim 1, comprising at least two optical sensors configured to track displacement of the stylus in three, four, five, or six dimensions.

### Claim 7 (depends on 1)

The stylus of claim 1, wherein the optical sensor is disposed at a tip of the stylus such that the optical sensor is configured to track motion of the tip of the stylus relative to the one or more surfaces.

### Claim 8 (depends on 1)

The stylus of claim 1, comprising a trackball disposed at a tip of the stylus such that the optical sensor is configured to track motion of the trackball relative to the one or more surfaces.

### Claim 9 (depends on 1)

The stylus of claim 1, wherein a tip of the stylus is formed from an optically transparent or translucent material.

### Claim 10 (depends on 1)

The stylus of claim 1, wherein a tip of the stylus is formed from opaque material and the tip includes a plurality of windows for beams from one or more optical sensors to exit and return to the stylus.

### Claim 11 (depends on 1)

The stylus of claim 1, wherein using the information from the stylus as input comprises rendering writing or drawing strokes of the stylus on a non-touch sensitive surface.

### Claim 12

A system comprising:
a first device comprising:
a plurality of first sensors configured to track displacement in position and angle of the first device relative to a non-touch sensitive surface; and
first communication circuitry coupled to the plurality of first sensors and configured to transmit information from the plurality of first sensors to a second device;
the second device comprising:
one or more second sensors configured to track a distance between the first device and the non-touch sensitive surface;
second communication circuitry configured to receive the information from the plurality of first sensors;
processing circuitry configured to receive the displacement in position and angle of the first device from the plurality of first sensors and generate content using the displacement in position and angle of the first device and the distance between the first device and the non-touch sensitive surface; and
a display configured to display the content generated by the processing circuitry on the non-touch sensitive surface.

### Claim 13 (depends on 12)

The system of claim 12, wherein the plurality of first sensors comprises at least two optical sensors.

### Claim 14 (depends on 12)

The system of claim 12, wherein the plurality of first sensors comprises an optical sensor and an additional sensor comprising a force sensor, a magnetic sensor, an inertial magnetic unit (IMU) sensor, or any combination thereof.

### Claim 15 (depends on 12)

The system of claim 12, wherein the plurality of first sensors comprises as least an optical flow sensor or a laser speckle flow sensor.

### Claim 16 (depends on 12)

The system of claim 12, wherein the first device comprises a stylus and the second device comprises a head-mounted display device.

### Claim 17 (depends on 12)

The system of claim 12, wherein the one or more second sensors comprise a camera, a magnetic sensor, audio sensor, or an electromagnetic sensor configured to track the distance between the first device and the non-touch sensitive surface using changes in light field, magnetic field, acoustic field, or electromagnetic field.

### Claim at position 18

18-20. (canceled)

### Claim 21 (depends on 12)

The system of claim 12, wherein the plurality of first sensors comprises a light emitter configured to emit light and a light detector configured to detect the light reflected from the non-touch sensitive surface.

### Claim 22 (depends on 12)

The system of claim 12, wherein the plurality of first sensors comprises a sensor configured to measure a tilt, an orientation, an axial rotation, or any combination thereof of the first device to compensate for gain error in the displacement in position and angle of the first device.

### Claim 23 (depends on 12)

The system of claim 12, wherein the plurality of first sensors comprises at least two optical sensors configured to track displacement of the first device in six dimensions.
