# US 2019/0317659 A1 · Dynamic Image Stabilization Using Motion Sensors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0317659 A1
- Document ID: US20190317659A1
- Status: Published application
- Kind: A1
- Application number: 16/366,503
- Filed: 2019-03-27
- Published: 2019-10-17
- Earliest claimed: 2018-04-17
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Paul V. Johnson; Ahmad Rahmati; Chaohao Wang; Cheng Chen; Graham B. Myhre; Jiaying Wu; Paolo Sacchetto; Sheng Zhang; Yunhui Hou; Xiaokai Li; Tim H. Cornelissen
- Web page: https://patentgrep.com/patent/US20190317659A1

## CPC

- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 1/1626: {with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]}
- G06F 1/163: {Wearable computers, e.g. on a belt}
- G06F 1/1643: {the display being associated to a digitizer, e.g. laptops that can be used as penpads (details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615)}
- G06F 1/1686: {the I/O peripheral being an integrated camera}
- G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer}
- G06F 1/1698: {the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone (details of antennas disposed inside a computer H01Q1/2266)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An electronic device may include a display for displaying image content to a user and dynamic image stabilization circuitry for dynamically compensating the image content if the device is moving unpredictably to help keep the image content aligned with the user's gaze. The electronic device may include sensors for detecting the displacement of the device. The dynamic image stabilization circuitry may include a usage scenario detection circuit and a content displacement compensation calculation circuit. The usage scenario detection circuit receives data from the sensors and infers a usage scenario based on the sensor data. The content displacement compensation calculation circuit uses the inferred usage scenario to compute a displacement amount by which to adjust image content. When motion stops, the image content may gradually drift back to the center of the display.

## Claims

### Claim 1

An electronic device, comprising:
a display configured to output an image content;
a motion sensor configured to detect movement of the electronic device; and
dynamic image stabilization circuitry that is used to adjust the image content on the display based on the detected movement of the electronic device.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the dynamic image stabilization circuitry comprises:
a usage scenario detection circuit configured to determine in which one of a plurality of predetermined usage scenarios the electronic device is currently operating.

### Claim 3 (depends on 2)

The electronic device of claim 2, wherein the dynamic image stabilization circuitry further comprises:
a calculation circuit configured to compute an amount by which to adjust the image content based on which of the plurality of usage scenarios is determined by the usage scenario detection circuit.

### Claim 4 (depends on 3)

The electronic device of claim 3, wherein the motion sensor is configured to output sensor data, and wherein the usage scenario detection circuit is configured to determine in which one of the plurality of predetermined usage scenario the electronic device is currently operating by applying a classification process to the sensor data.

### Claim 5 (depends on 4)

The electronic device of claim 4, wherein the usage scenario detection circuit is further configured to extract features from the sensor data, to compute probabilities for each of the plurality of predetermined usage scenarios based on the extracted features, and to output a selected one of the predetermined usage scenarios with the highest computed probability.

### Claim 6 (depends on 3)

The electronic device of claim 3, wherein the image content is adjusted using a spring-damper system, wherein the usage scenario detection circuit is further configured to generate a damping factor for the spring-damper system, and wherein the calculation circuit receives the damping factor.

### Claim 7 (depends on 3)

The electronic device of claim 3, wherein the calculation circuit is configured to output a first amount of compensation for a first usage scenario in the plurality of predetermined usage scenarios and to output a second amount of compensation that is different than the first amount of compensation for a second usage scenario in the plurality of predetermined usage scenarios.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein the display is further configured to provide a margin around the image content to prevent clipping while the image content is adjusted.

### Claim 9 (depends on 1)

The electronic device of claim 1, wherein the image content is adjusted by dynamically shifting the image content along the plane of the display.

### Claim 10 (depends on 1)

The electronic device of claim 1, wherein the image content is adjusted by dynamically magnifying or minifying the image content on the display.

### Claim 11 (depends on 1)

The electronic device of claim 1, further comprising:
a head tracking system configured to detect motion of a user's head relative to the electronic device, wherein the dynamic image stabilization circuitry also uses the detected motion of the user's head to adjust the image content on the display.

### Claim 12 (depends on 1)

The electronic device of claim 1, wherein the motion sensor comprises a selected one of an accelerometer and a gyroscope.

### Claim 13

A method of operating an electronic device having a display, the method comprising:
outputting an image content on the display;
with a motion sensor within the electronic device, detecting movement of the electronic device;
in response to detecting the movement of the electronic device, adjusting the image content in a way that opposes the detected movement.

### Claim 14 (depends on 13)

The method of claim 13, wherein the motion sensor comprises a selected one of an accelerometer and a gyroscope.

### Claim 15 (depends on 13)

The method of claim 13, further comprising:
with a head tracking system within the electronic device, detecting movement of a user's head with respect to the display of the electronic device; and
adjusting the image content only when the movement of the user's head is out of sync with the movement of the electronic device.

### Claim 16 (depends on 13)

The method of claim 13, further comprising:
with a usage scenario detection circuit within the electronic device, determining a current usage scenario from a plurality of predetermined usage scenarios for the electronic device based on the detected movement; and
with a compensation calculation circuit within the electronic device, computing an amount by which to adjust the image content based on the current usage scenario determined by the usage scenario detection circuit.

### Claim 17 (depends on 13)

The method of claim 13, wherein adjusting the image content comprises shifting the image content along the plane of the display, the method further comprising:
with the motion sensor, detecting when the electronic device has stopped moving; and
when the motion sensor detects that the electronic device has stopped moving, gradually shifting the image content back to the center of the display.

### Claim 18

A system, comprising:
a first electronic device having a display configured to output an image content and a first motion sensor configured to output first motion sensor data; and
a second electronic device having a second motion senor configured to output second motion sensor data, wherein the first electronic device is configured to adjust the image content on the display based on the first and second motion sensor data.

### Claim 19 (depends on 18)

The system of claim 18, wherein the first electronic device further includes a head tracking system for detecting motion of a user's head relative to the first electronic device, and wherein the first electronic device is further configured to adjust the image content only when the head tracking system determines that the user's head is uncoordinated with the first electronic device.

### Claim 20 (depends on 18)

The system of claim 18, wherein the second electronic device is configured to transmit the second motion sensor data to the first electronic device via a wireless link.

### Claim 21 (depends on 20)

The system of claim 20, wherein the second electronic device comprises at least one earbud.
