# US 2017/0255257 A1 · Systems and methods for delivering imagery to head-worn display systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0255257 A1
- Document ID: US20170255257A1
- Status: Published application
- Kind: A1
- Application number: 15/061,922
- Filed: 2016-03-04
- Published: 2017-09-07
- Earliest claimed: 2016-03-04
- Assignees: Rockwell Collins, Inc.
- Applicants: Rockwell Collins, Inc.
- Inventors: Carlo L. Tiana; Weston J. Lahr; Robert B. Wood
- Web page: https://patentgrep.com/patent/US20170255257A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0141: {characterised by the informative content of the display}
- G02B 2027/0147: {comprising a device modifying the resolution of the displayed image}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G09G 5/377: for mixing or overlaying two or more graphic patterns (G09G5/02, G09G5/397 take precedence)
- G09G 5/391: Resolution modifying circuits, e.g. variable screen formats
- G09G 2320/0261: in the context of movement of objects on the screen or movement of the observer relative to the screen
- G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas
- G09G 2340/12: Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
- G09G 2380/12: Avionics applications
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A method for providing a display on a head-worn display includes receiving imagery from one or more sensors configured to capture details of an environment surrounding an aircraft, receiving a current gaze of a user of the head-worn display, and processing the imagery. Processing the imagery comprises identifying one or more attributes of the imagery. The method further includes identifying at least two layers of the imagery. At least one of the at least two layers has a different resolution than another layer. The method further includes determining a priority of each of the at least two layers based on the one or more attributes and the current gaze of the user. The method further includes transmitting one or more of the at least two layers to the head-worn display based on the determined priority of the layers.

## Claims

### Claim 1

A method, comprising:
receiving, by a processing circuit of a head-worn display system, imagery from one or more sensors configured to capture details of an environment surrounding an aircraft;
receiving, by the processing circuit, a current gaze of a user of the head-worn display;
processing, by the processing circuit, the imagery, wherein the processing comprises identifying one or more attributes of the imagery;
identifying, by the processing circuit, at least two layers of the imagery, wherein at least one of the at least two layers has a different resolution than another layer;
determining, by the processing circuit, a priority of each of the at least two layers based on the one or more attributes and the current gaze of the user; and
transmitting, by an output circuit of the head-worn display system, one or more of the at least two layers to the head-worn display based on the determined priority of the layers.

### Claim 2 (depends on 1)

The method of claim 1, wherein transmitting one or more of the at least two layers to the head-worn display comprises:
transmitting a first layer comprising a low resolution image of a region to be displayed in a full view of the head-worn display; and
transmitting one or more second layers comprising one or more images having a higher resolution than the first layer, the one or more second layers further comprising a portion of a region nearest to a focal point of the current gaze of the user;
wherein the one or more second layers are transmitted to the head-worn display at a higher rate than the first layer.

### Claim 3 (depends on 1)

The method of claim 1, wherein the current gaze of the user defines a field of view and a peripheral view of the user, and wherein transmitting one or more of the at least two layers to the head-worn display comprises:
transmitting a first layer comprising a low resolution image of a region to be displayed in a full view of the head-worn display; and
transmitting one or more second layers comprising one or more images having a higher resolution than the first layer, the one or more second layers further comprising a portion of the region corresponding to the field of view of the user.

### Claim 4 (depends on 3)

The method of claim 3, wherein the current gaze of the user is based on eye tracking information and an orientation of the head-worn display; and
wherein the eye tracking information is used to determine the field of view and the peripheral view of the user.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
receiving, by the processing circuit, updated eye tracking information:
identifying, by the processing circuit, an updated field of view of the user; and
transmitting, by the output circuit, one or more of the at least two layers corresponding to the updated field of view of the user to the head-worn display.

### Claim 6 (depends on 4)

The method of claim 4, wherein determining the field of view of the user comprises:
determining a current head orientation of the user:
determining a focal area of the display based on the current head orientation; and
determining a range proximate the focal area;
wherein the one or more second layers comprise imagery having a higher resolution than the first layer, and wherein the one or more second layers comprise imagery within the range proximate the focal area.

### Claim 7 (depends on 1)

The method of claim 1, wherein the processing circuit determines a priority for each of the at least two layers based on the resolution of each layer; and
wherein transmitting the at least two layers comprises communicating the at least two layers to the head-worn display based on the determined priority of the layers.

### Claim 8 (depends on 1)

The method of claim 1, wherein the one or more attributes identified in each layer comprises one or more of a tower, a runway, a ground, a sky, and an obstacle; and
wherein prioritizing the one or more layers includes prioritizing the one or more attributes such that imagery having a higher resolution than the first layer of the one or more attributes is transmitted to the head-worn display.

### Claim 9 (depends on 1)

The method of claim 1, wherein determining the priority of the one or more layers is further based on a setting inputted by the user of the head-worn display.

### Claim 10 (depends on 1)

The method of claim 1, wherein determining the priority of the layers is further based on a phase of flight of the aircraft.

### Claim 11 (depends on 1)

The method of claim 1, wherein the processing circuit is further configured to assign a refresh rate to each layer, the method further comprising:
receiving, by the processing circuit, updated imagery from the one or more sensors after receiving and processing the imagery;
processing, by the processing circuit, the updated imagery;
identifying, by the processing circuit, one or more layers of the imagery; and
transmitting, by the output circuit, based on the refresh rate of layers previously transmitted to the head-worn display, one or more layers to the head-worn display to replace one or more previous layers.

### Claim 12

An image delivery system for a head-worn display, comprising:
an input module coupled to a processor of an image delivery system and configured to cause the processor to receive a current gaze of a user of the head-worn display and imagery from one or more sensors configured to capture details of an environment surrounding an aircraft;
an image processing module coupled to the processor and configured to cause the processor to process the imagery, wherein processing the imagery comprises identifying one or more attributes of the imagery and identifying at least two layers of the imagery, wherein at least one of the at least two layers has a different resolution than another layer;
a prioritization module coupled to the processor and configured to cause the processor to determine a priority of each of the at least two layers based on the one or more attributes and the current gaze user; and
an output module coupled to the processor and configured to cause the processor to transmit one or more of the at least two layers to the head-worn display based on the determined priority of the layers;
wherein the input module, the processor, the image processing module, the prioritization module, and the output module are each remote from the head-worn display.

### Claim 13 (depends on 12)

The system of claim 12, wherein transmitting one or more of the at least two layers to the head-worn display comprises:
transmitting a first layer comprising a low resolution image of a region to be displayed in a full view of the head-worn display; and
transmitting one or more second layers comprising one or more images having a higher resolution than the first layer, the one or more second layers further comprising a portion of the region nearest to a focal point of the current gaze of the user;
wherein the one or more second layers are transmitted to the head-worn display at a higher rate than the first layer.

### Claim 14 (depends on 12)

The system of claim 12, wherein the current status of the user comprises a field of view and a peripheral view of the user, and wherein transmitting one or more layers to the head-worn display comprises:
transmitting a first layer comprising a low resolution image of a region to be displayed in a full view of the head-worn display; and
transmitting one or more second layers comprising one or more images having a higher resolution than the first layer, the one or more second layers further comprising a portion of the region corresponding to the field of view of the user.

### Claim 15 (depends on 14)

The method of claim 14, wherein the current gaze of the user is based on eye tracking information and an orientation of the head-worn display; and
wherein the eye tracking information is used to determine the field of view and the peripheral view of the user.

### Claim 16 (depends on 15)

The system of claim 15, wherein the input module is further configured to cause the processor to receive updated eye tracking information, and wherein the image processing module is further configured to cause the processor to identify an updated field of view of the user; and
wherein the one or more of the at least two layers are transmitted to the head-worn display based on the one or more layers corresponding to the updated field of view of the user.

### Claim 17 (depends on 15)

The system of claim 15, wherein determining the field of view of the user comprises:
determining a current head orientation of the user;
determining a focal area of the display based on the current head orientation; and
determining a range proximate the focal area;
wherein the one or more second layers comprise imagery having a higher resolution than the first layer, and wherein the one or more second layers comprise imagery within the range proximate the focal area.

### Claim 18 (depends on 12)

The system of claim 12, wherein the prioritization module is further configured to cause the processor to determine a priority of each of the at least two layers based on the resolution of each layer, and
wherein the output module is further configured to cause the processor to transmit the at least two layers to the head-worn display in an order of increasing resolution.

### Claim 19 (depends on 12)

The system of claim 12, wherein the one or more attributes identified in each layer comprises one or more of a tower, a runway, a ground, a sky, and an obstacle; and
wherein prioritizing the one or more layers includes prioritizing the one or more attributes such that imagery having a higher resolution than the first layer of one or more attributes is transmitted to the head-worn display.

### Claim 20

A system, comprising:
a vision system configured to generate an image of an environment surrounding an aircraft, wherein the vision system receives inputs from a plurality of sensors of the aircraft;
an image delivery system configured to:
process the image generated by the vision system to identify one or more attributes of the image;
identify at least two layers of the image, wherein at least one of the at least two layers has a different resolution than another layer; and
determine a priority of each of the at least two layers based on the one or more attributes and the current gaze of the user; and
transmit one or more of the at least two layers to a head-worn display based on the determined priority of the layers.
