# US 2017/0371407 A1 · Method and Apparatus for Communication Between Humans and Devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0371407 A1
- Document ID: US20170371407A1
- Status: Published application
- Kind: A1
- Application number: 15/429,733
- Filed: 2017-02-10
- Published: 2017-12-28
- Earliest claimed: 2003-03-21
- Assignees: Queen's University at Kingston
- Applicants: Queen's University at Kingston
- Inventors: Roel Vertegaal; Jeffrey S. Shell
- Web page: https://patentgrep.com/patent/US20170371407A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G05B 23/02: Electric testing or monitoring
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

This invention relates to methods and apparatus for improving communications between humans and devices. The invention provides a method of modulating operation of a device, comprising: providing an attentive user interface for obtaining information about an attentive state of a user; and modulating operation of a device on the basis of the obtained information, wherein the operation that is modulated is initiated by the device. Preferably, the information about the user's attentive state is eye contact of the user with the device that is sensed by the attentive user interface.

## Claims

### Claim at position 1

1-44. (canceled)

### Claim 45

Apparatus for modulating presentation of information on a screen of a device, comprising:
at least one sensor embedded in the device that outputs a sensor signal;
a processor that processes the sensor signal and produces a user state signal that is indicative of user visual attention toward the screen of the device;
wherein the apparatus uses the user state signal as a basis for determining whether to present the information on the screen of the device;
wherein the presentation of information is initiated by the device based on said determining.

### Claim 46 (depends on 45)

The apparatus of claim 45, wherein modulating presentation of information comprises the apparatus using the user state signal as a basis for determining whether to pause outputting of information on the screen of the device, and whether to resume the outputting of information on the screen of the device;
wherein the pausing and resuming of the outputting of information is initiated by the device based on said determining.

### Claim 47 (depends on 45)

The apparatus of claim 45, wherein the information includes audio information.

### Claim 48 (depends on 45)

The apparatus of claim 45, wherein the device is selected from the group consisting of a personal computer, a cellular telephone, a telephone, a personal digital assistant (PDA), an electronic device, a machine, a smart appliance, and an appliance.

### Claim 49 (depends on 45)

The apparatus of claim 45, wherein the device is a cellular telephone.

### Claim 50 (depends on 45)

The apparatus of claim 45, wherein the outputting of information on the screen of the device includes the device soliciting user input.

### Claim 51 (depends on 45)

The apparatus of claim 45, wherein the user state signal is based on user head or face orientation.

### Claim 52 (depends on 45)

The apparatus of claim 45, wherein the user state signal is based on user eye contact.

### Claim 53 (depends on 45)

The apparatus of claim 45, wherein the user state signal is based on user activity.

### Claim 54 (depends on 45)

The apparatus of claim 45, wherein the sensor comprises an infrared sensor.

### Claim 55

A method for modulating presentation of information on a screen of a device, comprising:
using at least one sensor embedded in the device to output a sensor signal;
processing the sensor signal to produce a user state signal that is indicative of user visual attention toward the screen of the device;
using the user state signal as a basis for determining whether to present the information on the screen of the device;
wherein the presentation of information is initiated by the device based on said determining.

### Claim 56 (depends on 55)

The method of claim 55, further comprising using the user state signal as a basis for determining whether pause the outputting of information, and whether to resume the outputting of information on the screen of the device;
wherein the pausing and resuming of the outputting of information is initiated by the device based on said determining.

### Claim 57 (depends on 55)

The method of claim 55, wherein the information includes audio information.

### Claim 58 (depends on 55)

The method of claim 55, wherein the device is selected from the group consisting of a personal computer, a cellular telephone, a telephone, a personal digital assistant (PDA), an electronic device, a machine, a smart appliance, and an appliance.

### Claim 59 (depends on 55)

The method of claim 55, wherein the device is a cellular telephone.

### Claim 60 (depends on 55)

The method of claim 55, wherein the outputting of information on the screen of the device includes the device soliciting user input.

### Claim 61 (depends on 55)

The method of claim 55, wherein the user state signal is based on user head or face orientation.

### Claim 62 (depends on 55)

The method of claim 55, wherein the user state signal is based on user eye contact.

### Claim 63 (depends on 55)

The method of claim 55, wherein the user state signal is based on user activity.

### Claim 64 (depends on 55)

The method of claim 55, comprising using an infrared sensor.
