# US 2022/0036863 A1 · Smartphone Providing Radar-Based Proxemic Context

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036863 A1
- Document ID: US20220036863A1
- Status: Published application
- Kind: A1
- Application number: 17/504,323
- Filed: 2021-10-18
- Published: 2022-02-03
- Earliest claimed: 2018-08-22
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Leonardo Giusti; Ivan Poupyrev; Brandon Barbello; Patrick M. Amihood
- Web page: https://patentgrep.com/patent/US20220036863A1

## CPC

- H04M 1/72454: according to context-related or environment-related conditions
- G01S 7/41: using analysis of echo signal for target characterisation; Target signature; Target cross-section
- G01S 7/415: {Identification of targets based on measurements of movement associated with the target}
- G01S 7/417: {involving the use of neural networks}
- G01S 13/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S13/46)
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06V 10/143: Sensing or illuminating at different wavelengths
- G06V 10/242: {by image rotation, e.g. by 90 degrees}
- G09G 5/38: with means for controlling the display position
- H04M 1/724: User interfaces specially adapted for cordless or mobile telephones
- G06F 1/3203: Power management, i.e. event-based initiation of a power-saving mode
- G06F 1/3231: Monitoring the presence, absence or movement of users
- G06F 2203/041: Indexing scheme relating to G06F3/041 - G06F3/045
- G06F 2203/04101: 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
- G06F 2203/04108: Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
- G09G 2340/0492: Change of orientation of the displayed image, e.g. upside-down, mirrored
- G09G 2354/00: Aspects of interface with display user
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

This document describes techniques and systems that enable a smartphone providing radar-based proxemic context. The techniques and systems use a radar field to accurately determine a user's location and/or physical orientation with respect to an electronic device, such as a smartphone. The radar field also enables the device to receive 3D gestures from the user to interact with the device. The techniques allow the device to provide functionality based on the user's presence and/or orientation, and to appropriately adjust the timing, content, and format of the device's interactions with the user.

## Claims

### Claim 1

A method implemented in an electronic device that includes a radar system and an electronic assistant application, the method comprising:
providing, by the radar system, a radar field;
sensing, by the radar system, reflections from an object in the radar field;
analyzing the reflections from the object in the radar field;
providing, based on the analysis of the reflections, radar data;
determining, based on a first subset of the radar data, a distance between the object and the electronic device;
based on the distance between the object and the electronic device, operating the electronic assistant application in a first mode;
determining, based on a second subset of the radar data, a change in the distance between the object and the electronic device; and
responsive to determining the change of the distance between the object and the electronic device, operating the electronic assistant application in a second mode.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the first mode comprises:
providing communication from the electronic assistant application in a non-visual format, the non-visual format comprising audible communication; or
providing communication from the electronic assistant application in a visual format, the visual format comprising communication using a visible element; and
the second mode comprises:
providing communication from the electronic assistant application in a non-visual format when the first mode comprises providing communication from the electronic assistant application in the visual format, the non-visual format comprising audible communication; or
providing communication from the electronic assistant application in a visual format when the first mode comprises providing communication from the electronic assistant application in the non-visual format, the visual format comprising communication using a visible element.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
determining, based on the first subset of the radar data, the distance between the object and the electronic device further comprises determining that the distance between the object and the electronic device is greater than a threshold distance;
the first mode comprises providing communication from the electronic assistant application in a non-visual format;
determining, based on the second subset of the radar data, the change in the distance between the object and the electronic device further comprises determining that the distance between the object and the electronic device is less than the threshold distance; and
the second mode comprises providing communication from the electronic assistant application in a visual format.

### Claim 4 (depends on 3)

The method of claim 3, wherein:
the first mode further comprises providing communication from the electronic assistant application in only the non-visual format; and
the second mode further comprises providing communication from the electronic assistant application in only the visual format.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
determining, based on the first subset of the radar data, the distance between the object and the electronic device further comprises determining that the distance between the object and the electronic device is less than a threshold distance;
the first mode comprises providing communication from the electronic assistant application in a visual format;
determining, based on the second subset of the radar data, the change in the distance between the object and the electronic device further comprises determining that the distance between the object and the electronic device is greater than the threshold distance; and
the second mode comprises providing communication from the electronic assistant application in a non-visual format.

### Claim 6 (depends on 5)

The method of claim 5, wherein:
the first mode further comprises providing communication from the electronic assistant application in only the visual format; and
the second mode further comprises providing communication from the electronic assistant application in only the non-visual format.

### Claim 7

An electronic device, comprising:
a radar system, implemented at least partially in hardware, configured to:
provide a radar field;
sense reflections from an object in the radar field;
analyze the reflections from the object in the radar field; and
provide, based on the analysis of the reflections, radar data;
a computer processor; and
a computer-readable media having instructions stored thereon that, responsive to execution by the one or more computer processors, implement an electronic assistant configured to:
determine, based on a subset of the radar data, a distance between the object and the electronic device; and
responsive to the determination of the distance between the object and the electronic device, operate in a first mode.

### Claim 8 (depends on 7)

The method of claim 7, wherein:
operation in the first mode comprises:
providing communication from the electronic assistant in a non-visual format, the non-visual format comprising audible communication; or
providing communication from the electronic assistant in a visual format, the visual format comprising communication using a visible element; and
the electronic assistant is further configured to:
determine, based on the subset of the radar data or another subset of the radar data, another distance between the object and the electronic device; and
responsive to the determination of the other distance between the object and the electronic device, operate in a second mode, operation in the second mode comprising:
providing communication from the electronic assistant in the non-visual format when the first mode comprises providing communication from the electronic assistant in the visual format, the non-visual format comprising audible communication; or
providing communication from the electronic assistant in the visual format when the first mode comprises providing communication from the electronic assistant in the non-visual format, the visual format comprising communication using a visible element.

### Claim 9 (depends on 7)

The electronic device of claim 7, wherein:
the distance between the object and the electronic device is within a threshold distance; and
operation in the first mode comprises providing communication from the electronic assistant in a visual format.

### Claim 10 (depends on 9)

The electronic device of claim 9, wherein the electronic assistant is further configured to:
determine, based on the subset of the radar data or on another subset of the radar data, that the object is outside the threshold distance; and
responsive to the determination that the object is outside the threshold distance, operate in a second mode, the second mode comprising providing communication from the electronic assistant in a non-visual format.

### Claim 11 (depends on 9)

The electronic device of claim 9, wherein:
the first mode further comprises presenting content, by the electronic assistant, in only the visual format; and
the electronic assistant is further configured to:
determine, based on the subset of the radar data or another subset of the radar data, that the object is outside the threshold distance; and
responsive to the determination that the object is outside the threshold distance, operate in a second mode, the second mode comprising providing communication from the electronic assistant in only a non-visual format.

### Claim 12 (depends on 7)

The electronic device of claim 7, wherein:
the distance between the object and the electronic device exceeds a threshold distance; and
operation in the first mode comprises providing communication from the electronic assistant in a non-visual format.

### Claim 13 (depends on 12)

The electronic device of claim 12, wherein the electronic assistant is further configured to:
determine, based on the subset of the radar data or on another subset of the radar data, that the object is within the threshold distance; and
responsive to the determination that the object is within the threshold distance, operate in a second mode, the second mode comprising providing communication from the electronic assistant in a visual format.

### Claim 14 (depends on 12)

The electronic device of claim 12, wherein:
the first mode further comprises providing communication from the electronic assistant in only the non-visual format; and
the electronic assistant is further configured to:
determine, based on another subset of the radar data, that the object is within the threshold distance; and
responsive to the determination that the object is within the threshold distance, operate in a second mode, the second mode comprising providing communication from the electronic assistant in only a visual format.

### Claim 15 (depends on 7)

The electronic device of claim 7, wherein:
the electronic assistant is further configured to determine whether the distance between the object and the electronic device exceeds a threshold distance, the threshold distance being between approximately three feet and approximately fourteen feet; and
the threshold distance is determined by at least one of:
a predefined setting;
a user selection; or
a machine-learning module of the electronic device.

### Claim 16

A system, comprising:
an electronic device;
a radar system, implemented at least partially in hardware, configured to:
provide a radar field;
sense reflections from an object in the radar field;
analyze the reflections from the object in the radar field; and
provide, based on the analysis of the reflections, radar data;
one or more computer processors; and
one or more computer-readable media having instructions stored thereon that, responsive to execution by the one or more computer processors, perform operations comprising:
determining, based on a first subset of the radar data, a distance between the object and the electronic device;
based on the distance between the object and the electronic device, providing notifications from the electronic device in a first mode;
determining, based on a second subset of the radar data, a change of the distance between the object and the electronic device; and
responsive to determining the change of the distance between the object and the electronic device, providing notifications from the electronic device in a second mode.

### Claim 17 (depends on 16)

The system of claim 16, wherein:
the first mode comprises:
providing notifications from the electronic device in a non-visual format, the non-visual format comprising audible communication; or
providing notifications from the electronic device in a visual format, the visual format comprising communication using a visible element; and
the second mode comprises:
providing notifications from the electronic device in the non-visual format when the first mode comprises providing notifications from the electronic device in the visual format, the non-visual format comprising audible communication; or
providing notifications from the electronic device in the visual format when the first mode comprises providing notifications from the electronic device in the non-visual format, the visual format comprising communication using a visible element.

### Claim 18 (depends on 16)

The system of claim 16, wherein the operations further comprise:
determining, based on the first subset of the radar data, the distance between the object and the electronic device further comprises determining that the distance between the object and the electronic device is greater than a threshold distance;
the first mode comprises providing notifications from the electronic device in a non-visual format;
determining, based on the second subset of the radar data, the change in the distance between the object and the electronic device further comprises determining that the distance between the object and the electronic device is less than the threshold distance; and
the second mode comprises providing notifications from the electronic device in a visual format.

### Claim 19 (depends on 18)

The system of claim 18, wherein:
the first mode further comprises providing notifications from the electronic device in only the non-visual format; and
the second mode further comprises providing notifications from the electronic device in only the visual format.

### Claim 20 (depends on 16)

The system of claim 16, wherein:
determining, based on the first subset of the radar data, the distance between the object and the electronic device further comprises determining that the distance between the object and the electronic device is less than a threshold distance;
the first mode comprises providing notifications from the electronic device in a visual format;
determining, based on the second subset of the radar data, the change in the distance between the object and the electronic device further comprises determining that the distance between the object and the electronic device is greater than the threshold distance; and
the second mode comprises providing notifications from the electronic device in a non-visual format.

### Claim 21 (depends on 20)

The system of claim 20, wherein:
the first mode further comprises providing notifications from the electronic device in only the visual format; and
the second mode further comprises providing notifications from the electronic device in only the non-visual format.

### Claim 22 (depends on 16)

The system of claim 16, wherein the radar system further comprises a digital beamformer and an angle estimator, and the radar system is configured to monitor angles in a field of view between approximately −90 degrees and approximately 90 degrees.
