# US 2016/0355188 A1 · Ambient vehicle and pedestrian state detection for device notification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0355188 A1
- Document ID: US20160355188A1
- Status: Published application
- Kind: A1
- Application number: 15/237,540
- Filed: 2016-08-15
- Published: 2016-12-08
- Earliest claimed: 2012-06-26
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Timothy S. Paek; Paramvir Bahl
- Web page: https://patentgrep.com/patent/US20160355188A1

## CPC

- B60W 40/04: Traffic conditions
- B60W 40/08: related to drivers or passengers
- G08G 1/0962: having an indicator mounted inside the vehicle, e.g. giving voice messages
- B60W 2040/0818: {Inactivity or incapacity of driver}
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

Described is a technology by which ambient data related to a vehicle is sensed and processed, for use in determining a state change related to external traffic awareness. Based upon the state change, an allowed level of interactivity with a user interface may be changed, and/or a notification may be output. Images and/or depth data may be sensed as part of determining whether a user who is interacting with a device in a stopped vehicle is to be made aware of the changed condition with respect to other vehicles, pedestrians and/or the like.

## Claims

### Claim 1

A method comprising:
allowing a first level of user interaction with an interactive device while a vehicle associated with the interactive device is moving;
monitoring ambient activity relative to the vehicle based on data received from one or more sensors, the ambient activity corresponding to external traffic awareness;
based on the data received from the one or more sensors, determining a change with respect to the ambient activity indicating the vehicle is not moving; and
based on the determined change, allowing a second level of user interaction with the interactive device while the vehicle is not moving.

### Claim 2 (depends on 1)

The method of claim 1, further comprising calibrating a stopped state for the vehicle when the vehicle is not moving.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining the change with respect to the ambient activity indicating the vehicle is not moving further comprises receiving information from a global positioning system.

### Claim 4 (depends on 1)

The method of claim 1, wherein allowing the second level of user interaction with the interactive device further comprises changing the first level of user interaction to the second level of user interaction, wherein the user interaction with at least one program is not allowed at the first level of user interaction, and wherein the user interaction with the at least one program is allowed at the second level of interaction.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining the change with respect to the ambient activity further comprises processing the data received from the set of sensors to detect a change in a state of at least one brake light of another vehicle.

### Claim 6 (depends on 1)

The method of claim 1, wherein monitoring the ambient activity relative to the vehicle further comprises:
capturing depth data from a depth camera; and
processing the depth data to determine that an object has decreased its depth relative to the vehicle over time.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining the change with respect to the ambient activity further comprises changing the first level of user interaction to the second level of user interaction based on a set of interaction-allowed rules.

### Claim 8 (depends on 1)

The method of claim 1, wherein ambient activity comprises pedestrian sensing, and wherein the method further comprising:
reducing pedestrian sensing sensitivity for a highway situation; and
increasing the pedestrian sensing sensitivity for city driving.

### Claim 9 (depends on 1)

The method of claim 1, further comprising allowing a third level of user interaction with the interactive device while the vehicle associated with the interactive device is in park mode, the park mode different from the vehicle being stopped.

### Claim 10 (depends on 1)

The method of claim 1, wherein the second level of user interaction and the third level of user interaction are allowed when the vehicle is in the park mode.

### Claim 11

A system comprising:
an interactive device associated with a vehicle, the interactive device providing a first level of interaction while the vehicle is stopped and a second level of user interaction with the interactive device while the vehicle is moving;
one or more sensors for detecting ambient activity relative to the vehicle; and
a processor programmed to:
allow the second level of user interaction with the interactive device while the vehicle associated with the interactive device is moving;
monitor ambient the activity relative to the vehicle based on data received from the one or more sensors;
based on the data received from the one or more sensors, determine a change with respect to the ambient activity indicating the vehicle is not moving; and
based on the determined change, allow the first level of user interaction with the interactive device while the vehicle is not moving.

### Claim 12 (depends on 11)

The system of claim 11, wherein determining the change with respect to the ambient activity indicating the vehicle is not moving further comprises receiving information from a global positioning system.

### Claim 13 (depends on 11)

The system of claim 11, wherein allowing the second level of user interaction with the interactive device further comprises changing the first level of user interaction to the second level of user interaction, wherein the user interaction with at least one program is not allowed at the first level of user interaction, and wherein the user interaction with the at least one program is allowed at the second level of interaction.

### Claim 14 (depends on 11)

The system of claim 11, wherein the ambient activity comprises at least one sensed object coming within a field of view of the vehicle, the at least one sensed object comprising a pedestrian or an animal.

### Claim 15 (depends on 11)

The system of claim 11, wherein the one or more sensors comprises at least one depth sensor configured to capture depth data.

### Claim 16

One or more computer-readable storage devices having computer-executable instructions, which when executed cause at least one processor to:
allow a first level of user interaction with the interactive device while the vehicle associated with the interactive device is moving;
monitor ambient activity relative to the vehicle based on data received from one or more sensors;
based on the data received from the one or more sensors, determine a change with respect to the ambient activity indicating the vehicle is not moving; and
based on the determined change, allow a second level of user interaction with the interactive device while the vehicle is not moving.

### Claim 17 (depends on 16)

The one or more computer-readable storage devices of claim 16, wherein the computer executable instructions are executed to further cause the at least one processor to sense an object coming within a field of view of the vehicle, wherein the sensed object comprises a pedestrian or an animal.

### Claim 18 (depends on 16)

The one or more computer-readable storage devices of claim 16, wherein determining the change with respect to the ambient activity indicating the vehicle is not moving further comprises receiving information from a global positioning system.

### Claim 19 (depends on 16)

The one or more computer-readable storage devices of claim 16, wherein allowing the second level of user interaction with the interactive device further comprises changing the first level of user interaction to the second level of user interaction, wherein the user interaction with at least one program is not allowed at the first level of user interaction, and wherein the user interaction with the at least one program is allowed at the second level of interaction.

### Claim 20 (depends on 16)

The one or more computer-readable storage devices of claim 16, wherein the one or more sensors comprises at least one depth sensor configured to capture depth data.
