# US 7,613,670 B2 · Precomputation of context-sensitive policies for automated inquiry and action under uncertainty

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 7,613,670 B2
- Document ID: US7613670B2
- Status: Granted
- Kind: B2
- Application number: 11/969,053
- Filed: 2008-01-03
- Published: 2008-07-03
- Granted: 2009-11-03
- Earliest claimed: 2005-06-29
- Assignees: Microsoft Corporation
- Inventors: Eric J. Horvitz; Paul B. Koch; Raman K. Sarin
- Web page: https://patentgrep.com/patent/US7613670B2

## CPC

- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06F 8/35: model driven
- G06F 8/36: Software reuse
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Learning, inference, and decision making with probabilistic user models, including considerations of preferences about outcomes under uncertainty, may be infeasible on portable devices. The subject invention provides systems and methods for pre-computing and storing policies based on offline preference assessment, learning, and reasoning about ideal actions and interactions, given a consideration of uncertainties, preferences, and/or future states of the world. Actions include ideal real-time inquiries about a state, using pre-computed value-of-information analyses. In one specific example, such pre-computation can be applied to automatically generate and distribute call-handling policies for cell phones. The methods can employ learning of Bayesian network user models for predicting whether users will attend meetings on their calendar and the cost of being interrupted by incoming calls should a meeting be attended.

## Claims

### Claim 1

A method to facilitate mobile device operations, comprising:
employing a processor to execute computer executable instructions stored on a computer readable storage medium to implement the following acts:
utilizing each of a training input and a user input to develop logical rules or probabilistic user models in an offline computing environment, the probabilistic user models are associated with a model of interruptibilitv or a model of attendance;
generating a policy component that includes decision-making instructions derived from at least one of the logical rules or probabilistic user models developed in the offline computing environment; and
transmitting the policy component to a mobile device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising providing policies associated with the policy component to facilitate making decisions about call handling, including one or more of: whether to send a call silently to a voicemail system: ring or vibrate with a particular chime and intensity: display information visually: play a particular outgoing message: let a caller know when a user will next be available; allow the caller to indicate a desire to ring through to the user after hearing an outgoing message describing aspects of the user's current context: or allow the caller to reschedule the call to another available time.

### Claim 3 (depends on 1)

The method of claim 1, further comprising providing policies that allow for sensing of local state or state changes with one or more of a microphone, photocell, camera, accelerometer, thermometer, barometer, GPS device, or wireless radio signals, where the wireless signals include GSM strengths, which are embedded in a device or connected to a device via wire or wireless connection.

### Claim 4 (depends on 1)

The method of claim 1, further comprising using at least one user interface to train the associated probabilistic user models.

### Claim 5 (depends on 1)

The method of claim 1, further comprising communicating via a client component on the device to interact with a desktop computing application.

### Claim 6

A computer implemented method for developing model policies system, comprising:
employing a processor to execute computer executable instructions stored on a computer readable storage medium to implement the following acts:
recording a set of appointment properties associated with a user's electronic calendar;
utilizing the set of appointment properties associated with the user's electronic calendar to develop logical rules or probabilistic user models in an offline computing environment;
developing a policy component that includes decision-making instructions that are derived from the logical rules or probabilistic user models; and
encoding the policy component for implementation onto a mobile device.

### Claim 7 (depends on 6)

The method of claim 6, further comprising recording a set of appointment properties by crawling a user's online calendar.

### Claim 8 (depends on 6)

The method of claim 6, further comprising recording for each appointment, one or more of: time of day of the appointment, day of week of the appointment, appointment duration, subject of the appointment, location of the appointment, organizer of the appointment, the response status of the user, whether the appointment is recurrent or not, whether the time is marked as busy or free on the user's calendar, whether the user was required or optional, the number of invitees, the organizational relationships of the invitees to the user, and the role of the user.

### Claim 9 (depends on 6)

The method of claim 6, wherein the probabilistic user models are associated with a model of interruptibility or a model of attendance.

### Claim 10 (depends on 6)

The method of claim 6, further comprising using at least one user interface to train the models associated with the probabilistic user models.

### Claim 11 (depends on 6)

The method of claim 6, further comprising training the probabilistic user models from explicit or implicit user input.

### Claim 12 (depends on 6)

The method of claim 6, further comprising implementing at least one Bayesian network model or graphical model with the probabilistic user models.

### Claim 13 (depends on 6)

The method of claim 6, further comprising executing a cost/benefit analysis associated with the probabilistic user models, where policies are created in accordance with a goal of maximizing expected value or minimizing an expected cost based upon a consideration of a likelihood of different future situations.

### Claim 14 (depends on 6)

The method of claim 6, further comprising determining an expected cost of interruption (ECI) computation associated with the probabilistic user models.

### Claim 15 (depends on 14)

The method of claim 14, further comprising determining the ECI computation using the following equation:
ECI
=
p
⁡
(
A
❘
E
)
⁢
∑
i
⁢
⁢
p
⁡
(
c
i
❘
E
)
⁢
c
i
+
(
1
-
p
⁡
(
A
❘
E
)
)
⁢
c
b
⁡
(
S
)
;
where p(A|E) is a likelihood that users will attend a meeting, given evidential properties E associated with the meeting obtained via calendar appointment properties, p(ci|E) is a probability that users assign a cost ci to the meeting, where i indexes the meeting as being low, medium, or high cost, and cb(S) is a background cost of being interrupted in a context S, representing where a user does not attend a meeting, as captured by a time of day and day of week.

### Claim 16 (depends on 14)

The method of claim 14, further comprising determining a default ECI value when a user is not attending an appointment.

### Claim 17 (depends on 6)

The method of claim 6, further comprising determining an expected cost of deferral associated with the probabilistic user models based, at least in part on, organizational relationships and activities.

### Claim 18 (depends on 6)

The method of claim 6, further comprising determining at least one of an expected cost of communications (ECC) and an expected value of information (EVI) to determine at least one dialog query.

### Claim 19 (depends on 18)

The method of claim 18, further comprising executing the following equation where the ECC for a meeting of duration t based on a consideration of current evidence E with respect to properties of an upcoming meeting is,
ECC
⁡
(
E
,
t
)
=
(
∑
i
⁢
⁢
f
i
⁢
c
i
defer
+
∑
j
⁢
⁢
f
j
⁢
c
j
ring
)
⁢
t
and where fi is a frequency of calls in each caller group i that has a cost of deferral lower than a cost of interruption, fj is a frequency of calls in each caller group j that has a cost of deferral higher than the cost of interruption, and cdefer and cring are the costs of deferral and cost of interruption of respective caller classes.

### Claim 20

A system to facilitate mobile device operations, comprising:
a processor;
a memory communicatively coupled to the processor and storing computer readable instructions for implementing the following components;
a learning component for learning user models in an offline computing environment, the user models learned by utilizing each of a training input and a user input in the offline computing environment, the user models are associated with a model of interruptibility or a model of attendance;
an encoding component for encoding an output from the user model according to one or more control policies; and
a transmission component for transmitting the control policies to a remote device, the control policies executable in an online computing environment at the remote device apart from the offline computing environment.