# US 10,313,422 B2 · Controlling a device based on log and sensor data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,313,422 B2
- Document ID: US10313422B2
- Status: Granted
- Kind: B2
- Application number: 15/294,826
- Filed: 2016-10-17
- Published: 2018-04-19
- Granted: 2019-06-04
- Earliest claimed: 2016-10-17
- Assignees: Hitachi, Ltd.
- Applicants: Hitachi, Ltd.
- Inventors: Hiroaki Ozaki; Abhay Mehta; Hsiu-Khuern Tang; Shuang Feng; Haiyan Wang
- Web page: https://patentgrep.com/patent/US10313422B2

## CPC

- H04L 67/025: for remote control or remote monitoring of applications
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/09: Supervised learning
- G16H 40/60: for the operation of medical equipment or devices
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- H04L 43/10: Active monitoring, e.g. heartbeat, ping or trace-route
- H04L 67/306: User profiles
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

In some examples, a computing device may receive sensor data for a target and at least one of: log data for the target, or historical log data and historical sensor data for a plurality of other targets. The computing device may determine at least one event classified as a non-uniform event in at least one of the log data or the historical log data, and may determine combined features, such as a feature vector, based on the sensor data and the non-uniform event(s). The computing device may determine an analysis result from the combined features. Further, based on the analysis result, the computing device may send a control signal to a device associated with the target for controlling the device, and/or may send a communication related to the target to another computing device.

## Claims

### Claim 1

A system comprising:
one or more processors; and
one or more non-transitory computer-readable media maintaining executable instructions, which, when executed by the one or more processors, program the one or more processors to perform operations comprising:
receiving sensor data for a patient and at least one of:
log data for the patient; or
historical log data and historical sensor data for a plurality of patients;
determining at least one event classified as a non-uniform event in at least one of the log data or the historical log data;
determining a feature vector based on the sensor data and the at least one non-uniform event;
determining, based on the feature vector, an analysis result related to the patient; and
sending at least one of:
a notification or determination to a computing device; or
a control signal to a device for controlling a device.

### Claim 2 (depends on 1)

The system as recited in claim 1, wherein the operation of determining at least one event classified as the non-uniform event in at least one of the log data or the historical log data comprises:
determining a plurality of events in the at least one of the log data or the historical log data;
classifying events of the plurality of events as either uniform events or non-uniform events to determine the at least one non-uniform event; and
excluding at least one uniform event when determining the feature vector.

### Claim 3 (depends on 2)

The system as recited in claim 2, wherein the operation of classifying events of the plurality of events as either uniform events or non-uniform events comprises:
determining a plurality of events of different event categories, each occurrence of each event category associated with a timing;
determining a standard deviation of the timing of each occurrence of each event for each event category; and
classifying events associated with a standard deviation less than a threshold standard deviation as uniform and events associated with a standard deviation greater than the threshold standard deviation as non-uniform.

### Claim 4 (depends on 1)

The system as recited in claim 1, wherein the operation of determining the feature vector based on the sensor data and the at least one non-uniform event comprises:
determining time segments in a time window based on the at least one non-uniform event;
for each time segment, determining features from the sensor data corresponding to the time segment;
aggregating the features for time segments corresponding to a same order of occurrence of non-uniform events; and
determining the feature vector by concatenating the aggregated features for the time segments.

### Claim 5 (depends on 4)

The system as recited in claim 4, the operations further comprising:
for each time segment, determining an N-gram label based on the order of occurrence of the non-uniform events; and
aggregating the features for the time segments corresponding to a same order of occurrence of non-uniform events by aggregating features corresponding to the same N-gram, wherein the features include at least one of a mean of the sensor data within each time segment, a median of the sensor data within each time segment, or a standard deviation of the sensor data within each time segment.

### Claim 6 (depends on 1)

The system as recited in claim 1, the operations further comprising:
training a statistical model using the historical sensor data and at least one non-uniform event from the historical log data;
executing the statistical model using the sensor data and at least one non-uniform event from the log data in a selected time window;
determining time segments based at least in part on a hidden state of the statistical model;
for each time segment, determining one or more features from the sensor data;
aggregating the features based on matching hidden states; and
concatenating the aggregated features to determine the feature vector.

### Claim 7 (depends on 1)

The system as recited in claim 1, the operations further comprising:
training a neural network using the historical sensor data and at least one non-uniform event from the historical log data;
executing the neural network using the sensor data in a selected time window to determine one or more features from a cell of the neural network based on the sensor data; and
concatenating the features to determine the feature vector.

### Claim 8

A method comprising:
receiving, by a processor, sensor data for a target and at least one of:
log data for the target; or
historical log data and historical sensor data for a plurality of other targets, wherein the log data and the historical log data include information related to the target generated by a human;
determining at least one event classified as a non-uniform event in at least one of the log data or the historical log data;
determining combined features based on the sensor data and the at least one non-uniform event; and
determining, based on the combined features, an analysis result related to the target.

### Claim 9 (depends on 8)

The method as recited in claim 8, further comprising:
determining a feature vector as the combined features;
determining the analysis result from the combined features based at least in part on at least one of regression or classification applied to the feature vector; and
based on the analysis result, sending at least one of:
a notification or determination related to the target to a computing device; or
a control signal to a device associated with the target for controlling the device.

### Claim 10 (depends on 8)

The method as recited in claim 8, further comprising:
determining a plurality of events in the at least one of the log data or the historical log data;
classifying events of the plurality of events as either uniform events or non-uniform events to determine the at least one event classified as the non-uniform event; and
excluding at least one event classified as a uniform event when determining the features.

### Claim 11 (depends on 8)

The method as recited in claim 8, further comprising:
determining time segments in a time window based on occurrences of a plurality of non-uniform events;
for individual time segments, determining features from the sensor data corresponding to the time segment;
aggregating the features according to time segments that correspond to a same order of occurrence of non-uniform events; and
determining the combined features by concatenating the aggregated features for the time segments.

### Claim 12 (depends on 11)

The method as recited in claim 11, further comprising determining the features from the sensor data for the individual time segments by determining at least one of:
a mean of the sensor data for each time segment,
a median of the sensor data for each time segment, or
a standard deviation of the sensor data for each time segment.

### Claim 13 (depends on 8)

The method as recited in claim 8, further comprising:
training a hidden Markov model (HMM) using the historical sensor data and at least one non-uniform event from the historical log data;
executing the HMM using the sensor data and at least one non-uniform event from the log data in a selected time window;
determining time segments based at least in part on a hidden state of the HMM;
for each time segment, determining one or more features from the sensor data;
aggregating the features based on matching hidden states; and
concatenating the aggregated features to determine the combined features.

### Claim 14 (depends on 8)

The method as recited in claim 8, further comprising:
training a neural network using the historical sensor data and at least one non-uniform event from the historical log data;
executing the neural network using the sensor data in a selected time window to determine one or more features from a cell of the neural network based on the sensor data; and
concatenating the features to determine the combined features.

### Claim 15

A system comprising:
one or more processors; and
one or more non-transitory computer-readable media maintaining executable instructions, which, when executed by the one or more processors, program the one or more processors to:
receive sensor data for a target and at least one of:
log data for the target; or
historical log data and historical sensor data for a plurality of other targets;
determine at least one event classified as a non-uniform event in at least one of the log data or the historical log data;
determine a feature vector based on the sensor data and the at least one non-uniform event; and
determine, based on the feature vector, an analysis result related to the target.

### Claim 16 (depends on 15)

The system as recited in claim 15, wherein, based on the analysis result, the one or more processors are further programmed to send a control signal to a device associated with the target for controlling the device in response to a condition indicated by the analysis result.

### Claim 17 (depends on 15)

The system as recited in claim 15, wherein the one or more processors are further programmed to determine at least one event classified as the non-uniform event by:
determining a plurality of events in the at least one of the log data or the historical log data; and
classifying events of the plurality of events as either uniform events or non-uniform events to determine the at least one non-uniform event.

### Claim 18 (depends on 15)

The system as recited in claim 15, wherein the one or more processors are further programmed to:
determine time segments in a time window based on the at least one non-uniform event;
for individual time segments, determine features from the sensor data corresponding to the time segment;
aggregate the features for time segments corresponding to a same order of occurrence of non-uniform events; and
determine the feature vector by concatenating the aggregated features for the time segments.

### Claim 19 (depends on 15)

The system as recited in claim 15, wherein the one or more processors are further programmed to:
train a statistical model using the historical sensor data and at least one non-uniform event from the historical log data;
execute the statistical model using the sensor data and at least one non-uniform event from the log data in a selected time window;
determine time segments based at least in part on a hidden state of the statistical model;
for each time segment, determine one or more features from the sensor data;
aggregate the features based on matching hidden states; and
concatenate the aggregated features to determine the feature vector.

### Claim 20 (depends on 15)

The system as recited in claim 15, wherein the one or more processors are further programmed to:
train a neural network using the historical sensor data and at least one non-uniform event from the historical log data;
execute the neural network using the sensor data in a selected time window to determine one or more features from a cell of the neural network based on the sensor data; and
concatenate the features to determine the feature vector.