# US 11,909,611 B2 · Standardizing analysis metrics across multiple devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,909,611 B2
- Document ID: US11909611B2
- Status: Granted
- Kind: B2
- Application number: 17/869,485
- Filed: 2022-07-20
- Published: 2024-01-25
- Granted: 2024-02-20
- Earliest claimed: 2022-07-20
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Prithwish Mukherjee; Sujee Rajayogam
- Web page: https://patentgrep.com/patent/US11909611B2

## CPC

- G06F 11/3438: {monitoring of user actions (tracking the activity of the user H04L67/535)}
- H04L 43/065: related to network devices
- G06F 11/3419: {by assessing time}
- G06F 17/40: Data acquisition and logging (for input to computer G06F3/00)
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- H04L 41/02: Standardisation; Integration
- G06F 11/3428: {Benchmarking}
- G10L 2015/228: {of application context}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Implementations relate to generating standardized metrics from device specific metrics that are generated during an interaction between a user and an automated assistant. The metrics indicate events that occurred while processing an interaction of a user with the automated assistant and are specific to the particular configuration of the device with which the user is interacting. Conversion mappings are determined based on device characteristics that can be utilized to convert the device metrics into standardized metrics. Analysis metrics are generated based on the standardized metrics that are incapable of being generated from the device metrics. Some implementations include visually rendering the analysis metrics such that one or more of the analysis metrics are rendered more prominently than other metrics.

## Claims

### Claim 1

A method, performed by one or more processors, the method comprising:
identifying device-specific metrics generated during an interaction between a user and an automated assistant that is executing on a client device, wherein the device-specific metrics indicate timing of events that occurred during the interaction, including a device delay introduced by device hardware that is specific to the client device;
determining, based on one or more characteristics of the device hardware, a plurality of conversion mappings;
generating standardized metrics, wherein generating the standardized metrics includes converting one or more metrics of the device-specific metrics to one or more standard metrics using the conversion mappings; and
causing a remote computing device to process the standardized metrics in generating analysis metrics, wherein the remote computing device is incapable of generating the analysis metrics from the device-specific metrics.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the standardized metrics is performed by the client device, and wherein causing the remote computing device to process the standardized metrics in generating the analysis metrics comprises:
transmitting, by the client device, the standardized metrics to the remote computing device.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating the standardized metrics is performed by a second remote computing device, and further comprising:
transmitting, by the client device, the device-specific metrics to the second remote computing device;
wherein causing the remote computing device to process the standardized metrics in generating the analysis metrics comprises:
transmitting, by the second remote computing device, the standardized metrics to the remote computing device.

### Claim 4 (depends on 1)

The method of claim 1, wherein, in generating the analysis metrics, the remote computing device compares the standardized metrics to second standardized metrics.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
identifying second device-specific metrics generated during an interaction between an additional user and a second automated assistant that is executing on a second client device;
determining, based on one or more characteristics of the second client device, a plurality of second conversion mappings; and
generating the second standardized metrics, wherein generating the second standardized metrics includes converting one or more of the metrics of the second device-specific metrics using the second conversion mappings.

### Claim 6 (depends on 1)

The method of claim 1, wherein the client device is further executing an application, and wherein at least a given metric, of the device-specific metrics is based on timing of one or more events related to the application.

### Claim 7 (depends on 1)

The method of claim 1, wherein a given standardized metric, of the standardized metrics, indicates timing of a given event that is not included among the events indicated by the device-specific metrics.

### Claim 8 (depends on 7)

The method of claim 7, wherein generating the standardized metrics comprises using a given conversion mapping, of the conversion mappings, to infer the given standardized metric based on one or more of the device-specific metrics.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
generating the given conversion mapping based on a timing variation caused by a component, and based on determining the client device includes the component.

### Claim 10 (depends on 7)

The method of claim 7, wherein the given event is a response, from an application, to a request included in a spoken utterance that is processed by the automated assistant.

### Claim 11

A method, performed by on one or more processors, the method comprising:
receiving, from a first client device, first device-specific metrics generated during a first interaction between a first user and a first instance of an automated assistant that is executing on the first client device, wherein the first device-specific metrics indicate timing of events that occurred during the first interaction, including a first device delay introduced by first device hardware that is specific to the first device;
receiving, from a second client device, second device-specific metrics generated during a second interaction between a second user and a second instance of the automated assistant that is executing on the second client device, wherein the second device-specific metrics indicate timing of events that occurred during the second interaction, including a second device delay introduced by second device hardware that is specific the second device, wherein the second device delay is different than the first device delay;
determining, based on one or more first characteristics of the first device hardware, a plurality of first conversion mappings;
determining, based on one or more second characteristics of the second device hardware, a plurality of second conversion mappings;
generating first standardized metrics, wherein generating the first standardized metrics includes converting one or more of the first device-specific metrics to one or more standard metrics using the first conversion mappings;
generating second standardized metrics, wherein generating the second standardized metrics includes converting one or more of the second device-specific metrics to one or more of the standard metrics using the second conversion mappings;
comparing at least one of the first standardized metrics to at least one of the second standardized metrics; and
generating analysis metrics based on the comparison.

### Claim 12 (depends on 11)

The method of claim 11, wherein at least one of the characteristics of the first device hardware is different from at least one of the characteristics of the second device hardware.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
receiving, from the first client device, additional first device characteristics;
receiving, from the second client device, additional second device characteristics; and
prior to comparing at least one of the first device-specific metrics to at least one of the second device-specific metrics:
determining that at least one of the additional first device characteristics is similar to at least one of the additional second device characteristics.

### Claim 14 (depends on 13)

The method of claim 13, wherein the additional first device characteristics includes a geographic location of the first client device.

### Claim 15 (depends on 13)

The method of claim 13, wherein the additional first device characteristics includes a language preference setting of the first client device.

### Claim 16 (depends on 11)

The method of claim 11, further comprising:
rendering an interface that indicates one or more of the standardized metrics.

### Claim 17 (depends on 16)

The method of claim 16, further comprising:
identifying, for an analysis metric of interest, a benchmark standard; and
visually rendering the analysis metric of interest based on whether the analysis metric of interest satisfies the benchmark standard.

### Claim 18

A method, performed by on one or more processors, the method comprising:
receiving first device-specific metrics generated during a first interaction between a first user and a first instance of an automated assistant that is executing on a first client device, wherein the first device-specific metrics indicate timing of events that occurred during the first interaction, including a first device delay introduced by a first device operating system that is specific to the first client device;
receiving second device-specific metrics generated during a second interaction between a second user and a second instance of the automated assistant that is executing on a second client device, wherein the second device-specific metrics indicate timing of events that occurred during the second interaction, including a second device delay introduced by a second device operating system that is specific the second device, wherein the second device delay is different than the first device delay;
determining, based on one or more first characteristics of the first device operating system, a plurality of first conversion mappings;
determining, based on one or more second characteristics of the second device operating system, a plurality of second conversion mappings;
generating first standardized metrics, wherein generating the first standardized metrics includes converting one or more of the first device-specific metrics to one or more standard metrics using the first conversion mappings;
generating second standardized metrics, wherein generating the second standardized metrics includes converting one or more of the second device-specific metrics to one or more of the standard metrics using the second conversion mappings;
comparing at least one of the first standardized metrics to at least one of the second standardized metrics; and
generating analysis metrics based on the comparison.