# US 9,438,941 B2 · Using second screen devices to augment media engagement metrics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,438,941 B2
- Document ID: US9438941B2
- Status: Granted
- Kind: B2
- Application number: 14/974,009
- Filed: 2015-12-18
- Published: 2016-04-14
- Granted: 2016-09-06
- Earliest claimed: 2014-07-31
- Assignees: Google Inc.
- Applicants: Google Inc.
- Inventors: Alexander Collins; Brandon Bilinski
- Web page: https://patentgrep.com/patent/US9438941B2

## CPC

- H04N 21/252: {Processing of multiple end-users' preferences to derive collaborative data}
- H04N 21/24: Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests {(monitoring of server performance or load G06F11/34; arrangements for observation, testing or troubleshooting for broadcast or for distribution combined with broadcast H04H20/12)}
- H04N 21/4122: {additional display device, e.g. video projector (digital output for controlling a plurality of local displays G06F3/1423)}
- H04N 21/4126: {The peripheral being portable, e.g. PDAs or mobile phones}
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/44204: {Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched (monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31)}
- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)}
- H04N 21/4667: {Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections}
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)}
- H04N 21/8186: {specially adapted to be executed by a peripheral of the client device, e.g. by a reprogrammable remote control}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for using second screen devices to augment media engagement metrics. In one aspect, a method includes accessing user activity data describing user activities performed by a user on a first user device during a first time period, and accessing data describing presentation of one or more content items during the first time period on second user devices. For the first user device, determining based on the user activity data, for each of a plurality of second time periods that are sub-periods of the first time period, a first level of user engagement with the first user device, and determining, based on the user activity data, for each second time period that is a sub-period of the first time period, a second level of user engagement with the one or more content items presented.

## Claims

### Claim 1

A system, comprising:
a processing subsystem; and
a memory subsystem storing instructions that causes the processing subsystem to perform operations comprising:
accessing, for a plurality of first user devices, user activity data describing user activities for active applications performed by a user on the first user device during a first time period;
accessing data describing presentation of one or more content items during the first time period on second user devices, each second user device being different from each first user device;
for each first user device determining, based on the user activity data, for each of a plurality of second time periods that are sub-periods of the first time period, respective first levels of user engagement with the first user device for each second time period, wherein a first plurality of respective first levels of user engagement indicates engagement with the first user device and at least a second plurality of respective first levels of user engagement indicates disengagement with the first user device; and
determining, based on the user activity data, for each of the plurality of second time periods that are sub-periods of the first time period, respective second levels of user engagement with the one or more content items presented during the first time period, each determination based on a user model that models user engagement for a plurality of active applications and a plurality of content items, and wherein:
for a first active application according to the user model, a first level of user engagement indicating engagement with the first user device during a second time period is indicative of engagement with a content item presented on the second user device; and
for a second active application according to the user model, a first level of user engagement indicating engagement with the first user device during a second time period is indicative of disengagement with the content item presented on the second user device.

### Claim 2 (depends on 1)

The system of claim 1, wherein:
determining, for each of the plurality of second time periods, the respective first levels of user engagement with the first user device during the first time period comprises, for each second time period:
determining an active application on the first user device; and
determining user inputs to the active application.

### Claim 3 (depends on 2)

The system of claim 2, wherein determining, for each of the plurality of second time periods, the respective second level of user engagement with the one or more content items presented during the first time period comprises, for each second time period:
determining a particular content item being presented; and
determining, based on the particular content item being presented, the active application, and the user inputs to the active application, the second level of user engagement for the second time period.

### Claim 4 (depends on 1)

The system of claim 1, wherein the activity data describes binary levels of activity, and determining, for each of the plurality of second time periods, the respective first levels of user engagement with the first user device during the first time period comprises, for each second time period, determining the binary value of the activity data.

### Claim 5 (depends on 4)

The system of claim 4, wherein:
accessing data describing presentation of one or more content items during the first time period on a second user device that is different from the first user device comprises accessing log data for the second user device that is associated with the first user device by a user account.

### Claim 6 (depends on 2)

The system of claim 2, the operations further comprising:
determining, from the user activity data, a set of active applications, the set including at least two active applications; and
wherein:
determining, for each of the plurality of respective second time periods, the first level of user engagement with the first user device comprises determining the first level of user engagement for each active application in the set; and
determining, for each of the plurality of respective second time periods, the second level of user engagement with the one or more content items presented during the first time period comprises determining the second level of user engagement for each active application in the set.

### Claim 7 (depends on 2)

The system of claim 2, wherein accessing, for the plurality of first user devices, user activity data describing user activities performed by a user on the first user device during the first time period comprises receiving, from a reporting application on each first user device, the user activity data.

### Claim 8

A computer-implemented method, comprising:
accessing, for a plurality of first user devices, user activity data describing user activities for active applications performed by a user on the first user device during a first time period;
accessing data describing presentation of one or more content items during the first time period on second user devices, each second user device being different from each first user device;
for each first user device determining, based on the user activity data, for each of a plurality of second time periods that are sub-periods of the first time period, respective first levels of user engagement with the first user device for each second time period, wherein a first plurality of respective first levels of user engagement indicates engagement with the first user device and at least a second plurality of respective first levels of user engagement indicates disengagement with the first user device; and
determining, based on the user activity data, for each of the plurality of second time periods that are sub-periods of the first time period, respective second levels of user engagement with the one or more content items presented during the first time period, each determination based on a user model that models user engagement for a plurality of active applications and a plurality of content items, and wherein:
for a first active application according to the user model, a first level of user engagement indicating engagement with the first user device during a second time period is indicative of engagement with a content item presented on the second user device; and
for a second active application according to the user model, a first level of user engagement indicating engagement with the first user device during a second time period is indicative of disengagement with the content item presented on the second user device.

### Claim 9 (depends on 8)

The method of claim 8, wherein:
determining, for each of the plurality of second time periods, the respective first levels of user engagement with the first user device during the first time period comprises, for each second time period:
determining an active application on the first user device; and
determining user inputs to the active application.

### Claim 10 (depends on 9)

The method of claim 9, wherein determining, for each of the plurality of second time periods, the respective second level of user engagement with the one or more content items presented during the first time period comprises, for each second time period:
determining a particular content item being presented; and
determining, based on the particular content item being presented, the active application, and the user inputs to the active application, the second level of user engagement for the second time period.

### Claim 11 (depends on 8)

The method of claim 8, wherein the activity data describes binary levels of activity, and determining, for each of the plurality of second time periods, the respective first levels of user engagement with the first user device during the first time period comprises, for each second time period, determining the binary value of the activity data.

### Claim 12 (depends on 11)

The method of claim 11, wherein:
accessing data describing presentation of one or more content items during the first time period on a second user device that is different from the first user device comprises accessing log data for the second user device that is associated with the first user device by a user account.

### Claim 13 (depends on 9)

The method of claim 9, the operations further comprising:
determining, from the user activity data, a set of active applications, the set including at least two active applications; and
wherein:
determining, for each of the plurality of respective second time periods, the first level of user engagement with the first user device comprises determining the first level of user engagement for each active application in the set; and
determining, for each of the plurality of respective second time periods, the second level of user engagement with the one or more content items presented during the first time period comprises determining the second level of user engagement for each active application in the set.

### Claim 14 (depends on 9)

The method of claim 9, wherein accessing, for the plurality of first user devices, user activity data describing user activities performed by a user on the first user device during the first time period comprises receiving, from a reporting application on each first user device, the user activity data.