# US 2015/0178265 A1 · Content Recommendation System using a Neural Network Language Model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0178265 A1
- Document ID: US20150178265A1
- Status: Published application
- Kind: A1
- Application number: 14/136,111
- Filed: 2013-12-20
- Published: 2015-06-25
- Earliest claimed: 2013-12-20
- Assignees: Google Inc.
- Applicants: Google Inc.
- Inventors: Glen Anderson; Michael Schuster
- Web page: https://patentgrep.com/patent/US20150178265A1

## CPC

- G06Q 30/0255: {based on user history}
- G06F 16/3322: {using system suggestions (G06F16/3325 takes precedence)}
- G06F 16/951: Indexing; Web crawling techniques
- G06F 16/954: Navigation, e.g. using categorised browsing
- G06F 40/274: Converting codes to words; Guess-ahead of partial word inputs
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06N 3/02: Neural networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06Q 30/0631: {Recommending goods or services}
- H04N 21/44226: {on social networks}
- H04N 21/4666: {using neural networks, e.g. processing the feedback provided by the user}
- H04N 21/4668: {for recommending content, e.g. movies}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present disclosure relates to applying techniques similar to those used in neural network language modeling systems to a content recommendation system. For example, by associating consumed media content to words of a language model, the system may provide content predictions based on an ordering. Thus, the systems and techniques described herein may produce enhanced prediction results for recommending content (e.g. word) in a given sequence of consumed content. In addition, the system may account for additional user actions by representing particular actions as punctuation in the language model.

## Claims

### Claim 1

A computer-implemented method of providing recommendations, comprising:
obtaining a user history for a user;
associating a plurality of items in the obtained user history with a sequence of tokens;
applying the sequence of tokens as inputs to a neural network language model, wherein at least one token of the sequence of tokens corresponds to a word in the language model;
receiving a predicted word as an output from the language model in response to the applied sequence of tokens, the predicted word corresponding to a predicted item; and
providing a recommendation to the user based on the predicted item.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the neural network is a recurrent neural network.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein one or more of the plurality of items in the user history is an accessed media item.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the accessed media item is a viewed video.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein at least one of the sequence of tokens corresponds to a punctuation in the language model.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein one or more of the plurality of items in the user history is a performed action.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein the performed action is one of selecting an advertisement, performing a search, visiting a webpage, navigating a webpage, rating a media item, sharing a media item, and interrupting a played media item.

### Claim 8 (depends on 6)

The computer-implemented method of claim 6, wherein the performed action is ending a session.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the ending the session item is associated with a token corresponding to the punctuation in the language model.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein the punctuation represents an end of sentence indicator in the language model.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein the receiving the predicted word includes receiving a plurality of predicted words and each of the predicted words is associated with a probability.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, wherein the providing the recommendation to the user includes listing a plurality of predicted items corresponding to the plurality of predicted words, the listing ordered based on the probability associated with each of the predicted words.

### Claim 13

A system for providing recommendations, comprising:
a processor, the processor configured to:
obtain a user history for a user;
associate a plurality of items in the obtained user history with a sequence of tokens;
apply the sequence of tokens as inputs to a neural network language model, wherein at least one token of the sequence of tokens corresponds to a word in the language model;
receive a predicted word as an output from the language model in response to the applied sequence of tokens, the predicted word corresponding to a predicted item; and
provide a recommendation to the user based on the predicted item.

### Claim 14 (depends on 13)

The system of claim 13, wherein one or more of the plurality of items in the user history is an accessed media item.

### Claim 15 (depends on 13)

The system of claim 13, wherein the accessed media item is a viewed video.

### Claim 16 (depends on 13)

The system of claim 13, wherein at least one of the sequence of tokens corresponds to a punctuation in the language model.

### Claim 17 (depends on 16)

The system of claim 16, wherein one or more of the plurality of items in the user history is a performed action.

### Claim 18 (depends on 17)

The system of claim 17, wherein the performed action is one of selecting an advertisement, performing a search, visiting a webpage, navigating a webpage, rating a media item, sharing a media item, and interrupting a played media item.

### Claim 19 (depends on 17)

The system of claim 17, wherein the performed action is ending a session.

### Claim 20 (depends on 19)

The system of claim 19, wherein the ending the session item is associated with a token corresponding to the punctuation in the language model.
