# US 2021/0342864 A1 · System and method for evaluating black-box recommendation systems in infotainment systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342864 A1
- Document ID: US20210342864A1
- Status: Published application
- Kind: A1
- Application number: 16/862,633
- Filed: 2020-04-30
- Published: 2021-11-04
- Earliest claimed: 2020-04-30
- Assignees: Robert Bosch GmbH
- Applicants: Robert Bosch GmbH
- Inventors: Hyeongsik KIM; Monireh EBRAHIMI; Lu ZHOU
- Web page: https://patentgrep.com/patent/US20210342864A1

## CPC

- G06Q 30/0203: Market surveys; Market polls
- G06Q 30/0631: {Recommending goods or services}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A method of evaluating a recommendation system, including conducting an online survey including a questionnaire regarding preferences to receive answers from a plurality of participants, utilizing answers from the survey at a recommendation system, outputting a recommendation at the recommendation system for the participants, wherein the recommendation include a feedback option indicating a score for the recommendation, receiving the score associated with the recommendation, and sending the score to a behavioral log.

## Claims

### Claim 1

A method of evaluating a recommendation system of a vehicle multimedia system, comprising:
conducting an online survey including a questionnaire regarding preferences utilizing one or more computers;
receiving answers associated with the survey from a plurality of participants utilizing the one or more computers;
utilizing answers from the survey at the recommendation system of the vehicle multimedia system;
creating a user profile in response to the answers from the survey and utilizing the user profile at the recommendation system, wherein the user profile is associated with a key fob associated with the vehicle or a mobile device associated with the vehicle
outputting a recommendation at the recommendation system for the participants, wherein the recommendation include a feedback option indicating a score for the recommendation, wherein the recommendation system is trained utilizing the user profile;
receiving the score associated with the recommendation at a remote server; and
sending the score to a behavioral log at the remote server.

### Claim 2 (depends on 1)

The method of claim 1, wherein the method includes weighting a similarity of recommended sequence of composite time-stamped events with a user's sequence of activities captured from the behavioral log.

### Claim 3 (depends on 1)

The method of claim 1, wherein the behavioral log includes time-stamped events associated with a user's sequence of activities captured.

### Claim 4 (depends on 1)

The method of claim 1, wherein the method includes outputting recommendations of music.

### Claim 5 (depends on 1)

The method of claim 1, wherein the method includes outputting recommendations of route guidance.

### Claim 6

(canceled)

### Claim 7

(canceled)

### Claim 8 (depends on 1)

The method of claim 1, wherein the method further includes conducting a user study utilizing a testbed application.

### Claim 9

A method of evaluating a recommendation system of a vehicle multimedia system, comprising:
conducting a survey utilizing one or more computers, wherein the survey includes a questionnaire regarding preferences that include answers from a plurality of participants;
creating a user profile in response to the answers from the survey and utilizing the user profile at the recommendation system, wherein the user profile is associated with a key fob or mobile phone of a vehicle;
utilizing answers from the survey at the recommendation system of the vehicle multimedia system;
outputting a recommendation at the recommendation system for the participants, wherein the recommendation include a feedback option indicating a score for the recommendation and the recommendation system is trained utilizing the user profile;
receiving the score associated with the recommendation at a remote server; and
sending the score to a behavioral log at the remote server.

### Claim 10 (depends on 9)

The method of claim 9, wherein the method includes categorizing the plurality of participants in a plurality of groups in response to the answers from the survey.

### Claim 11 (depends on 9)

The method of claim 9, wherein the method includes outputting a second set of recommendations in response to the score.

### Claim 12 (depends on 9)

The method of claim 9, wherein the recommendation system is in a vehicle multimedia system.

### Claim 13 (depends on 9)

The method of claim 9, wherein outputting the recommendation is further in response to the user profile.

### Claim 14

A computer-program product storing instructions on a non-transitory computer-readable medium of a computer which, when executed by the computer, cause the computer to:
conduct a survey utilizing one or more computers, wherein the survey includes a questionnaire regarding preferences that include answers from a plurality of participants;
creating a user profile in response to the answers from the survey and utilizing the user profile at the recommendation system, wherein the user profile is associated with a key fob or mobile phone of a vehicle;
utilize answers from the survey at a recommendation system of a vehicle multimedia system;
output a recommendation at the recommendation system for the participants, wherein the recommendation include a feedback option indicating a score for the recommendation and the recommendation system is trained utilizing the user profile;
receive the score associated with the recommendation; and
send the score to a behavioral log.

### Claim 15 (depends on 14)

The computer-program product of claim 14, wherein the computer is further caused to request login information for the survey.

### Claim 16 (depends on 14)

The computer-program product of claim 14, wherein the computer is further caused to compare the score across recommendations associated with genres of media.

### Claim 17 (depends on 14)

The computer-program product of claim 14, wherein the computer is further caused to compare the score across recommendations associated with navigation route guidance.

### Claim 18 (depends on 14)

The computer-program product of claim 14, wherein the computer is further caused to create a user profile in response to the answers from the survey and utilizing the user profile at the recommendation system.

### Claim 19 (depends on 14)

The computer-program product of claim 14, wherein the survey is an offline survey.

### Claim 20 (depends on 14)

The computer-program product of claim 14, wherein the computer is further caused to output recommendations of route guidance.
