# US 2016/0358486 A1 · Methods and systems for providing evaluation resources for users of an electronic learning system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0358486 A1
- Document ID: US20160358486A1
- Status: Published application
- Kind: A1
- Application number: 14/729,303
- Filed: 2015-06-03
- Published: 2016-12-08
- Earliest claimed: 2015-06-03
- Assignees: D2L Corporation
- Applicants: D2L Corporation
- Inventors: Jugoslav Bilic; Stephen John Michaud; Martin David Goodenough Bayly; Ryan Clayton Ogg
- Web page: https://patentgrep.com/patent/US20160358486A1

## CPC

- G09B 5/00: Electrically-operated educational appliances (working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F)
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

Methods and systems for providing an evaluation resource for a user of an electronic learning system. The methods can include identifying a user profile associated with the user, each user profile including user interaction data representing an interaction by a respective user with one or more learning paths provided by the electronic learning system; detecting, from the user interaction data in the identified user profile, an evaluation request from the user in respect of a learning path, the evaluation request indicating the evaluation resource is to be provided and the evaluation request is associated with at least one resource in the learning path; and in response to detecting the evaluation request, generating the evaluation resource, otherwise, continuing to monitor the user interaction data, wherein generating the evaluation resource comprises selecting one or more evaluation items based on, at least, resource data associated with the at least one resource.

## Claims

### Claim 1

A method for providing an evaluation resource for a user of an electronic learning system, the electronic learning system including a processor and a memory in electronic communication with the processor, the memory storing, at least, one or more user profiles for one or more respective users, the method comprising:
identifying, from the one or more user profiles, a user profile associated with the user, each user profile comprising user interaction data representing an interaction by a respective user with one or more learning paths provided by the electronic learning system;
detecting, from the user interaction data in the identified user profile, an evaluation request from the user in respect of a learning path of the one or more learning paths, the evaluation request indicating the evaluation resource is to be provided and the evaluation request being associated with at least one resource in the learning path; and
in response to detecting the evaluation request, generating the evaluation resource, otherwise, continuing to monitor the user interaction data, wherein generating the evaluation resource comprises selecting one or more evaluation items based on, at least, resource data associated with the at least one resource.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the memory stores, at least, a plurality of evaluation items for the electronic learning system; and
generating the evaluation resource further comprises:
conducting a semantic analysis of the resource data with each evaluation item of the plurality of evaluation items, the semantic analysis comprising a comparison of a content of the resource data with a content of each evaluation item;
generating a relevance score for each evaluation item based on the semantic analysis; and
selecting the one or more evaluation items from the plurality of evaluation items based on, at least, the respective relevance score.

### Claim 3 (depends on 2)

The method of claim 2, wherein:
the resource data comprises: (i) a resource content of the at least one resource and (ii) one or more resource learning objectives of a set of learning objectives associated with the learning path, the at least one resource being associated with the learning path for fulfilling the one or more resource learning objectives;
conducting the semantic analysis of the resource data with each evaluation item comprises:
conducting the semantic analysis of the resource content with the content of each evaluation item; and
conducting the semantic analysis of a content of each resource learning objective with the content of each evaluation item; and
generating the relevance score for each evaluation item based on the semantic analysis comprises:
generating a resource relevance score for each evaluation item based on the semantic analysis of the resource content with the content of each evaluation item;
generating a learning objective relevance score for each evaluation item based on the semantic analysis of the content of each resource learning objective with the content of each evaluation item; and
combining the resource relevance score and the learning objective relevance score to generate the relevance score.

### Claim 4 (depends on 3)

The method of claim 3, wherein combining the resource relevance score and the learning objective relevance score to generate the relevance score comprises:
applying a resource weight to the resource relevance score;
applying a learning objective weight to the learning objective relevance score; and
summing the weighted resource relevance score and the weighted learning objective relevance score;
wherein a sum of the resource weight and the learning objective weight is one.

### Claim 5 (depends on 4)

The method of claim 4, wherein each of the resource weight and the learning objective weight is variable by one of (i) the one or more respective users and (ii) an operator of the electronic learning system.

### Claim 6 (depends on 3)

The method of claim 3, wherein selecting the one or more evaluation items from the plurality of evaluation items comprises:
selecting a set of evaluation items from the plurality of evaluation items based on the respective relevance score;
determining whether each evaluation item of the set of evaluation items is associated with a respective system learn value, wherein the system learn value for each evaluation item represents a likelihood of that evaluation item in determining a level of understanding the user has in respect of the one or more learning objectives; and
in response to determining each feedback item is associated with the system learn value, selecting the one or more evaluation items from the set of evaluation items based on the respective system learn values, otherwise, selecting the one or more evaluation items from the plurality of evaluation items based on the respective relevance score.

### Claim 7 (depends on 6)

The method of claim 6, wherein selecting the one or more evaluation items from the set of evaluation items based on the system learn values comprises:
determining a median value for the system learn values associated with the set of evaluation items; and
selecting the one or more evaluation items from the set of evaluation items based on the median value, wherein selecting the one or more evaluation items from the set of evaluation items based on the median value comprises selecting an evaluation item associated with a first system learn value prior to selecting another evaluation item associated with a second system learn value, the first system learn value being at least closer in value to the median value than the second system learn value.

### Claim 8 (depends on 2)

The method of claim 2, wherein selecting the one or more evaluation items from the plurality of evaluation items based on, at least, the respective relevance score comprises, for each evaluation item,
determining whether the respective relevance score exceeds a relevance threshold value assigned to the evaluation resource, the relevance threshold value being a minimum relevance score required for that evaluation item to be included in the evaluation resource; and
in response to determining the respective relevance score exceeds the relevance threshold value assigned to the evaluation resource, including that evaluation item in the one or more evaluation items, otherwise, excluding that evaluation item from the one or more evaluation items.

### Claim 9 (depends on 8)

The method of claim 8, wherein:
the evaluation resource is associated with a desired number of evaluation items; and
generating the evaluation resource further comprises:
determining a number of relevant evaluation items from the plurality of evaluation items, the number of relevant evaluation items corresponding to a number of evaluation items associated with the relevance score exceeding the relevance threshold value;
determining whether the number of relevant evaluation items is less than the desired number of evaluation items; and
in response to determining the number of relevant evaluation items is less than the desired number of evaluation items, varying the relevance threshold value, otherwise, maintaining the relevance threshold value.

### Claim 10

An electronic learning system comprising:
a memory for storing, at least, one or more user profiles for one or more respective users; and
a processor in electronic communication with the memory, the processor operating to:
identify, from the one or more user profiles, a user profile associated with the user, each user profile comprising user interaction data representing an interaction by a respective user with one or more learning paths provided by the electronic learning system;
detect, from the user interaction data in the identified user profile, an evaluation request from the user in respect of a learning path of the one or more learning paths, the evaluation request indicating the evaluation resource is to be provided and the evaluation request being associated with at least one resource in the learning path; and
in response to detecting the evaluation request, generate the evaluation resource, otherwise, continue to monitor the user interaction data, wherein generating the evaluation resource comprises selecting one or more evaluation items based on, at least, resource data associated with the at least one resource.

### Claim 11 (depends on 10)

The electronic learning system of claim 10, wherein:
the memory further stores, at least, a plurality of evaluation items for the electronic learning system; and
the processor further operates to:
conduct a semantic analysis of the resource data with each evaluation item of the plurality of evaluation items, the semantic analysis comprising a comparison of a content of the resource data with a content of each evaluation item;
generate a relevance score for each evaluation item based on the semantic analysis; and
select the one or more evaluation items from the plurality of evaluation items based on, at least, the respective relevance score.

### Claim 12 (depends on 11)

The electronic learning system of claim 11, wherein:
the resource data comprises: (i) a resource content of the at least one resource and (ii) one or more resource learning objectives of a set of learning objectives associated with the learning path, the at least one resource being associated with the learning path for fulfilling the one or more resource learning objectives;
the processor further operates to:
conduct the semantic analysis of the resource content with the content of each evaluation item;
conduct the semantic analysis of a content of each resource learning objective with the content of each evaluation item;
generate a resource relevance score for each evaluation item based on the semantic analysis of the resource content with the content of each evaluation item;
generate a learning objective relevance score for each evaluation item based on the semantic analysis of the content of each resource learning objective with the content of each evaluation item; and
combine the resource relevance score and the learning objective relevance score to generate the relevance score.

### Claim 13 (depends on 12)

The electronic learning system of claim 12, wherein:
the processor further operates to:
apply a resource weight to the resource relevance score;
apply a learning objective weight to the learning objective relevance score; and
sum the weighted resource relevance score and the weighted learning objective relevance score; and
a sum of the resource weight and the learning objective weight is one.

### Claim 14 (depends on 13)

The electronic learning system of claim 13, wherein each of the resource weight and the learning objective weight is variable by one of (i) the one or more respective users and (ii) an operator of the electronic learning system.

### Claim 15 (depends on 12)

The electronic learning system of claim 12, wherein the processor further operates to:
select a set of evaluation items from the plurality of evaluation items based on the respective relevance score;
determine whether each evaluation item of the set of evaluation items is associated with a respective system learn value, wherein the system learn value for each evaluation item represents a likelihood of that evaluation item in determining a level of understanding the user has in respect of the one or more learning objectives; and
in response to determining each feedback item is associated with the system learn value, select the one or more evaluation items from the set of evaluation items based on the respective system learn values, otherwise, select the one or more evaluation items from the plurality of evaluation items based on the respective relevance score.

### Claim 16 (depends on 15)

The electronic learning system of claim 15, wherein the processor further operates to:
determine a median value for the system learn values associated with the set of evaluation items; and
select the one or more evaluation items from the set of evaluation items based on the median value, wherein the processor is operated to select an evaluation item associated with a first system learn value prior to selecting another evaluation item associated with a second system learn value, the first system learn value being at least closer in value to the median value than the second system learn value.

### Claim 17 (depends on 11)

The electronic learning system of claim 11, wherein the processor further operates to:
determine whether the respective relevance score exceeds a relevance threshold value assigned to the evaluation resource, the relevance threshold value being a minimum relevance score required for that evaluation item to be included in the evaluation resource; and
in response to determining the respective relevance score exceeds the relevance threshold value assigned to the evaluation resource, include that evaluation item in the one or more evaluation items, otherwise, exclude that evaluation item from the one or more evaluation items.

### Claim 18 (depends on 17)

The electronic learning system of claim 17, wherein:
the evaluation resource is associated with a desired number of evaluation items; and
the processor further operates to:
determine a number of relevant evaluation items from the plurality of evaluation items, the number of relevant evaluation items corresponding to a number of evaluation items associated with the relevance score exceeding the relevance threshold value;
determine whether the number of relevant evaluation items is less than the desired number of evaluation items; and
in response to determining the number of relevant evaluation items is less than the desired number of evaluation items, vary the relevance threshold value, otherwise, maintain the relevance threshold value.
