# US 2014/0156547 A1 · Methods and systems for assessing computer applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0156547 A1
- Document ID: US20140156547A1
- Status: Published application
- Kind: A1
- Application number: 13/690,962
- Filed: 2012-11-30
- Published: 2014-06-05
- Earliest claimed: 2012-11-30
- Assignees: The Children's Hospital of Philadelphia
- Applicants: The Children's Hospital of Philadelphia
- Inventors: FLAURA KOPLIN WINSTON; Nancy Kassam-Adams; José A. B. Fortes
- Web page: https://patentgrep.com/patent/US20140156547A1

## CPC

- G06Q 30/018: Certifying business or products
- G06Q 10/10: Office automation; Time management
- G06Q 50/10: Services
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Methods for assessing the effectiveness of computer applications are disclosed. Methods and systems for creating studies to assess computer applications are also disclosed. An assessable computer application may be created by storing application activities and assessment activities in a database, defining study arms for assessing a computer application, defining each study arm with a computer by receiving selections of application activities and assessment activities from the database, and storing the defined study arms as a study for assessing the effectiveness of the computer application. A computer application for achieving a goal may be assessed by defining application activities corresponding to the goal, defining at least one assessment activity corresponding to the goal, presenting the defined application activities and at least one assessment activity to study participants, collecting application interaction and assessment data, and analyzing the collected data to determine the effectiveness of the application at achieving the goal.

## Claims

### Claim 1

A method for creating studies to assess the effectiveness of computer applications, the method comprising the steps of:
storing a plurality of application activities and assessment activities in a database;
defining, by a processor, a number of study arms of a study for assessing a computer application;
defining, by the processor, each of the number of study arms by receiving selections of at least two of the plurality of application activities and at least one of the plurality of assessment activities from the database for each study arm;
enabling selection, by the processor, of the order and conditions under which the at least one of the plurality of assessment activities and the at least two of the plurality of application activities will be delivered in each study arm independent from an assessment of a participant in the study; and
storing the defined study arms as the study for assessing the effectiveness of the computer application.

### Claim 2 (depends on 1)

The method of claim 1, further comprising the steps of:
setting rules for assignment of study participants or groups of study participants to one of the number of study arms;
presenting the study to study participants for electronic interaction.

### Claim 3 (depends on 1)

The method of claim 1, wherein the storing of the plurality of application activities and assessment activities in the database comprises:
parsing the computer application to identify candidate application activities; and
storing at least one of the candidate application activities as at least two of the plurality of application activities.

### Claim 4

(canceled)

### Claim 5 (depends on 1)

The method of claim 1, wherein a researcher creates the study and study participants interact with the application and assessment activities of the study arms and wherein the method further comprises:
defining permission levels for each of the researcher and study participants.

### Claim 6

A system for creating studies to assess the effectiveness of computer applications, the system comprising:
an assessment module configured to store a plurality of assessment activities;
an application module configured to store a plurality of application activities; and
a study module coupled to the assessment module and the application module, the study module configured to create a study including a number of study arms to assess the effectiveness of a computer application by enabling selection, by a user, of at least one of the stored plurality of assessment activities, at least two of the stored plurality of application activities and the order and conditions under which the at least one of the plurality of assessment activities and at least two of the plurality of application activities will be delivered within each study arm.

### Claim 7

(canceled)

### Claim 8 (depends on 6)

The system of claim 6, wherein the study module further enables setting rules for assignment of study participants to each of the number of study arms.

### Claim 9 (depends on 8)

The system of claim 8, the system further comprising:
a presentation and collection module coupled to the study module, the presentation and collection module configured to present the study arms to the study participants based on the set rules for assignment and to collect data based on electronic interaction of the study participants and the number of study arms.

### Claim 10 (depends on 9)

The system of claim 9, the system further comprising:
an analysis module coupled to the presentation and collection module, the analysis module configured to analyze the collected data to determine the effectiveness of the computer application.

### Claim 11 (depends on 10)

The system of claim 10, wherein the analysis module is coupled to an external database to receive external data for use in determining the effectiveness of the computer application.

### Claim 12 (depends on 6)

The system of claim 6, wherein the study module further enables defining an order and conditions under which the selected at least two of the plurality of application activities and the selected at least one of the plurality of assessment activities will be presented within each study arm.

### Claim 13

A method for assessing the effectiveness of computer applications, the method comprising the steps of:
defining application activities corresponding to a computer application for achieving a goal;
defining at least one assessment activity corresponding to the goal;
presenting the defined application activities and at least one assessment activity to a first group of study participants for electronic interaction;
collecting application interaction data from at least one of the defined application activities;
collecting assessment data from the electronic interaction of the first group of study participants with the defined at least one assessment activity at a computer; and
analyzing the collected application interaction data and assessment data with the computer to determine the effectiveness of the computer application at achieving the goal.

### Claim 14 (depends on 13)

The method of claim 13, further comprising the step of:
collecting external data related to the goal; and
wherein the analyzing step further comprises analyzing the collected external data to determine the effectiveness of the computer application at achieving the goal.

### Claim 15 (depends on 13)

The method of claim 13, wherein the defining of the at least one assessment activity step comprises:
receiving user assessment selection of at least one predefined assessment activity; and
assembling the at least one assessment activity from the selected at least one predefined assessment activity.

### Claim 16 (depends on 13)

The method of claim 13, wherein the defining of the application activities comprises:
receiving user application selection of at least one predefined application activity; and
assembling the at least one assessment activity from the selected at least one predefined assessment activity.

### Claim 17 (depends on 16)

The method of claim 16, further comprising:
parsing an existing computer application to identify candidate application activities; and
storing the candidate application activities as the at least one predefined application activity.

### Claim 18 (depends on 16)

The method of claim 16, further comprising:
receiving user input information defining at least one new application activity; and
storing the at least one new application as the at least one predefined application activity.

### Claim 19 (depends on 13)

The method of claim 13, wherein the presented defined application activities and at least one assessment activity form one arm of a study and wherein the method further comprises:
forming at least one other arm of the study, the at least other arm including a variation of the defined application activities; and
presenting the at least one other arm to a second group of study participants for electronic interaction; and
collecting assessment data from the electronic interaction of the second group of study participants with the at least one other study arm;
wherein the analyzing step analyzes the collected assessment data from the first and second groups of study participants to determine the effectiveness of the computer application at achieving the goal.

### Claim 20 (depends on 13)

The method of claim 13, wherein the presenting step comprises:
distributing at least one of the application activities to at least one participant of the first group of study participants over a network.

### Claim 21 (depends on 13)

The method of claim 13, further comprising the step of:
defining the order and conditions under which the application activities and the at least one assessment activity will be presented within each study arm;
wherein the presenting step further comprises presenting the application activities and the at least one assessment activities in accordance with the defined order and conditions.

### Claim 22

A method for assessing the effectiveness of computer applications, the method comprising the steps of:
defining a first study arm including at least one assessment activity and application activities corresponding to a computer application for achieving a goal and at least one assessment activity corresponding to the goal;
defining a second study arm including the at least one assessment activity and a variation of the application activities of the first study arm;
presenting the first study arm to a first group of study participants and the second study arm to a second group of study participants for electronic interaction;
collecting data with a computer from the electronic interaction of the first and second groups of study participants with the defined at least one assessment activity; and
analyzing the collected data using the computer to determine the effectiveness of the computer application at achieving the goal.

### Claim 23 (depends on 22)

The method of claim 22, further comprising:
identifying number of study arms for assessing the computer application;
defining a unique study arm for each of the identified number of study arms.

### Claim 24 (depends on 22)

The method of claim 22, wherein the defining the at least one study arm comprises:
defining application activities corresponding to the computer application for achieving a goal;
defining at least one assessment activity corresponding to the goal.

### Claim 25 (depends on 24)

The method of claim 24, wherein the defining the at least one assessment activity comprises:
receiving user assessment selection of at least one predefined assessment activity; and
assembling the at least one assessment activity from the selected at least one predefined assessment activity.

### Claim 26 (depends on 25)

The method of claim 25, wherein the defining the application activities comprises:
receiving user application selection of at least one predefined application activity; and
assembling the at least one assessment activity from the selected at least one predefined assessment activity.

### Claim 27 (depends on 26)

The method of claim 26, further comprising:
parsing an existing computer application to identify candidate application activities; and
storing the candidate application activities as the at least one predefined application activity.

### Claim 28 (depends on 26)

The method of claim 26, further comprising:
receiving user input information defining at least one new application activity; and
storing the at least one new application as the at least one predefined application activity.

### Claim 29 (depends on 22)

The method of claim 22, wherein:
the step of defining the first study arm further includes defining the order and conditions for presenting the at least one assessment activity and the application activities; and
the step of defining the second study arm further includes defining the order and conditions for presenting the at least one assessment activity and the variation of the application activities;
wherein the presenting step further comprises presenting the application activities and the at least one assessment activity in accordance with the defined order and conditions for each study arm.

### Claim 30 (depends on 22)

The method of claim 22, further comprising the step of:
de-identifying study participants prior to analysis to maintain confidentiality.

### Claim 31 (depends on 1)

The method of claim 1, further comprising the step of:
enabling selection of a first study arm and a second study arm having the same at least two application activities, wherein the order and conditions of the application activities of the first study arm are different from the second study arm.

### Claim 32 (depends on 31)

The method of claim 31, wherein the first study arm and the second study arm have the same at least one assessment activity.

### Claim 33 (depends on 1)

The method of claim 1, further comprising the steps of:
enabling selection of at least two of the defined study arms; and
assessing the effectiveness of the computer application by comparatively analyzing data collected from the at least two of the defined study arms.

### Claim 34 (depends on 33)

The method of claim 33, wherein the data comprises assessment activity data linked to application activity data of a participant in the study.

### Claim 35 (depends on 34)

The method of claim 34, wherein the application activity data further comprises application activity interaction data and application activity tracking data.

### Claim 36 (depends on 6)

The system of claim 6, wherein the study module is further configured to enable selection, by a user, of a first study arm and a second study arm having the same at least one assessment activity.

### Claim 37 (depends on 6)

The system of claim 6, wherein the study module is further configured to enable selection, by a user, of a first study arm and a second study arm having the same at least two application activities and wherein the order and conditions of the application activities of the first study arm is different from the second study arm.

### Claim 38 (depends on 37)

The system of claim 37, further comprising:
a collection module configured to collect data from the first study arm and the second study arm; and
an analysis module configured to comparatively analyze the collected data to assess the effectiveness of the computer application.

### Claim 39 (depends on 6)

The system of claim 6, wherein the study module enables selection and enables assignment of a study arm to a participant of the study independent from an assessment of the participant of the study.

### Claim 40 (depends on 38)

The system of claim 38, wherein the data comprises assessment activity data linked to application activity data for a participant in the study.