# US 2021/0241865 A1 · Trial design benchmarking platform

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241865 A1
- Document ID: US20210241865A1
- Status: Published application
- Kind: A1
- Application number: 17/163,437
- Filed: 2021-01-30
- Published: 2021-08-05
- Earliest claimed: 2020-01-31
- Applicants: Cytel Inc.
- Inventors: Jaydeep Bhattacharya; James Bolognese; Alexandre Buer; Eric Edwards; Stanley Y. Huang; Yannis Jemiai; Cyrus Mehta; Nitin Patel; Anne Pelz; Ajay Prabhakar Sathe; Joshua A. Schultz; Pralay Senchaudhuri
- Web page: https://patentgrep.com/patent/US20210241865A1

## CPC

- G16H 10/20: for electronic clinical trials or questionnaires
- G06F 30/10: Geometric CAD
- G06F 30/12: characterised by design entry means specially adapted for CAD, e.g. graphical user interfaces [GUI] specially adapted for CAD
- G06F 30/20: Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30)
- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/04: Inference or reasoning models
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G06Q 10/06315: {Needs-based resource requirements planning or analysis}
- G06Q 10/067: Enterprise or organisation modelling
- G06Q 10/0674: {using process simulation}
- G06Q 10/101: Collaborative creation, e.g. joint development of products or services
- G06Q 30/0205: {based on location or geographical consideration}
- G16H 40/20: for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
- G16H 40/67: for remote operation
- G16H 50/50: for simulation or modelling of medical disorders
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 2111/02: CAD in a network environment, e.g. collaborative CAD or distributed simulation
- G06F 2111/04: Constraint-based CAD
- G06F 2111/06: Multi-objective optimisation, e.g. Pareto optimisation using simulated annealing [SA], ant colony algorithms or genetic algorithms [GA]
- G06F 2111/08: Probabilistic or stochastic CAD
- G06F 2111/16: Customisation or personalisation
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A method, according to some implementations, includes providing inputs to a plurality of trial design simulation engines, receiving first outputs of the plurality of trial design simulation engines in response to the inputs, and providing variations of the inputs to the plurality of trial design simulation engines. The method may further include receiving second outputs of the plurality of trial design simulation engines in response to the variations evaluating the first and the second outputs to determine delta values, and determining, based in part on the delta values, a plurality of normalization factors for the plurality of trial design simulation engines.

## Claims

### Claim 1

A method comprising:
providing inputs to a plurality of trial design simulation engines;
receiving first outputs of the plurality of trial design simulation engines in response to the inputs;
providing variations of the inputs to the plurality of trial design simulation engines;
receiving second outputs of the plurality of trial design simulation engines in response to the variations;
evaluating the first and the second outputs to determine delta values; and
determining, based in part on the delta values, a plurality of normalization factors for the plurality of trial design simulation engines.

### Claim 2 (depends on 1)

The method of claim 1, wherein a first trial design simulation engine of the plurality is structured to simulate a first trial design that is of a different type than a second trial design which a second trial design simulation engine of the plurality is structured to simulate.

### Claim 3 (depends on 2)

The method of claim 2, wherein a first trial design simulation engine of the plurality is a different version of a second trial design simulation engine of the plurality.

### Claim 4 (depends on 2)

The method of claim 2, wherein a first trial design simulation engine of the plurality was generated by a first entity and a second trial design simulation engine of the plurality was generated by a second entity of the plurality distinct from the first entity.

### Claim 5 (depends on 2)

The method of claim 2, wherein the first and the second outputs include metadata.

### Claim 6 (depends on 2)

The method of claim 2, further comprising:
multiplying at least one of the first or the second outputs by at least one of the plurality of normalization factors.

### Claim 7 (depends on 2)

The method of claim 2, wherein each of the plurality of normalization factors differ in value based at least in part on differing performance criteria defined, in part, by the inputs.

### Claim 8 (depends on 2)

The method of claim 2, further comprising:
determining, for each of the trial design simulation engines, an output variability; and
displaying the variabilities to a user.

### Claim 9 (depends on 2)

The method of claim 2, further comprising:
determining a valid range of output values based on the delta values; and
determining that at least one of the plurality of trial design simulation engines outputs invalid values.

### Claim 10 (depends on 2)

The method of claim 2, further comprising:
evaluating historical trial design selections to identify one or more trial design parameters based at least in part on one or more trial design criteria determined from a user via an interactive interface, wherein receiving the first outputs is based at least in part on a quick search data structure and the one or more trial design parameters;
generating a substitute for at least some of the first outputs based at least in part on a relationship between the first outputs and supplemental data;
generating a performance surface based at least in part on a set of trial designs corresponding to first outputs;
evaluating one or more trial designs based at least in part on the performance surface; and
calculating a score based on normalized score component values corresponding to the first outputs.

### Claim 11

A method comprising:
providing inputs to a plurality of trial design simulation engines;
receiving outputs of the plurality of trial design simulation engines in response to the inputs;
comparing outputs to expected outputs; and
determining, based on the comparing, a plurality of normalization factors for the plurality of trial design simulation engines.

### Claim 12 (depends on 11)

The method of claim 11, wherein a first trial design simulation engine of the plurality is structured to simulate a first trial design that is of a different type than a second trial design which a second trial design simulation engine of the plurality is structured to simulate.

### Claim 13 (depends on 11)

The method of claim 11, wherein a first trial design simulation engine of the plurality is a different version of a second trial design simulation engine of the plurality.

### Claim 14 (depends on 11)

The method of claim 11, wherein a first trial design simulation engine of the plurality was generated by a first entity and a second trial design simulation engine of the plurality was generated by a second entity of the plurality distinct from the first entity.

### Claim 15 (depends on 11)

The method of claim 11, wherein each of the plurality of normalization factors differ in value based at least in part on differing performance criteria defined, in part, by the inputs.

### Claim 16

An apparatus comprising:
an output processing circuit structured to interpret output data from a plurality of trial design simulation engines;
a comparison circuit structured to compare the interpreted output data to expected output data;
a normalization circuit structured to determine a plurality of normalization factors for the plurality of trial design simulation engines; and
a normalization provisioning circuit structured to transmit the plurality of normalization factors.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein a first trial design simulation engine of the plurality is structured to simulate a first trial design that is of a different type than a second trial design which a second trial design simulation engine of the plurality is structured to simulate.

### Claim 18 (depends on 16)

The apparatus of claim 16, wherein a first trial design simulation engine of the plurality is a different version of a second trial design simulation engine of the plurality.

### Claim 19 (depends on 16)

The apparatus of claim 16, wherein a first trial design simulation engine of the plurality was generated by a first entity and a second trial design simulation engine of the plurality was generated by a second entity of the plurality distinct from the first entity.

### Claim 20 (depends on 16)

The apparatus of claim 16, wherein each of the plurality of normalization factors differ in value based at least in part on differing performance criteria.