# US 2023/0267757 A1 · Program generation assisting system for assisting generation of program for analyzing clinical trial

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267757 A1
- Document ID: US20230267757A1
- Status: Published application
- Kind: A1
- Application number: 18/012,539
- Filed: 2021-06-22
- Published: 2023-08-24
- Earliest claimed: 2020-06-23
- Assignees: Shionogi & Co., Ltd.
- Applicants: Shionogi & Co., Ltd.
- Inventors: Ryo KIGUCHI; Masakazu FUJIWARA; Yoshitake KITANISHI
- Web page: https://patentgrep.com/patent/US20230267757A1

## CPC

- G06V 30/413: Classification of content, e.g. text, photographs or tables
- G06V 10/82: using neural networks
- G06V 30/19093: {Proximity measures, i.e. similarity or distance measures}
- G06V 30/19173: {Classification techniques}
- G16H 10/20: for electronic clinical trials or questionnaires
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

The purpose of the present disclosure is to assist automation of programming for automatic generation of an analysis material prepared by analysis work in a clinical trial during pharmaceutical development. Using the format information, the layout information, and the image data of an image/table analysis plan, this program generation assisting system performs form pattern prediction by two approaches: micro rule-based prediction and macro CNN-based prediction. In addition, the program generation assisting system performs model data/variable prediction by using the format information and the layout information of the image/table analysis plan, an ADaM specification, and ADaM/variable association information, and presents a combination of the model data/variable prediction and the pattern prediction.

## Claims

### Claim 1

A program generation assisting system for assisting generation of a program for analyzing a clinical trial, the system comprising:
an interface device configured to acquire text data and image data created from an image/table analysis plan that specifies a method of analyzing the clinical trial and an output format of an analysis result;
a storage device configured to store the text data and the image data; and
a processing circuit configured to execute each of a first candidate prediction method using the text data and a second candidate prediction method using the image data to classify the image/table analysis plan into at least one pattern among a plurality of predetermined patterns, and then to output a result as a classification candidate,
wherein the plurality of patterns are classified in advance according to the clinical trial analyzing method and the analysis result output format,
the program generation assisting system is provided in advance with a determination rule that defines a relationship of how each of a plurality of text strings corresponds to a pattern in which each text string is classified, and is provided with a trained artificial neural network constructed by training with, as labeled training data, image data of each image/table analysis plan from a plurality of past clinical trials and ground truth patterns obtained by classifying the image data of each image/table analysis plan, and
the processing circuit uses text strings included in the text data of the image/table analysis plan and the determination rule to output data indicating a first pattern among the plurality of patterns by executing the first candidate prediction method, inputs the image data of the image/table analysis plan into the artificial neural network to acquire data indicating a second pattern output from the artificial neural network by executing the second candidate prediction method, and outputs the first pattern and the second pattern as the classification candidates of an analysis material.

### Claim 2 (depends on 1)

The program generation assisting system according to claim 1, wherein the determination rule defines how the pattern to be classified for each text string corresponds to each of two or more text strings among respective text strings that indicate a reason for withdrawal from the clinical trial, an analysis target population, patient background, medication, an adverse event, and/or efficacy.

### Claim 3 (depends on 1)

The program generation assisting system according to claim 1, wherein the artificial neural network is a convolutional neural network.

### Claim 4 (depends on 3)

The program generation assisting system according to claim 3, wherein the convolutional neural network calculates a probability where the image data of the image/table analysis plan corresponds to each of the plurality of patterns.

### Claim 5 (depends on 4)

The program generation assisting system according to claim 4, wherein the convolutional neural network outputs data indicating a pattern having a maximum probability among probabilities corresponding to the respective patterns.

### Claim 6

A program generation assisting system for assisting generation of a program for analyzing a clinical trial, the system comprising:
a storage device configured to store a database;
an interface device configured to acquire data about an image/table analysis plan that specifies a method of analyzing the clinical trial and an output format of an analysis result; and
a processing circuit configure to output, as a candidate, a name of at least one model data to be used for analyzing the clinical trial with reference to the database and the data about the image/table analysis plan,
wherein the image/table analysis plan includes one or more forms, and each form specifies analysis content for each type of analysis of the clinical trial,
the database stores each form of analysis materials in past clinical trials and a name of model data used in each form, namely a name of model data that is a collection of a data set and metadata for the analysis, and
the processing circuit uses a given similarity evaluation function to calculate, for each form of the image/table analysis plan obtained, a similarity between description in the form and description in each form of the analysis materials in the past clinical trials, extracts, based on the calculated similarity, a name of at least one piece of the model data used in the form of the analysis materials in the past clinical trials, the name corresponding to the former description in the form, and then outputs a name of the extracted model data as the candidate.

### Claim 7 (depends on 6)

The program generation assisting system according to claim 6, wherein
the database further stores link information in which the model data used in the form of the analysis materials of the past clinical trials and a variable that is an item describing the model data and used in the form are linked to each other; and
the processing circuit compares a first item of text data in each form of the image/table analysis plan with a second item of the variable of the link information, generates, in a case of match, an associated form in which the first item of the text data in each form of the image/table analysis plan and the second item of the variable of the link information are used for association, adopts, when model data having the same name of the extracted model data in the associated form, as a variable of the extracted model data, a variable that is an item describing the model data having the same name in the associated form, and then outputs, in addition to the name of the extracted model data, the adopted variable as the candidate.

### Claim 8 (depends on 6)

The program generation assisting system according to claim 6, wherein
the interface device further acquires data about a model specification describing how model data prepared in the clinical trial corresponds to a definition of a variable describing the model data, and
the processing circuit calculates a degree of match between the first item of text data in each form of the image/table analysis plan and a third item indicating a variable described in the model specification by using a given algorithm, selects, from the model specification, model data associated with a variable with a larger degree of match calculated than a predetermined threshold, adopts, when the selected model data matches the extracted model data, a variable larger than the threshold as a variable of the extracted model data, and then outputs, in addition to a name of the extracted model data, the adopted variable as the candidate.

### Claim 9 (depends on 8)

The program generation assisting system according to claim 8, wherein
the first item of each form of the image/table analysis plan includes cell information of the variable, and
the third item includes label information of the variable.

### Claim 10 (depends on 8)

The program generation assisting system according to claim 8, wherein
the first item of each form of the image/table analysis plan includes cell information of the variable, and
the third item includes format information indicating the variable.

### Claim 11 (depends on 8)

The program generation assisting system according to claim 8, wherein
the processing circuit extracts a character string group of the first item and a character string group of the third item by an N-gram, and
calculates the degree of match based on the number of character strings common to the character string group of the first item and the character string group of the third item.

### Claim 12 (depends on 11)

The program generation assisting system according to claim 11, wherein the processing circuit calculates the degree of match by using a percentage of the number of common character strings with respect to the number of all combinations of the character string group of the first item and the character string group of the third item.

### Claim 13 (depends on 6)

The program generation assisting system according to claim 6, wherein the processing circuit uses a given similarity evaluation function to calculate, for each form of the image/table analysis plan obtained, a similarity between description in a title of the form and description in a title of each form of the analysis materials in the past clinical trials.

### Claim 14 (depends on 6)

The program generation assisting system according to claim 6, wherein the processing circuit
generates a first vector by vectorizing, for each form in the image/table analysis plan obtained, description of the form,
generates a second vector group by vectorizing each of a plurality of forms included in the image/table analysis plans of the past clinical trials, and
uses, as the similarity evaluation function, a function that outputs a distance between each member of the first vector and each member of the second vector group to calculate the similarity.

### Claim 15 (depends on 14)

The program generation assisting system according to claim 14, wherein the similarity is a Tanimoto coefficient or a Jaccard coefficient.

### Claim 16 (depends on 1)

An analysis program generation system for generating a program for analyzing a clinical trial, the system comprising:
an interface device;
a storage device; and
a processing circuit,
wherein the interface device is configured to:
acquire the first pattern and the second pattern output from the program generation assisting system according to claim 1,
acquires the name and the variable of the model data output from a program generation assisting system including:
a database configured to store link information in which a model data used in the form of analysis materials of past clinical trials and a variable that is an item describing the model data and used in the form are linked to each other; and
a processing circuit configured to compare a first item of text data in each form of an image/table analysis plan with a second item of the variable of link information, generate, in a case of match, an associated form in which the first item of the text data in each form of the image/table analysis plan and the second item of the variable of the link information are used for association, and adopt, when model data having the same name of the extracted model data in the associated form, as a variable of the extracted model data, a variable that is an item describing the model data having the same name in the associated form, and then output in addition to the name of the extracted model data, the adopted variable as the candidate, and
acquire correction data for correcting the first pattern, the second pattern, the name, and/or the variable,
the storage device stores a template program corresponding to each of the plurality of patterns and capable of specifying a method of analyzing the clinical trial and an output format of an analysis result, and
the processing circuit uses, when the correction data is acquired, the corrected first pattern, second pattern, name and/or variable and the text data or when the correction data is not acquired, the acquired first pattern, second pattern, name and/or variable and the text data to create metadata including pattern-specifying data that specifies one of the plurality of patterns, reads a template program corresponding to each pattern specified by the pattern-specifying data, and then generates the analysis program from the template program by using the metadata.

### Claim 17 (depends on 16)

The analysis program generation system according to claim 16, wherein
the text data includes an instruction of the output format of the analysis result, and
the processing circuit uses the text date to instruct the output format of the analysis result to the template program and then generates the analysis program capable of outputting the analysis result in a layout according to the instructed output format.

### Claim 18 (depends on 1)

A computer-readable storage medium comprising a stored computer program for causing a processing circuit to execute processing in the program generation assisting system according to claim 1.

### Claim 19 (depends on 16)

A computer-readable storage medium comprising a stored computer program for causing a processing circuit to execute processing in the analysis program generation system according to claim 16.
