Automated Detection of Safety Signals for Pharmacovigilance

An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations comprises:

• obtaining, from persistent storage, a corpus of documents, wherein each of the documents is labelled with its relevance to pharmacovigilance;
• performing data preparation operations on the documents, wherein the data preparation operations include:
• de-duplicating the documents, normalizing terminology within the documents, and extracting pre-defined features within the documents, wherein the pre-defined features relate to pharmacovigilance;
• fine-tuning a natural language model with the documents and their labels, wherein the natural language model was pre-trained using scientific or medical literature, and wherein the fine-tuning involves further training of one or more encoders within the natural language model so that the natural language model seeks to classify new documents in accordance with their relevance to pharmacovigilance;
• training a feature-based model with the pre-defined features extracted from the documents and the respective labels of the documents so that the feature-based model also seeks to classify the new documents in accordance with their relevance to pharmacovigilance, wherein the feature-based model utilizes a plurality of decision trees with nodes representing the pre-defined features;
• and
• training an aggregation model with classifications produced by the natural language model and the feature-based model so that the aggregation model seeks to produce further classifications of the new documents in accordance with their relevance to pharmacovigilance, wherein the further classifications are weighted combinations of classifications produced by the natural language model and the feature-based model for the new documents.

A computer-implemented method comprising:

• obtaining, from persistent storage, a set of pre-defined features and a new document related to a scientific or medical topic, wherein the pre-defined features relate to pharmacovigilance;
• normalizing terminology within the new document;
• extracting a subset of the pre-defined features from within new document;
• applying a natural language model to the new document, wherein the natural language model was pre-trained using scientific or medical literature and fine-tuned using a corpus of documents, wherein each of the documents was labelled with its relevance to pharmacovigilance, and wherein the fine-tuning involved further training of one or more encoders within the natural language model so that the natural language model seeks to classify the new document in accordance with its relevance to pharmacovigilance;
• applying a feature-based model to the subset of the pre-defined features extracted from the new document, wherein the feature-based model was trained with the pre-defined features and the respective labels of the documents so that the feature-based model also seeks to classify the new document in accordance with its relevance to pharmacovigilance, wherein the feature-based model utilizes a plurality of decision trees with nodes representing the pre-defined features;
• and
• applying an aggregation model to the classifications of the new document produced by the natural language model and the feature-based model, wherein the aggregation model was trained with prior classifications produced by the natural language model and the feature-based model so that the aggregation model seeks to produce a further classification of the new document in accordance with its relevance to pharmacovigilance, wherein the further classification is a weighted combination of classifications produced by the natural language model and the feature-based model for the new document.