Task specific processing of regulatory content

Abstract

A neural network system for performing a processing task on regulatory content and a method for training the system are disclosed. The method for training involves configuring a neural network language model capable of generating a language embedding output in response to receiving content. The method further involves fine-tuning the language model using regulatory content training data to generate a regulatory content language embedding output for regulatory content processed by the language model. The method also involves configuring at least one task specific output layer to generate task specific results in response to receiving the regulatory content language embedding output from the language model, and training the neural network system using task specific training data to output the task specific results, at least a portion of the task specific training data having been labeled prior to configuring the task specific neural network.

Dependencies are inferred from the claim text.