# US 10,373,278 B2 · Annotation of legal documents with case citations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,373,278 B2 - Document ID: US10373278B2 - Status: Granted - Kind: B2 - Application number: 15/432,982 - Filed: 2017-02-15 - Published: 2018-08-16 - Granted: 2019-08-06 - Earliest claimed: 2017-02-15 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Hernan Badenes; Rosanna S. Mannan; Siddharth A. Patwardhan - Web page: https://patentgrep.com/patent/US10373278B2 ## CPC - G06Q 50/18: Legal services - G06F 16/93: Document management systems - G06N 20/00: Machine learning - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06Q 10/067: Enterprise or organisation modelling - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A computer-implemented method includes detecting a first and second set of citations to a plurality of legal cases cited in a plurality of legal documents and a first legal document distinct from the plurality of legal documents, respectively. The computer-implemented method further includes determining a first and second set of tones corresponding to each citation in the first and second sets of citations, respectively. The computer-implemented method further includes determining a score for each tone in the first and second sets of tones, respectively. The computer-implemented method further includes detecting a first set of annotations corresponding to the first set of citations. The computer-implemented method further includes building a training model, based, at least in part, from at least a subset of the first set of annotations. The computer-implemented method further includes determining a second set of annotations corresponding to the second set of citations. ## Claims ### Claim 1 A computer-implemented method, comprising: detecting a first set of citations to a plurality of legal cases cited in a plurality of legal documents; determining a first set of tones corresponding to each citation in the first set of citations; determining a first score for each tone in the first set of tones; detecting a first set of annotations corresponding to the first set of citations, wherein the first set of annotations are case treatments associated with citations to legal cases cited in the first plurality of legal documents; aggregating one or more subsets from the first set of annotations; building a training model, wherein: the training model is built based, at least in part, from at least a first subset of the first set of annotations corresponding to the first set of citations; detecting a second set of citations to a plurality of legal cases cited in a first legal document, wherein: the first legal document is not part of the plurality of legal documents; and the second set of citations is devoid of any annotations; determining a second set of tones corresponding to each citation in the second set of citations; determining a second score for each tone in the second set of tones; and determining a second set of annotations corresponding to the second set of citations cited in the first legal document of the second plurality of legal documents, wherein: the second set of annotations are case treatments associated with citations to legal cases cited in the first legal document. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein: the first score of each tone in first set of tones is determined based, at least in part, on a level of intensity of a tone corresponding to a citation; and the second score of each tone in second set of tones is determined based, at least in part, on the level of intensity of the tone corresponding to the citation. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the training model is further built based, at least in part, on mapping each annotation in the first subset of the first set of annotations to one or more tones of the first set of tones. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, wherein the training model is further built based on mapping each annotation in the first subset of the first set of annotations to each first score corresponding to the one or more tones of the first set of tones. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the second set of annotations is determined based, at least in part, on: analyzing the training model; and mapping, according to the training model, at least one tone corresponding to each citation in the second set of citations to a case treatment. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein the second set of annotations is further determined based on: analyzing the training model; and mapping, according to the training model, the second score of at least one tone corresponding to each citation in the second set of citations to a case treatment. ### Claim 7 (depends on 6) The computer implemented method of claim 6, wherein a citation is associated with a case treatment if one or more tones corresponding to the citation have second scores above a given threshold. ### Claim 8 (depends on 1) The computer implemented method of claim 1, further comprising: annotating each citation in the second set of citations in the first legal document with a corresponding annotation from the second set of annotations. ### Claim 9 A computer program product, the computer program product comprising one or more computer readable storage media and program instructions stored on said one or more computer readable storage media, said program instructions comprising instructions to: detect a first set of citations to a plurality of legal cases cited in a plurality of legal documents; determine a first set of tones corresponding to each citation in the first set of citations; determine a first score for each tone in the first set of tones; detect a first set of annotations corresponding to the first set of citations, wherein the first set of annotations are case treatments associated with citations to legal cases cited in the first plurality of legal documents; aggregate one or more subsets from the first set of annotations; build a training model, wherein: the training model is built based, at least in part, from at least a first subset of the first set of annotations corresponding to the first set of citations; detect a second set of citations to a plurality of legal cases cited in a first legal document, wherein: the first legal document is not part of the plurality of legal documents; and the second set of citations is devoid of any annotations; determine a second set of tones corresponding to each citation in the second set of citations; determine a second score for each tone in the second set of tones; and determine a second set of annotations corresponding to the second set of citations cited in the first legal document of the second plurality of legal documents, wherein: the second set of annotations are case treatments associated with citations to legal cases cited in the first legal document. ### Claim 10 (depends on 9) The computer program product of claim 9, wherein: the first score of each tone in first set of tones is determined based, at least in part, on a level of intensity of a tone corresponding to a citation; and the second score of each tone in second set of tones is determined based, at least in part, on the level of intensity of the tone corresponding to the citation. ### Claim 11 (depends on 9) The computer program product of claim 9, wherein the training model is further built based, at least in part, on program instructions to map each annotation in the first subset of the first set of annotations to one or more tones of the first set of tones. ### Claim 12 (depends on 11) The computer program product of claim 11, wherein the training model is further built based on program instructions to map each annotation in the first subset of the first set of annotations to each first score corresponding to the one or more tones of the first set of tones. ### Claim 13 (depends on 9) The computer program product of claim 9, wherein the second set of annotations is determined based, at least in part, on program instructions to: analyze the training model; and map, according to the training model, at least one tone corresponding to each citation in the second set of citations to a case treatment. ### Claim 14 (depends on 13) The computer program product of claim 13, wherein the second set of annotations is further determined based on program instructions to: analyze the training model; and map, according to the training model, the second score of at least one tone corresponding to each citation in the second set of citations to a case treatment. ### Claim 15 (depends on 14) The computer program product of claim 14, wherein a citation is associated with a case treatment if one or more tones corresponding to the citation have second scores above a given threshold. ### Claim 16 (depends on 9) The computer program product of claim 9, further comprising program instructions to: annotate each citation in the second set of citations in the first legal document with a corresponding annotation from the second set of annotations. ### Claim 17 A computer system, the computer system comprising: one or more computer processors; one or more computer readable storage media; computer program instructions; said computer program instructions being stored on said one or more computer readable storage media for execution by said one or more computer processors; and said computer program instructions comprising instructions to: detect a first set of citations to a plurality of legal cases cited in a plurality of legal documents; determine a first set of tones corresponding to each citation in the first set of citations; determine a first score for each tone in the first set of tones; detect a first set of annotations corresponding to the first set of citations, wherein the first set of annotations are case treatments associated with citations to legal cases cited in the first plurality of legal documents; aggregate one or more subsets from the first set of annotations; build a training model, wherein: the training model is built based, at least in part, from at least a first subset of the first set of annotations corresponding to the first set of citations; detect a second set of citations to a plurality of legal cases cited in a first legal document, wherein: the first legal document is not part of the plurality of legal documents; and the second set of citations is devoid of any annotations; determine a second set of tones corresponding to each citation in the second set of citations; determine a second score for each tone in the second set of tones; and determine a second set of annotations corresponding to the second set of citations cited in the first legal document of the second plurality of legal documents, wherein: the second set of annotations are case treatments associated with citations to legal cases cited in the first legal document. ### Claim 18 (depends on 17) The computer system of claim 17, wherein: the first score of each tone in first set of tones is determined based, at least in part, on a level of intensity of a tone corresponding to a citation; and the second score of each tone in second set of tones is determined based, at least in part, on the level of intensity of the tone corresponding to the citation. ### Claim 19 (depends on 17) The computer system of claim 17, wherein the training model is further built based, at least in part, on program instructions to map each annotation in the first subset of the first set of annotations to one or more tones of the first set of tones. ### Claim 20 (depends on 19) The computer system of claim 19, wherein the training model is further built based on program instructions to map each annotation in the first subset of the first set of annotations to each first score corresponding to the one or more tones of the first set of tones.