# US 12,008,322 B2 · Machine learning techniques for semantic processing of structured natural language documents to detect action items > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,322 B2 - Document ID: US12008322B2 - Status: Granted - Kind: B2 - Application number: 17/385,563 - Filed: 2021-07-26 - Published: 2023-01-26 - Granted: 2024-06-11 - Earliest claimed: 2021-07-26 - Assignees: Atlassian Pty Ltd.; Atlassian US, Inc. - Applicants: Atlassian Pty Ltd; Atlassian US, Inc. - Inventors: Karthik Muralidharan; Shashank Prasad Rao; Krishna Sai; Sri Vardhamanan A; Bailur Arjun Kini - Web page: https://patentgrep.com/patent/US12008322B2 ## CPC - G06F 40/30: Semantic analysis - G06F 40/14: Tree-structured documents (parsing G06F40/205; validation G06F40/226) - G06F 40/205: Parsing - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/045: Combinations of networks - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Various embodiments of the present invention provide methods, apparatuses, systems, computing devices, and/or the like that are configured to accurately and concisely generate one or more action item logs of one or more document data objects. For example, certain embodiments of the present invention provide methods, apparatuses, systems, computing devices, and/or the like that are configured to generate an action item log of a document data object comprising one or more semantically complete or incomplete units of text data, by generating content segmentation units, determining action item presence predictions, generating action item sets from each content segmentation unit within a candidate action item subset, aggregating the action item sets to create an action item log, and storing the action item log. ## Claims ### Claim 1 An apparatus for generating action item log user interface data for a webpage that displays content data associated with a document data object, the apparatus comprising at least one processor and at least one memory including program code, the at least one memory and the program code configured to, with the processor, cause the apparatus to at least: parse the content data to identify a plurality of content segmentation units based on one or more of semantically complete unit of text data associated with the content data, semantically incomplete unit of text data associated with the content data, or a content hierarchy associated with the content data; for each content segmentation unit, process the content segmentation unit using an action item classification machine learning model to determine an action item presence prediction for the content segmentation unit, wherein the action item presence prediction for each content segmentation unit comprises one of an affirmative action item presence detection indicating that the content segmentation unit comprises one or more action items or a negative action item presence detection indicating that the content segmentation unit fails to comprise an action item; determine, based on each action item presence prediction, a candidate action item subset of the plurality of content segmentation units; for each content segmentation unit in the candidate action item subset, process the content segmentation unit using an action item extraction machine learning model to generate an action item set for the content segmentation unit, wherein the action item extraction machine learning model comprises: (i) a part-of-speech tagger model that is configured to generate a part-of-speech tag sequence for the content segmentation unit, and (ii) a sequence processing model that is configured to generate the action item set based on the part-of-speech tag sequence; and generate an action item log based on each action item set for the candidate action item subset, wherein the action item log is configured to be used to generate the action item log user interface data for an action item log user interface element, and wherein the action item log user interface element is configured to be displayed to an end user of a computing device. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the plurality of content segmentation units comprise one or more sentences of the document data object. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the document data object is a structured document data object that is associated with a structural scheme, and wherein the plurality of content segmentation units comprise one or more predefined structural elements of the document data object that are determined based on the structural scheme, and wherein the one or more predefined structural elements comprise action item declarations. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the action item classification machine learning model is a bidirectional attention-based text processing machine learning model. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the sequence processing model is a long-short term memory machine learning model. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the sequence processing model is a bidirectional attention-based text processing machine learning model. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the sequence processing model is characterized by one or more action item detection regular expression rules. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein generating the action item log based on each action item set for the candidate action item subset comprises: performing operations of a sequential machine learning model that is configured to process, at each timestep of a plurality of timesteps, the action item set for a content segmentation unit of the plurality of content segmentation units to generate a hidden state for the timestamp; and generating the action item presence prediction based on the hidden state for a terminal timestep of the plurality of timesteps. ### Claim 9 A computer-implemented method for generating action item log user interface data for a webpage that displays content data associated with a document data object, the computer-implemented method comprising: parsing the content data to identify a plurality of content segmentation units based on one or more of semantically complete unit of text data associated with the content data, semantically incomplete unit of text data associated with the content data, or a content hierarchy associated with the content data; for each content segmentation unit, processing the content segmentation unit using an action item classification machine learning model to determine an action item presence prediction for the content segmentation unit, wherein the action item presence prediction for each content segmentation unit comprises one of an affirmative action item presence detection indicating that the content segmentation unit comprises one or more action items or a negative action item presence detection indicating that the content segmentation unit fails to comprise an action item; determining, based on each action item presence prediction, a candidate action item subset of the plurality of content segmentation units; for each content segmentation unit in the candidate action item subset, processing the content segmentation unit using an action item extraction machine learning model to generate an action item set for the content segmentation unit, wherein the action item extraction machine learning model comprises: (i) a part-of-speech tagger model that is configured to generate a part-of-speech tag sequence for the content segmentation unit, and (ii) a sequence processing model that is configured to generate the action item set based on the part-of-speech tag sequence; and generating an action item log based on each action item set for the candidate action item subset, wherein the action item log is configured to be used to generate the action item log user interface data for an action item log user interface element, and wherein the action item log user interface element is configured to be displayed to an end user of a computing device. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, wherein the plurality of content segmentation units comprise one or more sentences of the document data object. ### Claim 11 (depends on 9) The computer-implemented method of claim 9, wherein the document data object is a structured document data object that is associated with a structural scheme, and wherein the plurality of content segmentation units comprise one or more predefined structural elements of the document data object that are determined based on the structural scheme. ### Claim 12 (depends on 9) The computer-implemented method of claim 9, wherein the action item classification machine learning model is a bidirectional attention-based text processing machine learning model. ### Claim 13 (depends on 9) The computer-implemented method of claim 9, wherein the sequence processing model is a long-short term memory machine learning model. ### Claim 14 (depends on 9) The computer-implemented method of claim 9, wherein the sequence processing model is a bidirectional attention-based text processing machine learning model. ### Claim 15 (depends on 9) The computer-implemented method of claim 9, wherein the sequence processing model is characterized by one or more action item detection regular expression rules. ### Claim 16 (depends on 9) The computer-implemented method of claim 9, wherein generating the action item log based on each action item set for the candidate action item subset comprises: performing operations of a sequential machine learning model that is configured to process, at each timestep of a plurality of timesteps, the action item set for a content segmentation unit of the plurality of content segmentation units to generate a hidden state for the timestamp; and generating the action item presence prediction based on the hidden state for a terminal timestep of the plurality of timesteps. ### Claim 17 A computer program product for generating action item log user interface data for a webpage that displays content data associated with a document data object, the computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions configured to: parse the content data to identify a plurality of content segmentation units based on one or more of semantically complete unit of text data associated with the content data, semantically incomplete unit of text data associated with the content data, or a content hierarchy associated with the content data; for each content segmentation unit, process the content segmentation unit using an action item classification machine learning model to determine an action item presence prediction for the content segmentation unit, wherein the action item presence prediction for each content segmentation unit comprises one of an affirmative action item presence detection indicating that the content segmentation unit comprises one or more action items or a negative action item presence detection indicating that the content segmentation unit fails to comprise an action item; determine, based on each action item presence prediction, a candidate action item subset of the plurality of content segmentation units; for each content segmentation unit in the candidate action item subset, process the content segmentation unit using an action item extraction machine learning model to generate an action item set for the content segmentation unit, wherein the action item extraction machine learning model comprises: (i) a part-of-speech tagger model that is configured to generate a part-of-speech tag sequence for the content segmentation unit, and (ii) a sequence processing model that is configured to generate the action item set based on the part-of-speech tag sequence; and generate an action item log based on each action item set for the candidate action item subset, wherein the action item log is configured to be used to generate the action item log user interface data for an action item log user interface element, and wherein the action item log user interface element is configured to be displayed to an end user of a computing device. ### Claim 18 (depends on 17) The computer program product of claim 17, wherein the plurality of content segmentation units comprise one or more sentences of the document data object. ### Claim 19 (depends on 17) The computer program product of claim 17, wherein the document data object is a structured document data object that is associated with a structural scheme, and wherein the plurality of content segmentation units comprise one or more predefined structural elements of the document data object that are determined based on the structural scheme, and wherein the one or more predefined structural elements comprise action item declarations. ### Claim 20 (depends on 17) The computer program product of claim 17, wherein the action item classification machine learning model is a bidirectional attention-based text processing machine learning model.