# US 12,762,191 B2 · Cross-application componentized document generation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,762,191 B2
- Document ID: US12762191B2
- Status: Granted
- Kind: B2
- Application number: 17/836,311
- Filed: 2022-06-09
- Published: 2023-12-14
- Granted: 2026-10-13
- Earliest claimed: 2022-06-09
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Sumit Mehra; Anish Chandran; Mukundan Bhoovaraghavan; Neeraj Kumar Verma; Srinivasa Chaitanya Kumar Reddy Gopireddy; Surabhi Bhatnagar; Soumyadeep Dey
- Web page: https://patentgrep.com/patent/US12762191B2

## CPC

- G06F 16/93: Document management systems
- G06N 20/00: Machine learning
- G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
- G06V 30/413: Classification of content, e.g. text, photographs or tables
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method may include presenting content of an electronic document on a mobile computing device within a mobile version of a computing application; classifying, using a set of machine learning models, by the mobile computing device, the content into a plurality of components; after the classifying, highlighting the plurality of components within the mobile version of the computing application; receiving a user input selecting a component of the plurality of components; and adding, by the mobile computing device, the component to a component data store with a type of the component, the type of the component based on output of the set of machine learning models.

## Claims

### Claim 1

A computer-implemented method comprising:
presenting content of an electronic document on a mobile computing device within a mobile version of a computing application;
classifying, using a first set of machine learning models, by the mobile computing device, the content into a plurality of components;
generating, at the mobile computing device, metadata based on the classifying the content into the plurality of components;
transmitting the metadata to a server having a second set of machine learning models, results of the server taking precedence over the results of the mobile computing device in the metadata;
after the classifying, highlighting the plurality of components within the mobile version of the computing application;
receiving, from the server, a second classifying from the second set of machine learning models of the presented content into a second plurality of components;
updating the highlighting based on the second classifying;
surfacing, within the mobile version of the computing application, an intelligent-copy element that is selectable by a user;
receiving a user selection of the intelligent-copy element;
in response to the selection, presenting on the mobile version of the computing application an overlay slide-up interface that includes selectable options to create a presentation document or a word processing document using one or more selected components;
receiving a user selection of one of the selectable options;
receiving a user input selecting a component of the second plurality of components;
automatically generating, on the mobile computing device and without further user input, a new document of the selected presentation document or word processing document that is pre-populated with the selected component as an object having a type corresponding to the component as classified by the second set of machine learning models; and
adding, by the mobile computing device, the component to a component data store with a type of the component, the type of the component based on an output of the second set of machine learning models.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
overlaying on the presented content, an intelligent copy element;
receiving a selection of the intelligent copy element; and
in response to receiving the selection, performing the highlighting.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the component is a first component and wherein the method further comprises:
receiving a user input selecting a second component of the plurality of components; and
in response to receiving the user input selecting the second component, updating the intelligent copy element to indicate two components were selected.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, further comprising, in further response to receiving the user input selecting the second component:
overlaying on the presented content a set of control elements with respect to the first component and the second component;
receiving a selection of a document creation control element of the set of control elements; and
in response to receiving the selection of the document creation control element;
presenting a set of document types.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, further comprising:
in response to the selection of a document type of the set of document types:
generating a new document of the document type; and
presenting representations of the first component and the second component in a selection interface.

### Claim 6 (depends on 3)

The computer-implemented method of claim 3, wherein the first component is a text element type and the second component is an image component type.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein classifying, using the first set of machine learning models, by the mobile computing device, the presented content into the plurality of components includes:
transforming the presented content into an image file; and
inputting the image file into the first set of machine learning models.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving a selection of one at least a text or an image from the updated highlighting; and
generating a text document or a presentation slide document based on the selection of the text or the image.

### Claim 9

A system comprising:
at least one processor; and
a storage device comprising instructions, which when executed by the at least one processor, configure the at least one processor to perform operations comprising:
presenting content of an electronic document on a mobile computing device within a mobile version of a computing application;
classifying, using a first set of machine learning models, by the mobile computing device, the content into a plurality of components;
generating, at the mobile computing device, metadata based on the classifying the content into the plurality of components;
transmitting the metadata to a server having a second set of machine learning models, results of the server taking precedence over the results of the mobile computing device in the metadata;
after the classifying, highlighting the plurality of components within the mobile version of the computing application;
receiving, from the server, a second classifying from the second set of machine learning models of the presented content into a second plurality of components;
updating the highlighting based on the second classifying;
surfacing, within the mobile version of the computing application, an intelligent-copy element that is selectable by a user;
receiving a user selection of the intelligent-copy element;
in response to the selection, presenting on the mobile version of the computing application an overlay slide-up interface that includes selectable options to create a presentation document or a word processing document using one or more selected components;
receiving a user selection of one of the selectable options;
receiving a user input selecting a component of the second plurality of components;
automatically generating, on the mobile computing device and without further user input, a new document of the selected presentation document or word processing document that is pre-populated with the selected component as an object having a type corresponding to the component as classified by the second set of machine learning models; and
adding, by the mobile computing device, the component to a component data store with a type of the component, the type of the component based on an output of the second set of machine learning models.

### Claim 10 (depends on 9)

The system of claim 9, wherein the storage device further comprises instructions, which when executed by the at least one processor, configure the at least one processor to perform operations comprising:
overlaying on the presented content, an intelligent copy element;
receiving a selection of the intelligent copy element; and
in response to receiving the selection, performing the highlighting.

### Claim 11 (depends on 10)

The system of claim 10, wherein the component is a first component and wherein the storage device further comprises instructions, which when executed by the at least one processor, configure the at least one processor to perform operations comprising:
receiving a user input selecting a second component of the plurality of components; and
in response to receiving the user input selecting the second component, updating the intelligent copy element to indicate two components were selected.

### Claim 12 (depends on 11)

The system of claim 11, wherein the storage device further comprises instructions, which when executed by the at least one processor, configure the at least one processor to perform operations comprising:
in further response to receiving the user input selecting the second component:
overlaying on the presented content a set of control elements with respect to the first component and the second component;
receiving a selection of a document creation control element of the set of control elements; and
in response to receiving the selection of the document creation control element:
presenting a set of document types.

### Claim 13 (depends on 12)

The system of claim 12, wherein the storage device further comprises instructions, which when executed by the at least one processor, configure the at least one processor to perform operations comprising:
in response to the selection of a document type of the set of document types:
generating a new document of the document type; and
presenting representations of the first component and the second component in a selection interface.

### Claim 14 (depends on 11)

The system of claim 11, wherein the first component is a text element type and the second component is an image component type.

### Claim 15 (depends on 9)

The system of claim 9, wherein classifying, using the first set of machine learning models, by the mobile computing device, the presented content into the plurality of components includes:
transforming the presented content into an image file; and
inputting the image file into the first set of machine learning models.

### Claim 16 (depends on 9)

The system of claim 9, wherein the storage device further comprises instructions, which when executed by the at least one processor, configure the at least one processor to perform operations comprising:
receiving a selection of one at least a text or an image from the updated highlighting; and
generating a text document or a presentation slide document based on the selection of the text or the image.

### Claim 17

A computer-readable medium comprising instructions, which when executed by at least one processor, configure the at least one processor to perform operations comprising:
presenting content of an electronic document on a mobile computing device within a mobile version of a computing application;
classifying, using a first set of machine learning models, by the mobile computing device, the content into a plurality of components;
generating, at the mobile computing device, metadata based on the classifying the content into the plurality of components;
transmitting the metadata to a server having a second set of machine learning models, results of the server taking precedence over the results of the mobile computing device in the metadata;
after the classifying, highlighting the plurality of components within the mobile version of the computing application;
receiving, from the server, a second classifying from the second set of machine learning models of the presented content into a second plurality of components;
updating the highlighting based on the second classifying;
surfacing, within the mobile version of the computing application, an intelligent-copy element that is selectable by a user;
receiving a user selection of the intelligent-copy element;
in response to the selection, presenting on the mobile version of the computing application an overlay slide-up interface that includes selectable options to create a presentation document or a word processing document using one or more selected components;
receiving a user selection of one of the selectable options;
receiving a user input selecting a component of the second plurality of components;
automatically generating, on the mobile computing device and without further user input, a new document of the selected presentation document or word processing document that is pre-populated with the selected component as an object having a type corresponding to the component as classified by the second set of machine learning models; and
adding, by the mobile computing device, the component to a component data store with a type of the component, the type of the component based on an output of the second set of machine learning models.

### Claim 18 (depends on 17)

The computer-readable medium of claim 17, wherein the instructions, which when executed by the at least one processor, further configure the at least one processor to perform operations comprising:
overlaying on the presented content, an intelligent copy element;
receiving a selection of the intelligent copy element; and
in response to receiving the selection, performing the highlighting.

### Claim 19 (depends on 18)

The computer-readable medium of claim 18, wherein the component is a first component and wherein the instructions, which when executed by the at least one processor, further configure the at least one processor to perform operations comprising:
receiving a user input selecting a second component of the plurality of components; and
in response to receiving the user input selecting the second component, updating the intelligent copy element to indicate two components were selected.

### Claim 20 (depends on 19)

The computer-readable medium of claim 19, wherein the instructions, which when executed by the at least one processor, further configure the at least one processor to perform operations comprising:
in further response to receiving the user input selecting the second component:
overlaying on the presented content a set of control elements with respect to the first component and the second component;
receiving a selection of a document creation control element of the set of control elements; and
in response to receiving the selection of the document creation control element:
presenting a set of document types.