# US 2021/0240920 A1 · Form template matching to populate forms displayed by client devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240920 A1 - Document ID: US20210240920A1 - Status: Published application - Kind: A1 - Application number: 17/236,075 - Filed: 2021-04-21 - Published: 2021-08-05 - Earliest claimed: 2018-11-28 - Assignees: Citrix Systems, Inc. - Applicants: Citrix Systems, Inc. - Inventors: Jeroen Mattijs Van Rotterdam - Web page: https://patentgrep.com/patent/US20210240920A1 ## CPC - G06F 16/3349: Reuse of stored results of previous queries - G06F 40/174: Form filling; Merging - G06F 16/95: Retrieval from the web - G06F 16/953: Querying, e.g. by the use of web search engines - G06F 40/186: Templates - G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables - G06V 30/418: Document matching, e.g. of document images - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A server includes a memory and a processor to receive from a client device a screenshot of an application page from an application. The application page includes a form requiring data to be filled in by a user of the client device. A form template is extracted from the screenshot, with the extracted form template not including form field values. The extracted form template is compared to a private form template database for a match. The private form template database includes private form templates from different applications, with each private form template having form field values previously filled in for the user. Form field values from a matched private form template are provided to the client device for the client device to populate the form in the screenshot. ## Claims ### Claim 1 A server comprising: a memory and a processor configured to perform the following: receive from a client device a screenshot of an application page from an application, with the application page including a form requiring data to be filled in by a user of the client device, extract a form template from the screenshot, with the extracted form template not including form field values, compare the extracted form template to a private form template database for a match, the private form template database comprising a plurality of private form templates from a plurality of different applications, with each private form template having form field values previously filled in for the user; and providing form field values from a matched private form template to the client device in order for the client device to populate the form in the screenshot. ### Claim 2 (depends on 1) The server according to claim 1 wherein the application providing the form is different from the applications associated with the private form templates in the private form template database. ### Claim 3 (depends on 1) The server according to claim 1 wherein the application providing the form is a web application accessed by the client device. ### Claim 4 (depends on 1) The server according to claim 1 wherein said processor is further configured to: extract labels and form field names from the form; correlate the labels with the form field names; and use the correlated labels and form field names to extract the form template. ### Claim 5 (depends on 4) The server according to claim 4 wherein said processor is further configured to access a public form template database comprising a plurality of public form templates from the plurality of different applications from different users, with each public form template including labels and form field names without form field values; and wherein said processor is further configured to: extract labels and form field names from the form; and match the label and form field names to one of the public form templates, with the matched public form template to be used as the form template extracted from the screenshot. ### Claim 6 (depends on 1) The server according to claim 1 wherein the extracted form template includes labels and form field names without form field values; and wherein the plurality of private form templates from the plurality of different applications include labels, form field names and form field values, with the form field values corresponding to the data previously filled in for the user. ### Claim 7 (depends on 6) The server according to claim 6 wherein the plurality of private form templates from the plurality of different applications include shared labels; and wherein said processor is further configured to group together different form field names corresponding to each shared label, with the compare for a match including the following: match a label from the extracted form template to one of the shared labels; and match the form field name for the label from the extracted form template to one of the form field names in the grouped together different form field names. ### Claim 8 (depends on 1) The server according to claim 1 wherein if more than one private form template matches the extracted form template, then said processor is further configured to determine a match based on a score having been assigned to each of the more than one private form matched templates. ### Claim 9 (depends on 1) The server according to claim 1 wherein if more than one private form template matches the extracted form template, then said processor is further configured to determine a match based on a context of the extracted form template. ### Claim 10 A method for operating a server comprising: receiving from a client device a screenshot of an application page from an application, with the application page including a form requiring data to be filled in by a user of the client device; extracting a form template from the screenshot, with the extracted form template not including form field values; comparing the extracted form template to a private form template database for a match, the private form template database comprising a plurality of private form templates from a plurality of different applications, with each private form template having form field values previously filled in for the user; and providing form field values from a matched private form template to the client device in order for the client device to populate the form in the screenshot. ### Claim 11 (depends on 10) The method according to claim 10 wherein the application providing the form is different from the applications associated with the private form templates in the private form template database. ### Claim 12 (depends on 10) The method according to claim 10 wherein the application providing the form is a web application accessed by the client device. ### Claim 13 (depends on 10) The method according to claim 10 further comprising: extracting labels and form field names from the form; correlating the labels with the form field names; and using the correlated labels and form field names to extract the form template. ### Claim 14 (depends on 13) The method according to claim 13 further comprising: accessing a public form template database comprising a plurality of public form templates from the plurality of different applications from different users, with each public form template including labels and form field names without form field values; extracting labels and form field names from the form; and matching the label and form field names to one of the public form templates, with the matched public form template to be used as the form template extracted from the screenshot. ### Claim 15 (depends on 10) The method according to claim 10 wherein the extracted form template includes labels and form field names without form field values; and wherein the plurality of private form templates from the plurality of different applications include labels, form field names and form field values, with the form field values corresponding to the data previously filled in for the user. ### Claim 16 (depends on 15) The method according to claim 15 wherein the plurality of private form templates from the plurality of different applications include shared labels; and wherein the processor is further configured to group together different form field names corresponding to each shared label, with the compare for a match including the following: matching a label from the extracted form template to one of the shared labels; and matching the form field name for the label from the extracted form template to one of the form field names in the grouped together different form field names. ### Claim 17 (depends on 16) The method according to claim 16 wherein if more than one private form template matches the extracted form template, then further comprising determining a match based on a score having been assigned to each of the more than one private form matched templates. ### Claim 18 (depends on 10) The method according to claim 10 wherein if more than one private form template matches the extracted form template, then further comprising determining a match based on a context of the extracted form template. ### Claim 19 A method for operating a client computing device comprising: displaying an application page from an application, with the application page including a form requiring data to be filled in by the user; generating a screenshot of the application page prior to the form being filled in with the data; transmitting the screenshot of the application page to a server that includes a private form template database comprising a plurality of private form templates from a plurality of different applications, with each private form template having different user data associated therewith previously filled in for a user; receiving, from the server, a matched private form template based on the server comparing a form template extracted from the screenshot to the private form template database for a match; and obtaining form field values from the matched private form template; and populating the displayed form with the form field values from the matched private form template. ### Claim 20 (depends on 19) The method according to claim 19 wherein the application providing the form being displayed by the client device is different from the applications associated with the private form templates in the private form template database.