# US 2021/0278957 A1 · Visualization of a Machine Learning Confidence Score > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0278957 A1 - Document ID: US20210278957A1 - Status: Published application - Kind: A1 - Application number: 17/328,387 - Filed: 2021-05-24 - Published: 2021-09-09 - Earliest claimed: 2019-03-12 - Assignees: Bottomline Technologies, Inc. - Applicants: Bottomline Technologies, Inc. - Inventors: John Canneto; Flora Kidani; Anne Baron; Jonathan Hewitt; William Cashman; Michael Marcinelli - Web page: https://patentgrep.com/patent/US20210278957A1 ## CPC - G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34) - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 40/109: Font handling; Temporal or kinetic typography - G06F 40/205: Parsing - G06F 40/216: using statistical methods - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A unique user interface for improving machine learning algorithms is described herein. The user interface comprises an icon with multiple visual indicators displaying the machine learning confidence score. When a mouse hovers over the icon, a set of icons are displayed to accept the teaching user's input. In addition, the words that drove the machine learning confidence score are highlighted with formatting so that the teaching user can understand what drove the machine learning confidence score. ## Claims ### Claim 1 An apparatus for visualization of a machine learning confidence score, the apparatus comprising: a display screen; a special purpose computer electrically connected to the display screen; a large capacity data storage facility with a machine learning training data set; a machine learning model built with the machine learning training data set; a machine learning algorithm programmed to operate on the special purpose computer and to interface with the machine learning model to convert a textual entry into the machine learning confidence score; and a user interface display algorithm programmed to operate on the special purpose computer and display an indication of the machine learning confidence score on the display screen, wherein the indication is a variable icon that varies a count of icon elements on the variable icon depending on a magnitude of the machine learning confidence score. ### Claim 2 (depends on 1) The apparatus of claim 1 wherein the variable icon is a lightbulb icon. ### Claim 3 (depends on 2) The apparatus of claim 2 wherein the icon elements are rays. ### Claim 4 (depends on 3) The apparatus of claim 3 wherein the user interface display algorithm is programmed to display three rays off of the lightbulb icon based on one range of the machine learning confidence score, and one ray off of the lightbulb icon based on a second range of the machine learning confidence score. ### Claim 5 (depends on 1) The apparatus of claim 1 wherein the count of the icon elements is at least three. ### Claim 6 (depends on 1) The apparatus of claim 1 wherein the textual entry is a portion of an invoice. ### Claim 7 (depends on 1) The apparatus of claim 1 wherein a natural language processing algorithm programmed to operate on the special purpose computer and to interface with the user interface display algorithm to convert the textual entry into a table of word stems; and the machine learning algorithm is programmed to operate on the special purpose computer and to interfaces with the natural language processing algorithm to convert the table of the word stems into the machine learning confidence score using the machine learning model. ### Claim 8 A special purpose computer implemented method of visualizing a machine learning confidence score, the method comprising: building a machine learning model with a machine learning training data set; operating a machine learning algorithm using the machine learning model to convert a textual entry into the machine learning confidence score; displaying an indication of the machine learning confidence score on a display screen, wherein the indication is a variable icon; and varying a count of icon elements on the variable icon depending on a magnitude of the machine learning confidence score. ### Claim 9 (depends on 8) The method of claim 8 wherein the variable icon is a lightbulb icon. ### Claim 10 (depends on 9) The method of claim 9 wherein the icon elements are rays. ### Claim 11 (depends on 10) The method of claim 10 further comprising displaying three rays off of the lightbulb icon based on one range of the machine learning confidence score, and one ray off of the lightbulb icon based on a second range of the machine learning confidence score. ### Claim 12 (depends on 8) The method of claim 8 wherein the count of the icon elements is at least three. ### Claim 13 (depends on 8) The method of claim 8 wherein the textual entry is a portion of an invoice. ### Claim 14 A non-transitory machine-readable media programmed to: build a machine learning model with a machine learning training data set; operate a machine learning algorithm using the machine learning model to convert a textual entry into a machine learning confidence score; display an indication of the machine learning confidence score on a display screen, wherein the indication is a variable icon; and vary a count of icon elements on the variable icon depending on a magnitude of the machine learning confidence score. ### Claim 15 (depends on 14) The non-transitory machine-readable media of claim 14 wherein the variable icon is a lightbulb icon. ### Claim 16 (depends on 15) The non-transitory machine-readable media of claim 15 wherein the icon elements are rays. ### Claim 17 (depends on 16) The non-transitory machine-readable media of claim 16 further programmed to display three rays off of the lightbulb icon based on one range of the machine learning confidence score, and one ray off of the lightbulb icon based on a second range of the machine learning confidence score. ### Claim 18 (depends on 14) The non-transitory machine-readable media of claim 14 wherein the count of the icon elements is at least three. ### Claim 19 (depends on 14) The non-transitory machine-readable media of claim 14 wherein the textual entry is a portion of an invoice. ### Claim 20 (depends on 14) The non-transitory machine-readable media of claim 14 further programmed to: parse a narrative section of an invoice into a table of words; convert the words in the table into stems; look up each stem in the machine learning model; place a weight for the stem, as found in the machine learning model, in the table of the words; and determine the machine learning confidence score by averaging the weights for each word in the table of the words.