US 12,591,418 B2 · Electronic system for developing a machine learning model

Abstract

An electronic system provides visual information to help a development person improve a machine learning model. The visual information includes logic flows, scatter plots, confusion matrices and instances of utterances which are related to poor performance. Guided by the logic flows, the development person is able to trace scatter plots and confusion matrices in order to identify weaknesses of the machine learning model being developed. A user interface help the development person to then specify or indicate a request for additional training data, improved labels or an improved labelling guide. The logic flows, scatter plots and confusion matrices are used iteratively by the development person to repeatedly discover how the machine learning model can be improved and then getting the data to improve the machine learning model. A relatively unskilled person is able to develop a highly effective machine learning model using the logic flows, scatter plots and confusion matrices.

Claims16

Dependent claims

A system for improving a statistical performance of a machine learning model, the system comprising:

one or more processors;
and
one or more memories storing instructions configured to cause the one or more processors to:
display, on a visual debugger screen, a first entity including a first chart and a second entity including a scatter plot, wherein the first chart is a flow chart including instructions for a user;
receive a first user input of the user on the first entity or the second entity and update the visual debugger screen in response to the first user input, wherein the first user input is based on the first chart;
display, on the visual debugger screen, a second chart and a confusion matrix, and
receive a second input, wherein the second input is configured to cause improvement of one or more performance parameters determining the statistical performance of the machine learning model,
wherein the first chart instructs the user to view the scatter plot, and the second chart instructs the user to make a selection of an off-diagonal entry of the confusion matrix with a largest entry, wherein the one or more processors are further configured to present, in response to the selection of the largest entry, first detailed data including a first wrongly-classified task.

5 more elements

A method for improving a statistical performance of a machine learning model, the method comprising:

displaying, on a visual debugger screen, a first entity including a first chart and a second entity including a scatter plot, wherein the first chart is a flow chart including instructions for a user;
receiving a first user input of the user on the first entity or the second entity and updating the visual debugger screen in response to the first user input, wherein the first user input is based on the first chart;
displaying, on the visual debugger screen, a second chart and a confusion matrix;
receiving a second input, wherein the second input is configured to cause improvement of one or more performance parameters determining the statistical performance of the machine learning model, wherein the first chart instructs the user to view the scatter plot, and the second chart instructs the user to make a selection of an off-diagonal entry of the confusion matrix with a largest entry;
and
presenting, in response to the selection of the largest entry, first detailed data including a first wrongly-classified task.

3 more elements

Dependencies are inferred from the claim text.

Electronic system for developing a machine learning model

Life of this patent

Electronic system for developing a machine learning model

Life of this patent

Abstract

Claims16