# US 2019/0171874 A1 · Using Machine Learning to Flag Gender Biased Words Within Free-Form Text, Such as Job Descriptions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171874 A1
- Document ID: US20190171874A1
- Status: Published application
- Kind: A1
- Application number: 16/267,788
- Filed: 2019-02-05
- Published: 2019-06-06
- Earliest claimed: 2017-11-02
- Assignees: Sap SE
- Applicants: Sap SE
- Inventors: Weiwei Shen; Manish Tripathi
- Web page: https://patentgrep.com/patent/US20190171874A1

## CPC

- G06F 16/353: into predefined classes
- G06F 18/2155: {characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling}
- G06F 18/2431: Multiple classes
- G06F 40/174: Form filling; Merging
- G06F 40/205: Parsing
- G06F 40/30: Semantic analysis
- G06N 20/00: Machine learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N 20/20: Ensemble learning
- G06V 30/418: Document matching, e.g. of document images
- G06F 16/34: Browsing; Visualisation therefor (browsing or visualisation for clustering or classification G06F16/358)
- G06Q 10/1053: Employment or hiring
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Under one aspect, first user input including free-form text is received in a first graphical user interface (GUI). A classification engine of the computer system incorporating a machine learning model classifies words of the free-form text into a male-biased class, a female-biased class, or a neutral class. At least one of the words is classified into the male-biased class or the female-biased class. At least one of the words classified into the male-biased class or the female-biased class is flagged in the first GUI. Second user input is received in the first GUI including at least one revision to at least one of the words of the free-form text classified into the male-biased class or the female-biased class responsive to the flagging. The revised free-form text is posted to a web site for display in a second GUI.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving, in a first graphical user interface (GUI) forming part of an end-user layer of a computer system, first user input comprising free-form text;
respectively classifying, by a software-based classification engine incorporating a machine learning model and being executed by the computer system, words of the free-form text into a male-biased class, a female-biased class, or a neutral class, at least one of the words being classified into the male-biased class or the female-biased class, the machine learning model being trained using a plurality of historical job descriptions for which a ratio of the gender of the applicants is known;
flagging, in the first GUI, at least one of the words classified into the male-biased class or the female-biased class;
receiving, in the first GUI, second user input comprising at least one revision to at least one of the words of the free-form text classified into the male-biased class or the female-biased class responsive to the flagging; and
posting, by the computer system, the revised free-form text to a web site for display in a second GUI forming part of the end-user layer of the computer system.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating, by a scoring engine of the computer system, a gender bias score for the free-form text based on the classification of the words of the free-form text; and
displaying, in the first GUI, the gender bias score.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
respectively classifying, by the classification engine of the computer system, each word of the revised free-form text into the male-biased class, the female-biased class, or the neutral class, at least one of the words being classified into the male-biased class or the female-biased class;
generating, by the scoring engine of the computer system, a revised gender bias score for the revised free-form text based on the classification of the words of the revised free-form text; and
displaying, in the first GUI, the revised gender bias score.

### Claim 4 (depends on 1)

The method of claim 1, further comprising training the classification engine of the computer system using a corpus of words, the words of the corpus being respectively labeled as male-biased, female-biased, or neutral.

### Claim 5 (depends on 4)

The method of claim 4, wherein a first subset of words of the corpus are labeled as male-biased, female-biased, or neutral using operations comprising:
generating, from a plurality of job descriptions, a first subset of the job descriptions that comprises data regarding the respective genders of applicants to that job, the jobs being within job families;
generating, for each job description of the first subset, a gender ratio based on the data regarding the respective genders of applicants to that job;
selecting, from each job family, a job description that has the highest gender ratio and a job description that has the lowest gender ratio;
generating a second subset of the job descriptions comprising the selected job descriptions of the job families that have the highest gender ratios;
generating a third subset of the job descriptions comprising the selected job descriptions of the job families that have the lowest gender ratios;
comparing frequencies of words of the job descriptions of the second and third subsets of the job descriptions; and
based on the comparing:
labeling at least one word of the job descriptions of the second subset of the job descriptions as being one of male-biased and female-biased, and
labeling at least one word of the job descriptions of the third set of the job descriptions as being the other of male-biased and female-biased.

### Claim 6 (depends on 1)

The method of claim 1, wherein the flagging comprises highlighting the word, changing a color of the word, changing an emphasis of the word, or changing a font of the word.

### Claim 7 (depends on 1)

The method of claim 1, wherein the flagging comprises displaying the word in an area of the first GUI that is separate from the free-form text.

### Claim 8

A computer system comprising:
at least one data processor; and
memory storing instructions which, when executed by the at least one data processor, result in operations comprising:
receiving, in a first graphical user interface (GUI) forming part of an end-user layer of the computer system, first user input comprising free-form text;
respectively classifying, by a software-based classification engine incorporating a machine learning model and being executed by the computer system, words of the free-form text into a male-biased class, a female-biased class, or a neutral class, at least one of the words being classified into the male-biased class or the female-biased class, the machine learning model being trained using a plurality of historical job descriptions for which a ratio of the gender of the applicants is known;
flagging, in the first GUI, at least one of the words classified into the male-biased class or the female-biased class;
receiving, in the first GUI, second user input comprising at least one revision to at least one of the words of the free-form text classified into the male-biased class or the female-biased class responsive to the flagging; and
posting, by the computer system, the revised free-form text to a web site for display in a second GUI forming part of the end-user layer of the computer system.

### Claim 9 (depends on 8)

The computer system of claim 8, the memory further storing instructions which, when executed by the at least one data processor, result in operations comprising:
generating, by a scoring engine of the computer system, a gender bias score for the free-form text based on the classification of the words of the free-form text; and
displaying, in the first GUI, the gender bias score.

### Claim 10 (depends on 9)

The computer system of claim 9, the memory further storing instructions which, when executed by the at least one data processor, result in operations comprising:
respectively classifying, by the classification engine of the computer system, each word of the revised free-form text into the male-biased class, the female-biased class, or the neutral class, at least one of the words being classified into the male-biased class or the female-biased class;
generating, by the scoring engine of the computer system, a revised gender bias score for the revised free-form text based on the classification of the words of the revised free-form text; and
displaying, in the first GUI, the revised gender bias score.

### Claim 11 (depends on 8)

The computer system of claim 8, the memory further storing instructions which, when executed by the at least one data processor, result in operations comprising:
training the classification engine of the computer system using a corpus of words, the words of the corpus being respectively labeled as male-biased, female-biased, or neutral.

### Claim 12 (depends on 11)

The computer system of claim 11, wherein a first subset of words of the corpus are labeled as male-biased, female-biased, or neutral using operations comprising:
generating, from a plurality of job descriptions, a first subset of the job descriptions that comprises data regarding the respective genders of applicants to that job, the jobs being within job families;
generating, for each job description of the first subset, a gender ratio based on the data regarding the respective genders of applicants to that job;
selecting, from each job family, a job description that has the highest gender ratio and a job description that has the lowest gender ratio;
generating a second subset of the job descriptions comprising the selected job descriptions of the job families that have the highest gender ratios;
generating a third subset of the job descriptions comprising the selected job descriptions of the job families that have the lowest gender ratios;
comparing frequencies of words of the job descriptions of the second and third subsets of the job descriptions; and
based on the comparing:
labeling at least one word of the job descriptions of the second subset of the job descriptions as being one of male-biased and female-biased, and
labeling at least one word of the job descriptions of the third set of the job descriptions as being the other of male-biased and female-biased.

### Claim 13 (depends on 8)

The computer system of claim 8, wherein the flagging comprises highlighting the word, changing a color of the word, changing an emphasis of the word, or changing a font of the word.

### Claim 14 (depends on 8)

The computer system of claim 8, wherein the flagging comprises displaying the word in an area of the first GUI that is separate from the free-form text.

### Claim 15

A non-transitory computer-readable medium storing instructions which, when executed by at least one data processor of a computer system, result in operations comprising:
receiving, in a first graphical user interface (GUI) forming part of an end-user layer of the computer system, first user input comprising free-form text;
respectively classifying, by a software-based classification engine incorporating a machine learning model and being executed by the computer system, words of the free-form text into a male-biased class, a female-biased class, or a neutral class, at least one of the words being classified into the male-biased class or the female-biased class, the machine learning model being trained using a plurality of historical job descriptions for which a ratio of the gender of the applicants is known;
flagging, in the first GUI, at least one of the words classified into the male-biased class or the female-biased class;
receiving, in the first GUI, second user input comprising at least one revision to at least one of the words of the free-form text classified into the male-biased class or the female-biased class responsive to the flagging; and
posting, by the computer system, the revised free-form text to a web site for display in a second GUI forming part of the end-user layer of the computer system.

### Claim 16 (depends on 15)

The computer-readable medium of claim 15, further storing instructions which, when executed by the at least one data processor, result in operations comprising:
generating, by a scoring engine of the computer system, a gender bias score for the free-form text based on the classification of the words of the free-form text; and
displaying, in the first GUI, the gender bias score.

### Claim 17 (depends on 16)

The computer-readable medium of claim 16, further storing instructions which, when executed by the at least one data processor, result in operations comprising:
respectively classifying, by the classification engine of the computer system, each word of the revised free-form text into the male-biased class, the female-biased class, or the neutral class, at least one of the words being classified into the male-biased class or the female-biased class;
generating, by the scoring engine of the computer system, a revised gender bias score for the revised free-form text based on the classification of the words of the revised free-form text; and
displaying, in the first GUI, the revised gender bias score.

### Claim 18 (depends on 15)

The computer-readable medium of claim 15, the memory further storing instructions which, when executed by the at least one data processor, result in operations comprising:
training the classification engine of the computer system using a corpus of words, the words of the corpus being respectively labeled as male-biased, female-biased, or neutral.

### Claim 19 (depends on 18)

The computer-readable medium of claim 18, wherein a first subset of words of the corpus are labeled as male-biased, female-biased, or neutral using operations comprising:
generating, from a plurality of job descriptions, a first subset of the job descriptions that comprises data regarding the respective genders of applicants to that job, the jobs being within job families;
generating, for each job description of the first subset, a gender ratio based on the data regarding the respective genders of applicants to that job;
selecting, from each job family, a job description that has the highest gender ratio and a job description that has the lowest gender ratio;
generating a second subset of the job descriptions comprising the selected job descriptions of the job families that have the highest gender ratios;
generating a third subset of the job descriptions comprising the selected job descriptions of the job families that have the lowest gender ratios;
comparing frequencies of words of the job descriptions of the second and third subsets of the job descriptions; and
based on the comparing:
labeling at least one word of the job descriptions of the second subset of the job descriptions as being one of male-biased and female-biased, and
labeling at least one word of the job descriptions of the third set of the job descriptions as being the other of male-biased and female-biased.

### Claim 20 (depends on 15)

The computer-readable medium of claim 15, wherein the flagging comprises highlighting the word, changing a color of the word, changing an emphasis of the word, changing a font of the word, or displaying the word in an area of the first GUI that is separate from the free-form text.
