Using machine learning to flag gender biased words within free-form text, such as job descriptions

A computer system comprising:

• at least one data processor;
• and
• memory storing instructions which, when executed by the at least one data processor, result in operations comprising:
• receiving, in a first graphical user interface (GUI) forming part of an end-user layer of the computer system, first user input comprising free-form text;
• respectively classifying, by the computer system incorporating a machine learning model, words of the free-form text into a male-biased class, a female-biased class, or a neutral class, at least one of the words being classified into the male-biased class or the female-biased class;
• flagging, in the first GUI, at least one of the words classified into the male-biased class or the female-biased class;
• receiving, in the first GUI, second user input comprising at least one revision to at least one of the words of the free-form text classified into the male-biased class or the female-biased class responsive to the flagging;
• and
• posting, by the computer system, the revised free-form text to a web site for display in a second GUI forming part of the end-user layer of the computer system;
• wherein:
• the computer system is trained using a corpus of words;
• the words of the corpus are respectively labeled as male-biased, female-biased, or neutral using operations comprising:
• generating, from a plurality of job descriptions, a first subset of the job descriptions that comprises data regarding the respective genders of applicants to that job, the jobs being within job families;
• generating, for each job description of the first subset, a gender ratio based on the data regarding the respective genders of applicants to that job;
• selecting, from each job family, a job description that has the highest gender ratio and a job description that has the lowest gender ratio;
• generating a second subset of the job descriptions comprising the selected job descriptions of the job families that have the highest gender ratios;
• generating a third subset of the job descriptions comprising the selected job descriptions of the job families that have the lowest gender ratios;
• comparing frequencies of words of the job descriptions of the second and third subsets of the job descriptions;
• and
• based on the comparing:
• labeling at least one word of the job descriptions of the second subset of the job descriptions as being one of male-biased and female-biased, and
• labeling at least one word of the job descriptions of the third set of the job descriptions as being the other of male-biased and female-biased.

A non-transitory computer-readable medium storing instructions which, when executed by at least one data processor of a computer system, result in operations comprising:

• receiving, in a graphical user interface (GUI) forming part of an end-user layer of the computer system, first user input comprising free-form text;
• respectively classifying, by the computer system incorporating a machine learning model, words of the free-form text into a male-biased class, a female-biased class, or a neutral class, at least one of the words being classified into the male-biased class or the female-biased class;
• flagging, in the first GUI, at least one of the words classified into the male-biased class or the female-biased class;
• receiving, in the first GUI, second user input comprising at least one revision to at least one of the words of the free-form text classified into the male-biased class or the female-biased class responsive to the flagging;
• generating, by the computer system, a gender bias score for the free-form text based on the classification of the words of the free-form text;
• and
• posting, by the computer system, the revised free-form text to a web site for display in a second GUI forming part of the end-user layer of the computer system;
• wherein:
• the computer system is trained using a corpus of words;
• the words of the corpus are respectively labeled as male-biased, female-biased, or neutral using operations comprising:
• generating, from a plurality of job descriptions, a first subset of the job descriptions that comprises data regarding the respective genders of applicants to that job, the jobs being within job families;
• generating, for each job description of the first subset, a gender ratio based on the data regarding the respective genders of applicants to that job;
• selecting, from each job family, a job description that has the highest gender ratio and a job description that has the lowest gender ratio;
• generating a second subset of the job descriptions comprising the selected job descriptions of the job families that have the highest gender ratios;
• generating a third subset of the job descriptions comprising the selected job descriptions of the job families that have the lowest gender ratios;
• comparing frequencies of words of the job descriptions of the second and third subsets of the job descriptions;
• and
• based on the comparing:
• labeling at least one word of the job descriptions of the second subset of the job descriptions as being one of male-biased and female-biased, and
• labeling at least one word of the job descriptions of the third set of the job descriptions as being the other of male-biased and female-biased.