# US 8,751,231 B1 · Model-driven candidate sorting based on audio cues > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,751,231 B1 - Document ID: US8751231B1 - Status: Granted - Kind: B1 - Application number: 14/183,393 - Filed: 2014-02-18 - Granted: 2014-06-10 - Earliest claimed: 2013-12-09 - Assignees: HireVue, Inc. - Applicants: HireVue, Inc. - Inventors: Loren Larsen; Benjamin Taylor - Web page: https://patentgrep.com/patent/US8751231B1 ## CPC - G06Q 10/1053: Employment or hiring - G06F 16/24578: {using ranking} - G06N 5/04: Inference or reasoning models - G06N 20/00: Machine learning - G10L 25/48: specially adapted for particular use - G10L 25/54: for retrieval - G10L 2015/088: {Word spotting} - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Methods and systems for model-driven candidate sorting based on audio cues for evaluating digital interviews are described. In one embodiment, an audio cue generator identifies utterances in audio data of a digital interview. The utterances each include a group of one or more words spoken by a candidate in the digital interview. The audio cue generator generates audio cues of the digital interview based on the identified utterances. The audio cues are applies to a prediction model to predict an achievement index for the candidate based on the audio cues. The candidate is displayed in a list of candidates based on the achievement index. The list of candidates is sorted according to the candidates' achievement index. ## Claims ### Claim 1 A method comprising: identifying, by an audio cue generator executing on a processing device, utterances in audio data of a digital interview, wherein the utterances each comprise a group of one or more words spoken by a candidate in the digital interview; generating, by the audio cue generator, audio cues of the digital interview based on the identified utterances; applying the audio cues to a prediction model to predict an achievement index for the candidate based on the audio cues; and displaying the candidate in a list of candidates based on the achievement index, wherein the list of candidates is sorted according to the candidates' achievement index. ### Claim 2 (depends on 1) The method of claim 1, further comprising: performing the identifying, generating, and applying for a plurality of additional candidates to predict additional achievement indices for the plurality of additional candidates; and sorting the candidate and the plurality of additional candidates in the list of candidates according to their respective achievement index. ### Claim 3 (depends on 1) The method of claim 1, wherein the identifying the utterances comprises: computing a first moving average using a first moving window of a first number of audio samples of the audio data; computing a second moving average using a second moving window of a second number of the audio samples with gradients to detect peak transitions of the audio samples, wherein the second number is greater than the first number; computing a voice activity indicator of the audio samples; segmenting the audio samples into the utterances using the first moving average, the second moving average and the voice activity indicator, wherein each of the utterances comprise an utterance length; and assigning an utterance identifier to each of the utterances. ### Claim 4 (depends on 3) The method of claim 3, further comprising: identifying, by the audio cue generator, gaps between the utterances in the audio data, wherein each of the gaps comprise a gap length; assigning a gap identifier to each of the gaps; and computing summary statistics on the gaps. ### Claim 5 (depends on 4) The method of claim 4, further comprising: generating an utterance vector comprising the utterance identifiers and corresponding utterance lengths; generating a gap vector comprising the gap identifiers and corresponding gap lengths; and computing a ratio of utterance length to gap length using the utterance vector and the gap vector. ### Claim 6 (depends on 3) The method of claim 3, further comprising: performing a spectral analysis of each of the identified utterances; and performing at least one of single-word or phrase detection or interview fingerprinting based on the spectral analysis. ### Claim 7 (depends on 3) The method of claim 3, further comprising: performing a spectral analysis of each of the identified utterances to generate utterance time-series data; and computing summary statistics for each window of a spectrum, wherein each window is defined by a unit step frequency, wherein the summary statistics comprise at least one of maximum, minimum, median, skew, standard deviation, mode, slope, or kurtosis. ### Claim 8 (depends on 3) The method of claim 3, further comprising: performing a spectral analysis of each of the identified utterances to generate utterance time-series data; generating a spectral signature of each of the identified utterances based on the spectral analysis; and detecting repetition of utterances based on the spectral signatures. ### Claim 9 (depends on 8) The method of claim 8, wherein the detecting the repetition of utterances comprises: building an utterance matrix to represent the digital interview; interpolating an utterance spectral response to fit within a predefined resolution of columns of the utterance matrix to allow vector combination; and computing a Euclidean norm of the utterance matrix to estimate repetition; and generating an interview repetition score based on the Euclidean norm. ### Claim 10 (depends on 3) The method of claim 3, wherein the generating the audio cues further comprises: performing an utterance Power Spectrum Density (PSD) function on the identified utterances data; and computing an utterance repetition metric based on the PSD function. ### Claim 11 (depends on 1) The method of claim 1, wherein the generating the audio cues further comprises: performing Fast Fourier Transform (FFT) variants on a sound spectrum of the audio data for frequency spectral analysis; and generating frequency statistics based on the frequency spectral analysis. ### Claim 12 (depends on 1) The method of claim 1, wherein the generating the audio cues further comprises: performing a spectral analysis of each of the identified utterances to obtain an utterance power spectrum density; and computing utterance power spectrum density performing at least one of single-word or phrase detection or interview fingerprinting based on the spectral analysis. ### Claim 13 (depends on 1) The method of claim 1, further comprising converting the audio data to text, and wherein generating the audio cues further comprises analyzing the text. ### Claim 14 (depends on 13) The method of claim 13, wherein analyzing the text comprises at least one of: calculating a grammar score of the text; calculating a sentiment score of the text, wherein the sentiment score represents a mood of the text based on weights given to each word or groups of words; calculating a count of words on a black list; calculating summary statistics on word lengths of the text or word difficulty of the text; calculating a frequency of filler words of the text; or calculating position-specific word clusters for predictive words. ### Claim 15 A non-transitory computer readable storage medium including instructions that, when executed by a processing device of a computing system, cause the computing system to perform operations comprising: identifying, by the processing device, utterances in audio data of a digital interview, wherein the utterances each comprise a group of one or more words spoken by a candidate in the digital interview; generating, by the processing device, audio cues of the digital interview based on the identified utterances; applying the audio cues to a prediction model to predict an achievement index for the candidate based on the audio cues; and displaying, by the processing device, the candidate in a list of candidates based on the achievement index, wherein the list of candidates is sorted according to the candidates' achievement index. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the operations further comprise: performing the identifying, generating, and applying for a plurality of additional candidates to predict additional achievement indices for the plurality of additional candidates; and sorting the candidate and the plurality of additional candidates in the list of candidates according to their respective achievement index. ### Claim 17 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the identifying the utterances comprises: computing a first moving average using a first moving window of a first number of audio samples of the audio data; computing a second moving average using a second moving window of a second number of the audio samples with gradients to detect peak transitions of the audio samples, wherein the second number is greater than the first number; computing a voice activity indicator of the audio samples; segmenting the audio samples into the utterances using the first moving average, the second moving average and the voice activity indicator, wherein each of the utterances comprise an utterance length; and assigning an utterance identifier to each of the utterances. ### Claim 18 A computing system comprising: a data storage device; and a processing device, coupled to the data storage device, to execute a digital interviewing platform comprising an audio cue generator, wherein the audio cue generator is to: identify utterances in audio data of a digital interview, wherein the utterances each comprise a group of one or more words spoken by a candidate in the digital interview; generate audio cues of the digital interview based on the identified utterances; apply the audio cues to a prediction model to predict an achievement index for the candidate based on the audio cues; and display the candidate in a list of candidates based on the achievement index, wherein the list of candidates is sorted according to the candidates' achievement index. ### Claim 19 (depends on 18) The computing system of claim 18, wherein the audio cue generator is further to: compute a first moving average using a first moving window of a first number of audio samples of the audio data; compute a second moving average using a second moving window of a second number of the audio samples with gradients to detect peak transitions of the audio samples, wherein the second number is greater than the first number; compute a voice activity indicator of the audio samples; segment the audio samples into the utterances using the first moving average, the second moving average and the voice activity indicator, wherein each of the utterances comprise an utterance length; and assign an utterance identifier to each of the utterances. ### Claim 20 (depends on 19) The computing system of claim 19, wherein the audio cue generator is further to: identify gaps between the utterances in the audio data, wherein each of the gaps comprise a gap length; assign a gap identifier to each of the gaps; and compute summary statistics on the gaps.