US 12,748,797 B2Granted

Hybrid approach for automated gap analysis with course competency

AssigneeZayed University

InventorSanaa Kaddoura

Life of this patent

Estimating the term…

Abstract

There is provided a computer-implemented method for automated gap analysis that performs the steps of: generating a job competencies dataset, generating a course competencies dataset; generating a missing competencies dataset based on the job competencies dataset and the course competencies dataset, and outputting a recommended course dataset. The method identifies job competencies missing in course competencies, and recommends courses in which to include the missing competencies. The approach to competency identification comprises either a hybrid approach composed of rule-based matching and/or similarity matching, or a pre-trained large language model (LLM).

Claims17

Dependent claims

A computer-implemented method that, when performed by one or more computing devices, causes the one or more computing devices to perform the steps of:

receiving, with a processor of the one or more computing devices, a user input comprising educational study identification;
generating, with the processor of the one or more computing devices, a job competencies dataset comprising the steps of:
extracting a set of job postings from an online database that corresponds to the educational study identification;
identifying job competencies with a skill extractor module of the one or more computing devices via:
using a preconfigured competencies dictionary to identify job competencies within the set of job postings, wherein the job competencies are identified using rule-based matching or similarity matching;
and
using a pre-trained large language model (LLM) to classify extracted phrases from the set of job postings as job competencies, wherein:
the phrases include information about soft skills, technical skills, and job-specific terms,
structured text, the extracted phrases, and labels are used to fine-tune the pre-trained LLM, wherein the fine-tuning includes:
generating a query describing the fine-tuning task,
merging the structured text, the phrases, and the labels with the query to create the input message required for fine-tuning,
generating a prompt, wherein the prompt includes a system message detailing the pre-trained LLM's behavior, and wherein the prompt and the input message are integrated into the pre-trained LLM to guide the fine-tuning process effectively, and
sending the prompt into the pre-trained LLM to initiate fine-tuning, wherein the prompt is used by the pre-trained LLM to adapt to a specific task described in the query and structured input data;
and the fine-tuning also includes:
dividing the labeled data into training data and validation data, wherein the dividing the labeled data is used to:
electronically assess the pre-trained LLM performance during the fine-tuning, and the efficiency of the fine-tuning process;
outputting the identified job competencies to generate the job competencies dataset;
generating, with the processor of the one or more computing devices, a course competencies dataset comprising the steps of:
identifying a plurality of educational course syllabi that correspond to the educational study identification;
identifying course competencies with the skill extractor module of the one or more computing devices via:
using the preconfigured competencies dictionary to identify course competencies within the educational coursesyllabi, wherein the course competencies are identified using rule-based matching or similarity matching;
and
using a pre-trained large language model (LLM) to classify extracted phrases from the educational course syllabi as course competencies;
outputting the identified course competencies to generate the course competencies dataset;
generating, with the processor of the one or more computing devices, a missing competencies dataset from the job competencies dataset and the course competencies dataset comprising the steps of:
mapping competencies within the course competencies dataset with competencies within the job competencies dataset to generate a list of common competencies;
generating the missing competencies dataset with a gap analysis module of the one or more computing devices;
and
outputting, with the processor of the one or more computing devices, a recommended course dataset comprising the steps of:
generating pairs of embeddings for both the educational course syllabi or course competencies dataset, and the missing competencies dataset;
measuring similarity between the each embedding pair;
outputting at least one educational course syllabus of the educational course syllabi, and corresponding missing competency of the missing competencies dataset to be added to the at least one educational course syllabus of the educational course syllabi with a skill recommendation module of the one or more computing devices, belonging to an embedding pair that has similarity within a predetermined range, as the recommended course dataset for the educational study identification;
and
displaying the output on an interactive dashboard to a user, wherein the output on the interactive dashboard is used to update the educational course syllabi.

31 more elements

A platform for displaying, to a user, a set of recommended competencies for an identified educational course, on an interactive dashboard, the platform comprising:

an input field configured to receive an input by a user into a computing device, the input comprising an educational study identification;
a search module executed by a processor of the computing device to retrieve a set of job postings from an online database that corresponds to the identified educational study, the search module configured to retrieve one or more educational course syllabi that correspond to the educational study identification;
a competencies extraction module executed by a processor of the computing device comprising:
a preconfigured competencies dictionary;
a rule-based matching sub-module configured to receive the set of job postings and the educational course syllabus as input, and use the preconfigured competencies dictionary to identify competencies;
a similarity based matching sub-module configured to receive the set of job postings and the educational course syllabus as input, and use the preconfigured competencies dictionary to identify competencies;
and
a machine learning based competencies extraction sub-module comprising a pre-trained large language model (LLM) to classify extracted phrases, configured to receive the set of job postings and the educational course syllabus as input, wherein:
the phrases include information about soft skills, technical skills, and job-specific terms,
structured text, the extracted phrases, and labels are used to fine-tune the pre-trained LLM, wherein the fine-tuning includes:
generating a query describing the fine-tuning task,
merging the structured text, the phrases, and the labels with the query to create the input message required for fine-tuning,
generating a prompt, wherein the prompt includes a system message detailing the pre- trained LLM's behavior, and wherein the prompt and the input message are integrated into the pre-trained LLM to guide the fine-tuning process effectively, and
sending the prompt into the pre-trained LLM to initiate fine-tuning, wherein the prompt is used by the pre-trained LLM to adapt to a specific task described in the query and structured input data, and the fine-tuning also includes:
dividing the labeled data into training data and validation data, wherein the dividing the labeled data is used to:
electronically assess the pre-trained LLM performance during the fine-tuning, and the efficiency of the fine-tuning process;
wherein each sub-module is configured to output course competencies and job competencies, the competencies extraction module being configured to output the course competencies and the job competencies to generate a course competencies dataset and a job competencies dataset;
a mapping module executed by a processor of the computing device to map competencies within the course competencies dataset with competencies within the job competencies dataset to output a list of common competencies;
a gap analysis module executed by a processor of the computing device to output a missing competencies dataset;
a recommendation module executed by a processor of the computing device to generate embeddings for at least the course syllabi or course competencies dataset, and the missing competencies dataset;
measure similarity between the embeddings;
and output at least one educational course syllabus of the educational course syllabi that has similarity within a predetermined range as a recommended course for inclusion of at least one missing competency of the missing competencies dataset;
and
an interactive dashboard configured to display the outputs of at least one of the mapping module, the gap analysis module, or the recommendation module to the user.

21 more elements

A device, comprising:

memory;
and
a processor, wherein the processor is to:
receive a user input comprising educational study identification;
generate a job competencies dataset comprising the steps of:
extracting a set of job postings from an online database that corresponds to the educational study identification;
identifying job competencies with a skill extractor module via a pre-trained large language model (LLM) to classify extracted phrases from the set of job postings as job competencies, wherein the identifying job competencies with a skill extractor module further includes the pre-trained LLM using an Artificial Intelligence (AI) module to transform the job postings into structured text, phrases, and labels, wherein:
the phrases include information about soft skills, technical skills, and job-specific terms,
the structured text, the phrases, and the labels are used to fine-tune the pre-trained LLM, wherein the fine-tuning includes:
generating a query describing the fine-tuning task,
merging structured text, the phrases, and the labels with the query to create the input message required for fine-tuning,
generating a prompt, wherein the prompt includes a system message detailing the pre-trained LLM's behavior, and wherein the prompt and the input message are integrated into the pre-trained LLM to guide the fine-tuning process effectively, and
sending the prompt into the pre-trained LLM to initiate fine-tuning, wherein the prompt is used by the pre-trained LLM to adapt to a specific task described in the query and structured input data, and the fine-tuning also includes:
dividing the labeled data into training data and validation data, wherein the dividing the labeled data is used to:
electronically assess the pre-trained LLM performance during the fine-tuning, and the efficiency of the fine-tuning process;
and
outputting the identified job competencies to generate the job competencies dataset;
generate a course competencies dataset comprising the steps of:
identifying a plurality of educational course syllabi that correspond to the educational study identification;
identifying course competencies with the skill extractor module of the one or more computing devices via one of:
using the preconfigured competencies dictionary to identify course competencies within the educational course syllabi;
and
using a pre-trained large language model (LLM) to classify extracted phrases from the educational course syllabi as course competencies;
outputting the identified course competencies to generate the course competencies dataset;
generate a missing competencies dataset from the job competencies dataset and the course competencies dataset comprising the steps of:
mapping competencies within the course competencies dataset with competencies within the job competencies dataset to generate a list of common competencies;
and
generating the missing competencies dataset with a gap analysis module;
output a recommended course dataset comprising the steps of:
generating pairs of embeddings for both the educational course syllabi or course competencies dataset, and the missing competencies dataset;
measuring similarity between the each embedding pair;
outputting at least one educational course syllabus of the educational course syllabi, and corresponding missing competency of the missing competencies dataset to be added to the at least one educational course syllabus of the educational course syllabi with a skill recommendation module of the one or more computing devices, belonging to an embedding pair that has similarity within a predetermined range, as the recommended course dataset for the educational study identification;
and
displaying the output on an interactive dashboard to a user, wherein the output on the interactive dashboard is used to update the educational course syllabi.

31 more elements

Dependencies are inferred from the claim text.

patentgrep

Loading patent…

Abstract

A computer-implemented method that, when performed by one or more computing devices, causes the one or more computing devices to perform the steps of:

receiving, with a processor of the one or more computing devices, a user input comprising educational study identification;
generating, with the processor of the one or more computing devices, a job competencies dataset comprising the steps of:
extracting a set of job postings from an online database that corresponds to the educational study identification;
identifying job competencies with a skill extractor module of the one or more computing devices via:
using a preconfigured competencies dictionary to identify job competencies within the set of job postings, wherein the job competencies are identified using rule-based matching or similarity matching;
and
using a pre-trained large language model (LLM) to classify extracted phrases from the set of job postings as job competencies, wherein:
the phrases include information about soft skills, technical skills, and job-specific terms,
structured text, the extracted phrases, and labels are used to fine-tune the pre-trained LLM, wherein the fine-tuning includes:
generating a query describing the fine-tuning task,
merging the structured text, the phrases, and the labels with the query to create the input message required for fine-tuning,
generating a prompt, wherein the prompt includes a system message detailing the pre-trained LLM's behavior, and wherein the prompt and the input message are integrated into the pre-trained LLM to guide the fine-tuning process effectively, and
sending the prompt into the pre-trained LLM to initiate fine-tuning, wherein the prompt is used by the pre-trained LLM to adapt to a specific task described in the query and structured input data;
and the fine-tuning also includes:
dividing the labeled data into training data and validation data, wherein the dividing the labeled data is used to:
electronically assess the pre-trained LLM performance during the fine-tuning, and the efficiency of the fine-tuning process;
outputting the identified job competencies to generate the job competencies dataset;
generating, with the processor of the one or more computing devices, a course competencies dataset comprising the steps of:
identifying a plurality of educational course syllabi that correspond to the educational study identification;
identifying course competencies with the skill extractor module of the one or more computing devices via:
using the preconfigured competencies dictionary to identify course competencies within the educational coursesyllabi, wherein the course competencies are identified using rule-based matching or similarity matching;
and
using a pre-trained large language model (LLM) to classify extracted phrases from the educational course syllabi as course competencies;
outputting the identified course competencies to generate the course competencies dataset;
generating, with the processor of the one or more computing devices, a missing competencies dataset from the job competencies dataset and the course competencies dataset comprising the steps of:
mapping competencies within the course competencies dataset with competencies within the job competencies dataset to generate a list of common competencies;
generating the missing competencies dataset with a gap analysis module of the one or more computing devices;
and
outputting, with the processor of the one or more computing devices, a recommended course dataset comprising the steps of:
generating pairs of embeddings for both the educational course syllabi or course competencies dataset, and the missing competencies dataset;
measuring similarity between the each embedding pair;
outputting at least one educational course syllabus of the educational course syllabi, and corresponding missing competency of the missing competencies dataset to be added to the at least one educational course syllabus of the educational course syllabi with a skill recommendation module of the one or more computing devices, belonging to an embedding pair that has similarity within a predetermined range, as the recommended course dataset for the educational study identification;
and
displaying the output on an interactive dashboard to a user, wherein the output on the interactive dashboard is used to update the educational course syllabi.

31 more elements