Resource management for machine learning systems and applications
AssigneeNVIDIA Corporation
InventorsDavid Yastremsky, Penn Tasinga
Life of this patent
Estimating the term…
Abstract
Example embodiments described herein relate to a resource scheduling system that incorporates user sentiment analysis to enhance responsiveness in computing environments. The embodiments include a lightweight model for estimating compute requirements based on user prompts and organization-specific inputs, as well as a second model for predicting response times based on hardware allocation. Additionally, it features a sentiment computation model that evaluates user engagement metrics and feedback to generate sentiment scores. A control panel allows system administrators to prioritize resource allocation by adjusting the influence of compute requirements, response times, and user sentiment.
Claims20
A system comprising:
- at least one scheduler processor configured to:
- determine, via a compute estimation model and based at least on a received query including one or more tokens, one or more compute requirements to generate a response to the received query;
- determine, via a response time estimation model and based at least on the received query, a response time for generating a response to the received query;
- generate, via a user sentiment model and based at least on the received query, a user sentiment score indicating, at least, an emotional state associated with the user;
- and
- schedule the response to the received query based at least on the compute requirements, the response time, and the user sentiment score.
A method, comprising:
- determining, via a compute estimation model and based at least on a query, one or more compute requirements for processing the query;
- determining, via a response time estimation model and based at least on the query, a response time for processing the query;
- generating, via a user sentiment model and based at least on the query, a user sentiment score;
- and
- scheduling a response to the query based on at least one of the compute requirements, the response time, or the user sentiment score.
At least one processor comprising one or more logical units to schedule a response to a query based at least on compute requirements for processing the query determined using a first model, response time for processing the query determined using a second model, and a user sentiment score based on the query determined using a third model.
Dependencies are inferred from the claim text.