Prioritized polling mechanism for efficiently managing distributed queues in content creation pipelines

A non-transitory computer-readable storage medium including instructions that, when executed by a processor, cause the processor to perform the steps of:

• populating a distributed queue with a plurality of job instances, wherein each job instance is tagged with a job type, a constraint type and an associated constraint value, and wherein the constraint type can be associated with one or more rules;
• mapping each constraint type and associated constraint value to a unique correlation identifier value, wherein the unique correlation identifier value represents an aggregate priority of a corresponding job instance;
• responsive to a call invocation to the distributed queue to return job instances associated with a particular job type and a particular constraint type in priority order, performing a filtering operation to dequeue job instances with the particular job type identified in the call invocation and a correlation identifier corresponding to the particular constraint type identified in the call invocation;
• sorting the dequeued job instances based on rules associated with the particular constraint type;
• and
• emitting the dequeued job instances in a sorted order of priority in accordance with the call invocation.

A system comprising:

• a memory storing a content production process orchestrator application;
• and
• a processor coupled to the memory, wherein when executed by the processor, the content production process orchestrator application causes the processor to:
• populate a distributed queue with a plurality of job instances, wherein each job instance is tagged with a job type, a constraint type and an associated constraint value, and wherein the constraint type can be associated with one or more rules;
• mapping each constraint type and associated constraint value to a unique correlation identifier value, wherein the unique correlation identifier value represents an aggregate priority of a corresponding job instance;
• responsive to a call invocation to the distributed queue to return job instances associated with a particular job type and a particular constraint type in priority order, performing a filtering operation to dequeue job instances with the particular job type identified in the call invocation and a correlation identifier corresponding to the particular constraint type identified in the call invocation;
• sorting the dequeued job instances based on rules associated with the particular constraint type;
• and
• emitting the dequeued job instances in a sorted order of priority in accordance with the call invocation.