Autonomous reinforcement learning method of receiver scan schedule control

A method of automatically determining and implementing updates to a scan schedule that dictates a pattern of frequencies to which an electromagnetic (“EM”) signal receiver is tuned and a timing thereof, the method comprising:

• performing by the signal receiver of at least one initial scan of a spectrum of interest according to an initial scan schedule implemented by a controller of the signal receiver;
• and
• performing by the signal receiver of a plurality of subsequent scans of the spectrum of interest according to a series of periodically updated scan schedules that are automatically created during the initial and subsequent scans by an agent instantiated in the controller,
• the agent creating each of the updated scan schedules by applying a schedule update to a preceding scan schedule, said schedule update being determined according to application by the agent of reinforcement learning to EM signal data arising from detections by the signal receiver of the signals of interest;
• said reinforcement learning including:
• estimating an actual degree of scanning success applicable to the preceding scan schedule by applying to the EM signal data a plurality of value scales applicable to a corresponding plurality of reward classes;
• awarding at least one reinforcement to the agent according to a comparison between the actual degree of scanning success and a previously determined expected degree of scanning success applicable to the preceding scan schedule;
• according to said reinforcement, determining the schedule update;
• applying the schedule update to the preceding scan schedule to create the updated scan schedule;
• and
• determining an expected degree of success for the updated scan schedule.

Non-transitory computer readable media comprising:

• software recorded thereupon that when executed by a controller configures the controller to:
• cause an electromagnetic (“EM”) signal receiver to scan a spectrum of interest according to a scan schedule that dictates a pattern of frequencies to which the signal receiver is tuned and a timing thereof;
• and
• implement updates to the scan schedule by causing the signal receiver to perform at least one initial scan of the spectrum of interest according to an initial scan schedule, followed by a plurality of subsequent scans of the spectrum of interest according to a series of periodically updated scan schedules that are automatically created during the initial and subsequent scans by an agent that is instantiated in the controller,
• wherein the executed software configures the agent to create each of the updated scan schedules by applying a schedule update to a preceding scan schedule, said schedule update being determined according to application by the agent of reinforcement learning to EM signal data arising from detections by the signal receiver of the signals of interest;
• said reinforcement learning including:
• estimating an actual degree of scanning success applicable to the preceding scan schedule by applying to the EM signal data a plurality of value scales applicable to a corresponding plurality of reward classes;
• awarding at least one reinforcement to the agent according to a comparison between the actual degree of scanning success and a previously determined expected degree of scanning success applicable to the preceding scan schedule;
• according to said reinforcement, determining the schedule update;
• applying the schedule update to the preceding scan schedule to create the updated scan schedule;
• and
• determining an expected degree of success for the updated scan schedule.