Detection of malicious direct memory access device used for direct device assignment

A virtualization computer system, comprising:

• a processor system comprising an input/output memory management unit (IOMMU);
• and
• a computer storage medium that stores computer-executable instructions that are executable by the processor system to at least:
• re-assign a peripheral device from a first guest partition at the virtualization computer system to a host-controlled operating context at the virtualization computer system, wherein the peripheral device is direct memory access (DMA) capable;
• monitor an IOMMU signal source while the peripheral device is assigned to the host-controlled operating context;
• based on monitoring the IOMMU signal source, identify a signal pattern characterizing DMA operations that are initiated by the peripheral device, the signal pattern characterizing at least one of a frequency of memory accesses by the peripheral device, an amount of data being transferred by the peripheral device, or a set of memory locations targeted by the peripheral device;
• determine, using the signal pattern, that the DMA operations initiated by the peripheral device are abnormal;
• identify the peripheral device as malicious based on the DMA operations initiated by the peripheral device being abnormal;
• and
• based on identifying the peripheral device as malicious, preventing assignment of the peripheral device to a second guest partition at the virtualization computer system.

A computer storage medium that stores computer-executable instructions that are executable by a processor system that includes an input/output memory management unit (IOMMU) to at least:

• reassign a peripheral device from a first operating context within a virtualization environment to a second operating context within the virtualization environment, wherein,
• the peripheral device is direct memory access (DMA) capable;
• the first operating context comprises a guest virtual machine (VM);
• and
• the second operating context comprises a root partition or a probationary VM;
• monitor an IOMMU signal source while the peripheral device is assigned to the second operating context;
• based on monitoring the IOMMU signal source, identify a signal pattern characterizing DMA operations that are initiated by the peripheral device, the signal pattern characterizing at least one of a frequency of memory accesses by the peripheral device, an amount of data being transferred by the peripheral device, or a set of memory locations targeted by the peripheral device;
• determine, using the signal pattern and an artificial intelligence or machine learning model, that the DMA operations initiated by the peripheral device are abnormal;
• identify the peripheral device as malicious based on the DMA operations initiated by the peripheral device being abnormal;
• and
• based on identifying the peripheral device as malicious, prevent assignment of the peripheral device to a third operating context within the virtualization environment.