# US 2025/0094998 A1 · Automated security monitoring of online agent-customer interactions using machine learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094998 A1
- Document ID: US20250094998A1
- Status: Published application
- Kind: A1
- Application number: 18/467,381
- Filed: 2023-09-14
- Published: 2025-03-20
- Earliest claimed: 2023-09-14
- Assignees: Twilio Inc.
- Applicants: Twilio Inc
- Inventors: Aman Srivastava; Nurul Quamar Khan
- Web page: https://patentgrep.com/patent/US20250094998A1

## CPC

- G06Q 30/015: Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
- G06N 20/00: Machine learning
- G06Q 30/016: After-sales
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Techniques and systems are described that perform automated protection of customer data by an interaction center that supports live agent-customer interactivity. The techniques include collecting agent activity data associated with an instance of a live agent-customer interaction. The instance of the live agent-customer interaction includes access by an agent to the customer data. The techniques further include generating one or more machine learning (ML)-readable feature vectors representative of at least one pattern in the agent activity data and processing the one or more ML-readable feature vectors using one or more ML models to generate an indication that the customer data is at risk. The techniques further include causing, responsive to the indication that the customer data is at risk, one or more remedial actions to be performed by the interaction center.

## Claims

### Claim 1

A computing server of an interaction center that supports live agent-customer interactivity, the server comprising:
a memory device storing customer data, and
one or more processing devices communicatively coupled to the memory device, the processing device to:
collect agent activity data associated with an instance of a live agent-customer interaction, wherein the instance of the live agent-customer interaction comprises access by an agent to the customer data;
generate one or more machine learning (ML)-readable feature vectors representative of at least one pattern in the agent activity data;
process the one or more ML-readable feature vectors using one or more ML models to generate an indication that the customer data is at risk; and
cause, responsive to the indication that the customer data is at risk, one or more remedial actions to be performed by the interaction center.

### Claim 2 (depends on 1)

The computing server of claim 1, wherein the one or more ML models comprise a voice recognition ML model, wherein the one or more ML-readable feature vectors comprise a representation of a voice sample of the agent collected during the live agent-customer interaction, and wherein to process the one or more ML-readable feature vectors, the one or more processing devices are to:
determine, using the voice recognition ML model, that the voice sample of the agent collected during the live agent-customer interaction does not match one or more stored voice samples of the agent.

### Claim 3 (depends on 1)

The computing server of claim 1, wherein to process the one or more ML-readable feature vectors, the one or more processing devices are to:
process, using an anomaly detection ML model, the one or more ML-readable feature vectors to detect an anomaly in the agent activity data.

### Claim 4 (depends on 3)

The computing server of claim 3, wherein to process the one or more ML-readable feature vectors, the one or more processing devices are further to:
process, responsive to the detected anomaly in the agent activity data, using an anomaly evaluation ML model, a representation of at least a portion of the agent activity data to generate the indication that the customer data is at risk.

### Claim 5 (depends on 3)

The computing server of claim 3, wherein the anomaly evaluation ML model is trained using a training dataset comprising a training input and a target output, wherein the training input comprises a representation of a training activity data and the target output comprises a classification output indicative of whether the customer data is at risk.

### Claim 6 (depends on 4)

The computing server of claim 4, wherein the anomaly evaluation ML model is pre-trained using one or more initial training datasets prior to a deployment of the anomaly evaluation ML model and re-trained using one more additional training datasets after the deployment of the anomaly evaluation ML model.

### Claim 7 (depends on 1)

The computing server of claim 1, wherein the agent activity data comprises at least one or more of:
a record of the agent accessing the customer data in association with the live agent-customer interaction,
a transcript of the live agent-customer interaction, or
the agent activity data associated with one or more previous agent-customer interactions involving the agent.

### Claim 8 (depends on 1)

The computing server of claim 1, wherein the one or more remedial actions to be performed by the interaction center comprises one or more of:
an authentication request to the agent,
a warning to the agent, or
a warning to a supervisor of the agent.

### Claim 9

A method of automated protection of customer data by an interaction center that supports live agent-customer interactivity, the method comprising:
collecting, by a processing device, agent activity data associated with an instance of a live agent-customer interaction, wherein the instance of the live agent-customer interaction comprises access by an agent to the customer data;
generating, by the processing device, one or more machine learning (ML)-readable feature vectors representative of at least one pattern in the agent activity data;
processing the one or more ML-readable feature vectors using one or more ML models to generate an indication that the customer data is at risk; and
responsive to the indication that the customer data is at risk, causing, by the processing device, one or more remedial actions to be performed by the interaction center.

### Claim 10 (depends on 9)

The method of claim 9, wherein the one or more ML models comprise a voice recognition ML model, wherein the one or more ML-readable feature vectors comprise a representation of a voice sample of the agent collected during the live agent-customer interaction, and wherein processing the one or more ML-readable feature vectors comprises:
determining, using the voice recognition ML model, that the voice sample of the agent collected during the live agent-customer interaction does not match one or more stored voice samples of the agent.

### Claim 11 (depends on 9)

The method of claim 9, wherein processing the one or more ML-readable feature vectors comprises:
processing, using an anomaly detection ML model, the one or more ML-readable feature vectors to detect an anomaly in the agent activity data.

### Claim 12 (depends on 11)

The method of claim 11, wherein processing the one or more ML-readable feature vectors further comprises:
responsive to the detected anomaly in the agent activity data, processing, using an anomaly evaluation ML model, a representation of at least a portion of the agent activity data to generate the indication that the customer data is at risk.

### Claim 13 (depends on 12)

The method of claim 12, wherein the anomaly evaluation ML model is trained using a training dataset comprising a training input and a target output, wherein the training input comprises a representation of a training activity data and the target output comprises a classification output indicative of whether the customer data is at risk.

### Claim 14 (depends on 12)

The method of claim 12, wherein the anomaly evaluation ML model is pre-trained using one or more initial training datasets prior to a deployment of the anomaly evaluation ML model and re-trained using one more additional training datasets after the deployment of the anomaly evaluation ML model.

### Claim 15 (depends on 9)

The method of claim 9, wherein the agent activity data comprises at least one or more of:
a record of the agent accessing the customer data in association with the live agent-customer interaction,
a transcript of the live agent-customer interaction, or
the agent activity data associated with one or more previous agent-customer interactions involving the agent.

### Claim 16 (depends on 9)

The method of claim 9, wherein the one or more remedial actions to be performed by the interaction center comprises one or more of:
an authentication request to the agent,
a warning to the agent, or
a warning to a supervisor of the agent.

### Claim 17

A non-transitory computer-readable storage medium storing instructions that, when executed by a processing device of an interaction center that supports live agent-customer interactivity, cause the processing device to:
collect agent activity data associated with an instance of a live agent-customer interaction, wherein the instance of the live agent-customer interaction comprises access by an agent to a customer data;
generate one or more machine learning (ML)-readable feature vectors representative of at least one pattern in the agent activity data;
process the one or more ML-readable feature vectors using one or more ML models to generate an indication that the customer data is at risk; and
cause, responsive to the indication that the customer data is at risk, one or more remedial actions to be performed by the interaction center.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the one or more ML models comprise a voice recognition ML model, wherein the one or more ML-readable feature vectors comprise a representation of a voice sample of the agent collected during the live agent-customer interaction, and wherein to process the one or more ML-readable feature vectors, the processing device is to:
determine, using the voice recognition ML model, that the voice sample of the agent collected during the live agent-customer interaction does not match one or more stored voice samples of the agent.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, wherein to process the one or more ML-readable feature vectors, the processing device is further to:
process, responsive to the detected anomaly in the agent activity data, and using an anomaly evaluation ML model, a representation of at least a portion of the agent activity data to generate the indication that the customer data is at risk.

### Claim 20 (depends on 12)

The method of claim 12, wherein the anomaly evaluation ML model is trained using a training dataset comprising a training input and a target output, wherein the training input comprises a representation of a training activity data and the target output comprises a classification output indicative of whether the customer data is at risk.
