US 2026/0228333 A1 · RISK AWARE FILTERING FOR INDIRECT PROMPT INJECTION ATTACK DETECTION ACROSS AI SAAS INTEGRATIONS

Abstract

A framework has been created to filter queries for prompt injection attack detection. Application connectors collect data of SaaS interactions. The connectors associate, with the collected data, metadata that identifies the third-party application integrated with a primary SaaS application and an indication of whether collected data is sourced from an external user or an internal user. The filtering service periodically obtains a list of applications and risk scores and determines a risk score of the third-party application with the list. If the risk exceeds a threshold and/or the data origin is indicated as an external user, then the filtering service submits the data to a prompt injection attack detection service. Otherwise, the filtering service filters out the data from being submitted to the prompt injection attack detection service.

Claims20

Dependent claims

A method comprising:

detecting application layer messages communicated between a first software-as-a- service (SaaS) application and a plurality of applications integrated with the SaaS application;
associating metadata with each of the application layer messages, wherein the metadata comprise an identifier of a corresponding one of the plurality of applications and a source attribute;
for each of the application layer messages,
determining a risk score for the one of the plurality of applications identified by the identifier associated with the application layer message;
and
filtering out the application layer message from being submitted to a prompt injection attack detection service based, at least partly, on the risk score of the application identified by the identifier and the source attribute.

3 more elements

A non-transitory, machine-readable medium having program code stored thereon,

the program code comprising instructions to:
based on receipt of a query request from one of a plurality of application connectors, update a tracking structure to track state of the query request, wherein the query request comprises application programming interface (API) traffic data and descriptors;
determine a risk score for an application identified by a first of the descriptors;
determine whether the risk score for the application identified by the first descriptor satisfied a defined risk threshold;
based, at least partly, on a determination that the risk score satisfies the defined risk threshold, query a prompt injection attack detection service with the API traffic data;
based, at least partly, on a determination that the risk score does not satisfy the defined risk threshold, filter out the query request;
and
respond to the query request based on either a verdict from the prompt injection attack detection service or the filtering out.

5 more elements

A system comprising:

a first processor;
and
a first machine-readable medium having filtering service instructions stored thereon that are executable by the first processor to cause a first apparatus to,
based on receipt of a query request from one of a plurality of application connectors, update a tracking structure to track state of the query request, wherein the query request comprises application programming interface (API) traffic data and descriptors;
determine a risk score for an application identified by a first of the descriptors;
determine whether the risk score for the application identified by the first descriptor satisfied a defined risk threshold;
based, at least partly, on a determination that the risk score satisfies the defined risk threshold, query a prompt injection attack detection service with the API traffic data;
based, at least partly, on a determination that the risk score does not satisfy the defined risk threshold, filter out the query request;
and
respond to the query request based on either a verdict from the prompt injection attack detection service or the filtering out.

7 more elements

Dependencies are inferred from the claim text.

Risk aware filtering for indirect prompt injection attack detection across ai saas integrations

Life of this patent

Risk aware filtering for indirect prompt injection attack detection across ai saas integrations

Life of this patent

Abstract

Claims20