# US 12,216,758 B2 · Training data protection in artificial intelligence model execution environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,216,758 B2 - Document ID: US12216758B2 - Status: Granted - Kind: B2 - Application number: 17/352,653 - Filed: 2021-06-21 - Published: 2022-12-22 - Granted: 2025-02-04 - Earliest claimed: 2021-06-21 - Assignees: EMC IP Holding Company LLC - Applicants: EMC IP Holding Company LLC - Inventors: Jinpeng Liu; Michael Estrin; Zhen Jia - Web page: https://patentgrep.com/patent/US12216758B2 ## CPC - G06F 21/53: by executing in a restricted environment, e.g. sandbox or secure virtual machine - G06F 21/6245: {Protecting personal data, e.g. for financial or medical purposes} - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/045: Combinations of networks - G06N 3/0499: Feedforward networks - G06N 3/08: Learning methods - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06F 2221/033: Test or assess software - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Techniques for training data protection in an artificial intelligence model execution environment are disclosed. For example, a method comprises executing a first portion of an artificial intelligence model within a trusted execution area of an information processing system and a second portion of the artificial intelligence model within an untrusted execution area of the information processing system, wherein data at least one of obtained and processed in the first portion of the artificial intelligence model is inaccessible to the second portion of the artificial intelligence model. Data obtained in the trusted execution area may comprise one or more data samples in an encrypted form usable to train the artificial intelligence model. ## Claims ### Claim 1 A method, comprising: executing a first portion of an artificial intelligence model within a trusted execution area of an information processing system and a second portion of the artificial intelligence model within an untrusted execution area of the information processing system, wherein data at least one of obtained and processed in the first portion of the artificial intelligence model is inaccessible to the second portion of the artificial intelligence model; wherein a plurality of operations, customizable in accordance with one or more interfaces of a deep learning framework and executed by the information processing system, each comprise one part executable in the trusted execution area and another part executable in the untrusted execution area, and wherein the one part and the other part of each of the plurality of operations communicate via one or more calls between the trusted execution area and the untrusted execution area; wherein the plurality of operations is computed with a feedforward pass prior to calculating gradients and updating parameters of the plurality of operations based on the computed plurality of operations in a backpropagation pass; wherein the plurality of operations comprise: a first operation configured to assign the at least one of obtained and processed data into a first layer of the trusted execution area; a second operation configured to compute a first parameter associated with the feedforward pass, to update the first parameter based on the calculated gradients during the backpropagation pass, and to export the first parameter from the first portion of the artificial intelligence model to the second portion of the artificial intelligence model; a third operation configured to compute a second parameter associated with the feedforward pass, to update the second parameter based on the calculated gradients during the backpropagation pass, and to export the second parameter from the first portion of the artificial intelligence model to the second portion of the artificial intelligence model; and a fourth operation configured to compute an activation function during the feedforward pass and to send its gradient to the third operation during the backpropagation pass; and wherein the information processing system comprises at least one processor and at least one memory storing computer program instructions wherein, when the at least one processor executes the computer program instructions, the information processing system performs the method. ### Claim 2 (depends on 1) The method of claim 1, further comprising obtaining data in the trusted execution area, wherein the obtained data comprises one or more data samples in an encrypted form usable to train the artificial intelligence model. ### Claim 3 (depends on 2) The method of claim 2, further comprising obtaining at least one secret in the trusted execution area, wherein the at least one secret is useable to convert the one or more data samples into a decrypted form within the trusted execution area. ### Claim 4 (depends on 3) The method of claim 3, wherein the artificial intelligence model comprises a network wherein the first portion of the artificial intelligence model comprises a first set of layers of the network and the second portion of the artificial intelligence model comprises a second set of layers of the network. ### Claim 5 (depends on 4) The method of claim 4, wherein the first set of layers comprises initial layers of the network and the second set of layers comprises remaining layers of the network. ### Claim 6 (depends on 4) The method of claim 4, further comprising: processing the one or more data samples in the decrypted form in the first set of layers of the network within the trusted execution area; generating, based on the processing, an output of the trusted execution area; and sending the output to the untrusted execution area wherein, given the output, the one or more data samples in the decrypted form and one or more model parameters generated by the first set of layers of the network are inaccessible to the untrusted execution area. ### Claim 7 (depends on 1) The method of claim 1, wherein the one part of each of the plurality of operations has access to the data at least one of obtained and processed in the first portion of the artificial intelligence model, while the data is inaccessible to the other part of each of the plurality of operations. ### Claim 8 (depends on 1) The method of claim 1, wherein the plurality of operations comprise at least one of an input operation, a matrix multiplication operation, an addition operation, and an activation operation. ### Claim 9 A system, comprising: a trusted execution area; and an untrusted execution area operatively coupled to the trusted execution area, wherein the trusted execution area is configured to execute a first portion of an artificial intelligence model and the untrusted execution area is configured to execute a second portion of the artificial intelligence model, wherein data at least one of obtained and processed in the first portion of the artificial intelligence model is inaccessible to the second portion of the artificial intelligence model; wherein a plurality of operations, customizable in accordance with one or more interfaces of a deep learning framework and executed by the system, each comprise one part executable in the trusted execution area and another part executable in the untrusted execution area, and wherein the one part and the other part of each of the plurality of operations communicate via one or more calls between the trusted execution area and the untrusted execution area; wherein the plurality of operations is computed with a feedforward pass prior to calculating gradients and updating parameters of the plurality of operations based on the computed plurality of operations in a backpropagation pass; and wherein the plurality of operations comprise: a first operation configured to assign the at least one of obtained and processed data into a first layer of the trusted execution area; a second operation configured to compute a first parameter associated with the feedforward pass, to update the first parameter based on the calculated gradients during the backpropagation pass, and to export the first parameter from the first portion of the artificial intelligence model to the second portion of the artificial intelligence model; a third operation configured to compute a second parameter associated with the feedforward pass, to update the second parameter based on the calculated gradients during the backpropagation pass, and to export the second parameter from the first portion of the artificial intelligence model to the second portion of the artificial intelligence model; and a fourth operation configured to compute an activation function during the feedforward pass and to send its gradient to the third operation during the backpropagation pass. ### Claim 10 (depends on 9) The system of claim 9, wherein the trusted execution area is further configured to obtain data, wherein the obtained data comprises one or more data samples in an encrypted form usable to train the artificial intelligence model. ### Claim 11 (depends on 10) The system of claim 10, wherein the trusted execution area is further configured to obtain at least one secret, wherein the at least one secret is useable to convert the one or more data samples into a decrypted form within the trusted execution area. ### Claim 12 (depends on 11) The system of claim 11, wherein the artificial intelligence model comprises a network wherein the first portion of the artificial intelligence model comprises a first set of layers of the network and the second portion of the artificial intelligence model comprises a second set of layers of the network. ### Claim 13 (depends on 12) The system of claim 12, wherein the first set of layers comprises initial layers of the network and the second set of layers comprises remaining layers of the network. ### Claim 14 (depends on 12) The system of claim 12, wherein the trusted execution area is further configured to: process the one or more data samples in the decrypted form in the first set of layers of the network; generate an output based on the processing; and send the output to the untrusted execution area wherein, given the output, the one or more data samples in the decrypted form and one or more model parameters generated by the first set of layers of the network are inaccessible to the untrusted execution area. ### Claim 15 (depends on 9) The system of claim 9, wherein the one part of each of the plurality of operations has access to the data at least one of obtained and processed in the first portion of the artificial intelligence model, while the data is inaccessible to the other part of each of the plurality of operations. ### Claim 16 A computer program product stored on a non-transitory computer-readable medium and comprising machine executable instructions, the machine executable instructions, when executed, causing a processing device to: execute a first portion of an artificial intelligence model within a trusted execution area of an information processing system and a second portion of the artificial intelligence model within an untrusted execution area of the information processing system, wherein data at least one of obtained and processed in the first portion of the artificial intelligence model is inaccessible to the second portion of the artificial intelligence model; wherein a plurality of operations, customizable in accordance with one or more interfaces of a deep learning framework and executed by the information processing system, each comprise one part executable in the trusted execution area and another part executable in the untrusted execution area, and wherein the one part and the other part of each of the plurality of operations communicate via one or more calls between the trusted execution area and the untrusted execution area; wherein the plurality of operations is computed with a feedforward pass prior to calculating gradients and updating parameters of the plurality of operations based on the computed plurality of operations in a backpropagation pass; and wherein the plurality of operations comprise: a first operation configured to assign the at least one of obtained and processed data into a first layer of the trusted execution area; a second operation configured to compute a first parameter associated with the feedforward pass, to update the first parameter based on the calculated gradients during the backpropagation pass, and to export the first parameter from the first portion of the artificial intelligence model to the second portion of the artificial intelligence model; a third operation configured to compute a second parameter associated with the feedforward pass, to update the second parameter based on the calculated gradients during the backpropagation pass, and to export the second parameter from the first portion of the artificial intelligence model to the second portion of the artificial intelligence model; and a fourth operation configured to compute an activation function during the feedforward pass and to send its gradient to the third operation during the backpropagation pass. ### Claim 17 (depends on 16) The computer program product of claim 16, wherein the trusted execution area is further configured to obtain data, wherein the obtained data comprises one or more data samples in an encrypted form usable to train the artificial intelligence model. ### Claim 18 (depends on 9) The system of claim 9, wherein the customizable plurality of operations comprise at least one of an input operation, a matrix multiplication operation, an addition operation, and an activation operation. ### Claim 19 (depends on 16) The computer program product of claim 16, wherein the customizable plurality of operations comprise at least one of an input operation, a matrix multiplication operation, an addition operation, and an activation operation. ### Claim 20 (depends on 16) The computer program product of claim 16, wherein the one part of each of the plurality of operations has access to the data at least one of obtained and processed in the first portion of the artificial intelligence model, while the data is inaccessible to the other part of each of the plurality of operations.