# US 12,001,514 B2 · Image classification modeling while maintaining data privacy compliance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,514 B2 - Document ID: US12001514B2 - Status: Granted - Kind: B2 - Application number: 18/047,324 - Filed: 2022-10-18 - Published: 2023-03-02 - Granted: 2024-06-04 - Earliest claimed: 2019-02-15 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Ji Li; Youjun Liu; Amit Srivastava - Web page: https://patentgrep.com/patent/US12001514B2 ## CPC - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06F 18/254: {of classification results, e.g. of results related to same input data} - G06F 21/6218: {to a system of files or objects, e.g. local or distributed file system or database} - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract The present disclosure relates to processing operations that execute image classification training for domain-specific traffic, where training operations are entirely compliant with data privacy regulations and policies. Image classification model training, as described herein, is configured to classify meaningful image categories in domain-specific scenarios where there is unknown data traffic and strict data compliance requirements that result in privacy-limited image data sets. Iterative image classification training satisfies data compliance requirements through a combination of online image classification training and offline image classification training. This results in tuned image recognition classifiers that have improved accuracy and efficiency over general image recognition classifiers when working with domain-specific data traffic. One or more image recognition classifiers are independently trained and tuned to detect an image class for image classification. Training of independent image recognition classifiers is also utilized for training and tuning of deeper learning models for image classification. ## Claims ### Claim 1 A method comprising: executing online training of an image recognition classifier based on a plurality of image content that is accessed through a cloud-based service, wherein the executing the online training comprises: determining that data privacy compliance requirements are satisfied for usage of a first image content of the plurality of image content during a first time period during the online training and not satisfied for usage of the first image content during a second time period during the online training, and using the first image content for the online training during the first time period and not during the second time period; tuning the image recognition classifier based at least in part on results of the online training of the image recognition classifier; and exposing the tuned image recognition classifier for image classification processing during execution of the cloud-based service. ### Claim 2 (depends on 1) The method of claim 1, wherein the determining that the data privacy compliance requirements are satisfied for the first image content comprises continuously detecting that the first image content is being utilized in an active online session of an application. ### Claim 3 (depends on 1) The method of claim 1, further comprising: executing an initial training iteration of the image recognition classifier using one or more public image data sets comprising publicly available images and one or more search engine image data sets comprising image content identified using search keywords in a search engine. ### Claim 4 (depends on 1) The method of claim 1, further comprising: generating labeling for the plurality of image content, wherein the labeling satisfies the data privacy compliance requirements; and executing offline training of the image recognition classifier using the labeling. ### Claim 5 (depends on 1) The method of claim 1, further comprising: generating a chart populated with attributes of the plurality of image content without storing the plurality of image content to comply with the data privacy compliance requirements; and executing offline training of the image recognition classifier using the chart. ### Claim 6 (depends on 1) The method of claim 1, further comprising: selecting and applying device configurations identified to ensure the online training is executed within a specific time period to complete without violating the data privacy compliance requirements. ### Claim 7 (depends on 1) The method of claim 1, further comprising: executing offline training of the image recognition classifier using multiple different image data sets. ### Claim 8 (depends on 1) The method of claim 1, further comprising: executing offline training of the image recognition classifier; determining a precision of the image recognition classifier; and repeating the executing the online training, the executing the offline training, and the determining the precision until the precision exceeds a threshold precision. ### Claim 9 (depends on 1) The method of claim 1, wherein the image recognition classifier is tuned to evaluate domain-specific traffic associated with the cloud-based service. ### Claim 10 A system comprising: at least one processor; and a memory, operatively connected with the at least one processor, having stored thereon instructions that, when executed by the at least one processor, causes the at least one processor to: execute online training of an image recognition classifier based on a plurality of image content that is accessed through a cloud-based service, wherein the instructions to execute the online training comprises further instructions that, when executed by the at least one processor, causes the at least one processor to: determine that data privacy compliance requirements are satisfied for usage of a first image content of the plurality of image content during a first time period during the online training and not satisfied for usage of the first image content during a second time period during the online training, and use the first image content for the online training during the first time period and not during the second time period; tune the image recognition classifier based at least in part on results of the online training of the image recognition classifier; and expose the tuned image recognition classifier for image classification processing during execution of the cloud-based service. ### Claim 11 (depends on 10) The system of claim 10, wherein the instructions to determine that the data privacy compliance requirements are satisfied comprise further instructions that, when executed by the at least one processor, causes the at least one processor to: continuously detect that the first image content is being utilized in an active online session of an application. ### Claim 12 (depends on 10) The system of claim 10, wherein the instructions comprise further instructions that, when executed by the at least one processor, causes the at least one processor to: execute an initial training iteration of the image recognition classifier using one or more public image data sets comprising publicly available images and one or more search engine image data sets comprising image content identified using search keywords in a search engine. ### Claim 13 (depends on 10) The system of claim 10, wherein the instructions comprise further instructions that, when executed by the at least one processor, causes the at least one processor to: generate labeling for the plurality of image content, wherein the labeling satisfies the data privacy compliance requirements; and execute offline training of the image recognition classifier using the labeling. ### Claim 14 (depends on 10) The system of claim 10, wherein the instructions comprise further instructions that, when executed by the at least one processor, causes the at least one processor to: generate a chart populated with attributes of the plurality of image content without storing the plurality of image content to comply with the data privacy compliance requirements; and execute offline training of the image recognition classifier using the chart. ### Claim 15 (depends on 10) The system of claim 10, wherein the instructions comprise further instructions that, when executed by the at least one processor, causes the at least one processor to: select and apply device configurations identified to ensure the online training is executed within a specific time period to complete without violating the data privacy compliance requirements. ### Claim 16 (depends on 10) The system of claim 10, wherein the instructions comprise further instructions that, when executed by the at least one processor, causes the at least one processor to: execute offline training of the image recognition classifier using multiple different image data sets. ### Claim 17 (depends on 10) The system of claim 10, wherein the instructions comprise further instructions that, when executed by the at least one processor, causes the at least one processor to: execute offline training of the image recognition classifier; determine a precision of the image recognition classifier; and repeat the execute the online training, the execute the offline training, and the determine the precision until the precision exceeds a threshold precision. ### Claim 18 (depends on 10) The system of claim 10, wherein the image recognition classifier is tuned to evaluate domain-specific traffic associated with the cloud-based service. ### Claim 19 A computer-readable storage device storing instructions that, upon execution by at least one processor, causes the at least one processor to: execute online training of an image recognition classifier based on a plurality of image content that is accessed through a cloud-based service, wherein the instructions to execute the online training comprise further instructions that, upon execution by the at least one processor, causes the at least one processor to: determine that data privacy compliance requirements are satisfied for usage of a first image content of the plurality of image content during a first time period during the online training and not satisfied for usage of the first image content during a second time period during the online training, and use the first image content for the online training during the first time period and not during the second time period; tune the image recognition classifier based at least in part on results of the online training of the image recognition classifier; and expose the tuned image recognition classifier for image classification processing during execution of the cloud-based service. ### Claim 20 (depends on 19) The computer-readable storage device of claim 19, wherein the instructions to determine that the data privacy compliance requirements are satisfied comprise further instructions that, upon execution by the at least one processor, causes the at least one processor to: continuously detect that the first image content is being utilized in an active online session of an application.