# US 11,113,829 B2 · Domain adaptation for analysis of images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,829 B2 - Document ID: US11113829B2 - Status: Granted - Kind: B2 - Application number: 16/545,674 - Filed: 2019-08-20 - Published: 2021-02-25 - Granted: 2021-09-07 - Earliest claimed: 2019-08-20 - Assignees: GM Global Technology Operations LLC; Carnegie Mellon University - Applicants: GM Global Technology Operations LLC; Carnegie Mellon University - Inventors: Jinsong Wang; Yang Zou; Guangyu J. Zou; Vijayakumar Bhagavatula - Web page: https://patentgrep.com/patent/US11113829B2 ## CPC - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 7/50: Depth or shape recovery - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/82: using neural networks - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A system for analyzing images includes a processing device including a receiving module configured to receive an image associated with a target domain, and a domain adaptation module configured to characterize one or more features represented in the received image based on a domain adaptation model. The domain adaptation model is generated using a machine learning algorithm to train the domain adaptation model, and the machine learning algorithm is configured to train the domain adaptation model based on one or more source domain images associated with a source domain, one or more previously acquired images associated with the target domain, and acquired characterization data associated with the target domain. The system also includes an output module configured to output the received image with characterization data identifying one or more features characterized by the domain adaptation module. ## Claims ### Claim 1 A system for analyzing images, comprising: a processor including a receiving module configured to receive an image associated with a target domain; a domain adaptation module configured to characterize one or more features represented in the received image based on a domain adaptation model, the domain adaptation model generated using a machine learning algorithm to train the domain adaptation model, the machine learning algorithm configured to train the domain adaptation model based on one or more source domain images associated with a source domain, one or more previously acquired images associated with the target domain, and acquired characterization data associated with the target domain, the one or more source domain images having one or more source domain classification labels assigned to one or more features of the one or more source domain images, the machine learning algorithm configured to generate a target domain classification label based on comparing the one or more features to the one or more previously acquired images associated with the target domain; and an output module configured to output the received image with characterization data identifying one or more features characterized by the domain adaptation module. ### Claim 2 (depends on 1) The system of claim 1, wherein the source domain includes at least one of a first geographic scenario and a first temporal scenario, and the target domain includes at least one of a second geographic region and a second temporal scenario. ### Claim 3 (depends on 1) The system of claim 1, wherein the source domain is associated with data acquired via a first modality and the target domain is associated with data associated with a second modality. ### Claim 4 (depends on 3) The system of claim 3, wherein the data associated with the first modality includes optical image data, and the data associated with the second modality includes optical image data and ground truth data acquired from another sensor. ### Claim 5 (depends on 1) The system of claim 1, wherein processor is configured to train the domain adaptation model by: acquiring the one or more source domain images and the one or more source domain characterization labels associated with the one or more source domain images, and classifying the one or more features and generating a pseudo label having a first level of confidence based on a first model; and applying the acquired characterization data to the first model to generate an updated model, and generating a second pseudo label having a second level of confidence, the second level of confidence being greater than the first level of confidence. ### Claim 6 (depends on 5) The system of claim 5, wherein the acquired characterization data includes ground truth data acquired from one or more sensors. ### Claim 7 (depends on 5) The system of claim 5, wherein the processor is configured to train the domain adaptation model by: acquiring depth data associated with the target image in the target domain; and applying the acquired depth data to the updated model, and generating a third pseudo label having a third level of confidence that is greater than the second level of confidence. ### Claim 8 (depends on 1) The system of claim 1, wherein the processor is configured to train the domain adaptation module using a multi-task learning algorithm, the multi-task learning algorithm configured perform a first task that includes characterizing the one or more features based on image data, and a second task that includes characterizing the one or more features based on additional data in the target domain. ### Claim 9 (depends on 1) The system of claim 1, wherein the processor is configured to train the domain adaptation model by acquiring the one or more source domain images and the one or more source domain classification labels associated with the one or more source domain images, comparing labeled data points of the one or more source domain images to unlabeled data points of the one or more previously acquired images associated with the target domain, and generating a pseudo label having a level of confidence based on the comparison. ### Claim 10 (depends on 1) The system of claim 1, wherein the machine learning algorithm is a deep learning algorithm configured to cause the processor to generate the domain adaptation model using a neural network. ### Claim 11 A method of analyzing images, comprising: receiving an image associated with a target domain; characterizing one or more features represented in the received image by a domain adaptation module based on a domain adaptation model, the domain adaptation model generated using a machine learning algorithm to train the domain adaptation model, the machine learning algorithm configured to train the domain adaptation model based on one or more source domain images associated with a source domain, one or more previously acquired images associated with the target domain, and acquired characterization data associated with the target domain, the one or more source domain images having one or more source domain classification labels assigned to one or more features of the one or more source domain images, the machine learning algorithm configured to generate a target domain classification label based on comparing the one or more features to the one or more previously acquired images associated with the target domain; and outputting the received image with characterization data identifying one or more features characterized by the domain adaptation module. ### Claim 12 (depends on 11) The method of claim 11, wherein the source domain includes at least one of a first geographic scenario and a first temporal scenario, and the target domain includes at least one of a second geographic region and a second temporal scenario. ### Claim 13 (depends on 11) The method of claim 11, wherein the source domain is associated with data acquired via a first modality and the target domain is associated with data associated with a second modality. ### Claim 14 (depends on 13) The method of claim 13, wherein the data associated with the first modality includes optical image data, and the data associated with the second modality includes optical image data and ground truth data acquired from a sensor. ### Claim 15 (depends on 11) The method of claim 11, further comprising training the domain adaptation model, the training including: acquiring the one or more source domain images and the one or more source domain characterization labels associated with the one or more source domain images, and classifying the one or more features and generating a pseudo label having a first level of confidence based on a first model; and applying the acquired characterization data to the first model to generate an updated model, and generating a second pseudo label having a second level of confidence, the second level of confidence being greater than the first level of confidence. ### Claim 16 (depends on 15) The method of claim 15, wherein the acquired characterization data includes ground truth data acquired from one or more sensors. ### Claim 17 (depends on 15) The method of claim 15, wherein training the domain adaptation model includes: acquiring depth data associated with the target image in the target domain; and applying the acquired depth data to the updated model, and generating a third pseudo label having a third level of confidence that is greater than the second level of confidence. ### Claim 18 (depends on 11) The method of claim 11, further comprising training the domain adaptation model using a multi-task learning algorithm, the multi-task learning algorithm configured perform a first task that includes characterizing the one or more features based on image data, and a second task that includes characterizing the one or more features based on additional data in the target domain. ### Claim 19 (depends on 11) The method of claim 11, wherein the acquired characterization data is selected from at least one of radar data and lidar data. ### Claim 20 (depends on 11) The method of claim 11, wherein the machine learning algorithm is a deep learning algorithm configured to cause the processor to generate the domain adaptation model using a neural network.