Learning apparatus, system for generating captured image classification apparatus, apparatus for generating captured image classification apparatus, learning method, and program

A system for generating a captured image classification apparatus, the system comprising:

• a server;
• and
• a plurality of apparatuses, wherein
• the server includes a transmission section that transmits a machine learning model trained by a computer graphics image to each of the plurality of apparatuses, the machine learning model including a feature quantity extraction section and a classification section, the feature quantity extraction section outputting feature quantity data including a feature map corresponding to each of a given plurality of classes in accordance with an input of the computer graphics image, the classification section outputting a result of a process of classifying, into the plurality of classes, an image that is inputted to the feature quantity extraction section in accordance with an input of the feature quantity data, and
• each of the plurality of apparatuses includes
• a reception section that receives the machine learning model transmitted from the server,
• a learning section that executes training of a captured image feature quantity extraction section in accordance with an image captured in an environment of a respective user using a given one of the plurality of apparatuses and with the trained machine learning model, the captured image feature quantity extraction section outputting feature quantity data including a feature map corresponding to each of a given plurality of classes in accordance with an input of the image captured in the environment of the user, and
• a generation section that generates a captured image classification apparatus in accordance with the trained captured image feature quantity extraction section and with the trained classification section included in the machine learning model, the captured image classification apparatus executing a process of classifying the image captured in the environment of the user into the plurality of classes.

An apparatus for generating a captured image classification apparatus, the apparatus comprising:

• a reception section that receives a machine learning model that is trained by a computer graphics image and transmitted from a server, the machine learning model including a feature quantity extraction section and a classification section, the feature quantity extraction section outputting feature quantity data including a feature map corresponding to each of a given plurality of classes in accordance with an input of the computer graphics image, the classification section outputting a result of a process of classifying, into the plurality of classes, an image that is inputted to the feature quantity extraction section in accordance with an input of the feature quantity data;
• a learning section that executes training of a captured image feature quantity extraction section in accordance with an image captured in an environment of a user and with the trained machine learning model, the captured image feature quantity extraction section outputting feature quantity data including a feature map corresponding to each of a given plurality of classes in accordance with an input of the image captured in the environment of the user;
• and
• a generation section that generates a captured image classification apparatus in accordance with the trained captured image feature quantity extraction section and with the trained classification section included in the machine learning model, the captured image classification apparatus executing a process of classifying the image captured in the environment of the user into the plurality of classes.

A learning method for executing training of a classification apparatus that includes a feature quantity extraction section and a classification section, the feature quantity extraction section outputting feature quantity data including a feature map corresponding to each of a given plurality of classes in accordance with an input of an image, the classification section outputting a result of a process of classifying the image into the plurality of classes in accordance with an input of the feature quantity data, the learning method comprising:

• executing training of the feature quantity extraction section and training of the classification section in accordance with a result of comparison between an output generated when the classification section inputs the feature quantity data outputted from the feature quantity extraction section in accordance with an input of a classification training image belonging to a source domain and training data regarding the plurality of classes associated with the classification training image;
• dividing the feature quantity data into a plurality of pieces of partial feature quantity data corresponding to the domain training image including the feature map of one of or a plurality of the classes, the feature quantity data being outputted from the feature quantity extraction section in accordance with an input of a domain training image belonging to either the source domain or a target domain;
• and
• executing training of the feature quantity extraction section in accordance with a result of comparison between an output generated when the partial feature quantity data corresponding to the domain training image is inputted to a domain identification section and data indicating whether the domain training image belongs to the source domain or the target domain, the domain identification section distinguishing between an image belonging to the source domain and an image belonging to the target domain.

A non-transitory, computer-readable storage medium containing a computer program, which when executed by a computer, causes the computer to execute training of a classification apparatus that includes a feature quantity extraction section and a classification section, the feature quantity extraction section outputting feature quantity data including a feature map corresponding to each of a given plurality of classes in accordance with an input of an image, the classification section outputting a result of a process of classifying the image into the plurality of classes in accordance with an input of the feature quantity data, the computer program causing the computer to execute actions, comprising:

• executing training of the feature quantity extraction section and training of the classification section in accordance with a result of comparison between an output generated when the classification section inputs the feature quantity data outputted from the feature quantity extraction section in accordance with an input of a classification training image belonging to a source domain and training data regarding the plurality of classes associated with the classification training image;
• dividing the feature quantity data into a plurality of pieces of partial feature quantity data corresponding to the domain training image including the feature map of one of or a plurality of the classes, the feature quantity data being outputted from the feature quantity extraction section in accordance with an input of a domain training image belonging to either the source domain or a target domain;
• and
• executing training of the feature quantity extraction section in accordance with a result of comparison between an output generated when the partial feature quantity data corresponding to the domain training image is inputted to a domain identification section and data indicating whether the domain training image belongs to the source domain or the target domain, the domain identification section distinguishing between an image belonging to the source domain and an image belonging to the target domain.